MaartenGr/BERTopic_Wikipedia

tags:

• —bertopic libraryname: bertopic pipelinetag: text-classification ---

Wikipedia

This is a BERTopic model. BERTopic is a flexible and modular topic modeling framework that allows for the generation of easily interpretable topics from large datasets.

• —Trained on ~1000000 Wikipedia pages (first paragraph of each page).
• —Data was retrieved from: https://huggingface.co/datasets/Cohere/wikipedia-22-12-en-embeddings

Usage

To use this model, please install BERTopic:

pip install -U bertopic
pip install -U safetensors

You can use the model as follows:

from bertopic import BERTopic
topic_model = BERTopic.load("MaartenGr/BERTopic_Wikipedia")

topic_model.get_topic_info()

Topics 2D

The top 50 topics visualized and reduced to 2-dimensional space using cuML's UMAP:

[image]

To generate this image, you can follow along with this tutorial: 

Topic overview

• —Number of topics: 2377
• —Number of training documents: 1000001

<details> <summary>Click here for an overview of all topics.</summary>