Labs / NLP Lab
TF-IDF calculator
Edit a corpus and see term frequency, document frequency, IDF, and TF-IDF for every word.
Theory and worked examples: From text to vectors
TF-IDF lab
Edit the corpus (one document per line). Click any cell to see its calculation. Formula: TF × ln(N / DF), with raw counts for TF.
| d1 | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| d2 | ||||||||||||||||
| d3 | ||||||||||||||||
| d4 | ||||||||||||||||
| IDF | 1.39 | 0.69 | 1.39 | 1.39 | 1.39 | 1.39 | 1.39 | 1.39 | 0.29 | 1.39 | 1.39 | 1.39 | 1.39 | 1.39 | 0.00 | 1.39 |
Cosine similarity between documents (TF-IDF vectors)
| d1 | d2 | d3 | d4 | |
|---|---|---|---|---|
| d1 | 1.00 | 0.02 | 0.00 | 0.09 |
| d2 | 0.02 | 1.00 | 0.00 | 0.01 |
| d3 | 0.00 | 0.00 | 1.00 | 0.00 |
| d4 | 0.09 | 0.01 | 0.00 | 1.00 |
Documents sharing no rare words score 0, even if they mean similar things. Embeddings fix that; see the next lesson.