Switch model
This model has no "Inside the model" view, so it opens here. That tab shows the page's own model.
Banking77 intent classifier
A 23 MB model distilled from a large language model with tinydistill. It runs in your browser, and nothing you type leaves this page. Test accuracy 0.872 on 3080 held-out texts. Distilled from Qwen/Qwen2.5-7B-Instruct (16 similar examples, 4-bit, probe) into sentence-transformers/all-MiniLM-L6-v2.
| Teacher (LLM) | Student (this page) | |
|---|---|---|
| Model | Qwen/Qwen2.5-7B-Instruct (16 similar examples, 4-bit, probe) | sentence-transformers/all-MiniLM-L6-v2 |
| Size | 5.4 GB | 23 MB |
| Time per text | 155 ms on a GPU | measured as you type |
| Test accuracy | 85.9% | 87.2% |
From the run's results on its held-out test texts; the student's time is measured in your browser.
The answer updates as you type.
Loading the model...
See inside the model on this textEvery layer, every attention head and the math, as the model reads it, in your browser.
Guided tour See insideCheck it yourself
All 77 labels
How does it work?
Every step below is computed by the model in this page as it classifies your text. "Show the steps" plays them in order; slow motion plays them slowly every time.
The tokenizer splits the text
The model never sees letters or words, only pieces from a fixed vocabulary of 30,522 pieces. Rare words are split into several pieces (marked with a blue edge); hover a piece for its number. [CLS] is added at the start: the classifier reads the model's output at that position.
Layer by layer
The pieces pass through 6 transformer layers. Each layer lets every piece look at the others and update itself. If we stop after a layer and apply the final classifier, we see what the model would answer at that depth: early layers usually do not know yet, and the answer forms on the way up.
Where the classifier looks
Attention: how strongly the [CLS] position draws on each piece, averaged over the layer's attention heads. Darker means more. Pick a layer. [CLS] and [SEP] often collect much of the attention as a resting place, so compare the word pieces with each other.
From scores to probabilities
The classifier gives one score per label (77 labels) and a softmax turns them into probabilities. The top five are below; the strip shows all of them, largest first. The small probabilities on other labels are what distillation passes on: the teacher's soft labels tell the student which labels are similar, not just which one is right.
Which words matter
Each word is removed in turn and the model runs again on what is left. Words whose removal lowers the probability of support it; words whose removal raises it count against it. Hover a word for the change.
supports the answer counts against itTeacher and student
The student learned to imitate the soft labels of the teacher, Qwen/Qwen2.5-7B-Instruct (16 similar examples, 4-bit, probe). The teacher is far too large to run here, but its saved soft labels for the test texts are in this page.
When to ask the teacher
In a cascade the student answers and passes the texts it is least sure about to the teacher. The slider sets how many; accuracy and the confidence threshold are measured on the held-out test set.