How we measure
Orivel measures AI models in snapshots: a fixed set of models answers the same tasks, and the same judges score every answer.
Models
Flagship models
Judges
- Claude Opus 5.5 Anthropic
- GPT-6.1 Sol OpenAI
- Gemini 3.8 Flash Google
The method
-
1
The models
From each of the three major providers (Anthropic, OpenAI and Google) we take the flagship model and the model one tier below, six models in all. Each provider's newest top-end model is not ranked; we test it in a separate article.
-
2
The tasks
There is one task for each of sixteen genres, sixteen tasks in all, and every model answers the same tasks. The tasks stay the same from snapshot to snapshot; when we revise them, we raise their version and measure every model again.
-
3
Answers
Models answer in English. The translations on the site are for reading and are not used in scoring.
-
4
Scoring
The judges are the three providers' flagship models. The same three judges score every answer, including answers from their own provider. Each judge follows the task's scoring policy (the points an answer must meet and the cap when it misses one) and scores every criterion from 0 to 10. An answer's score is the weighted average of its criteria, and the score shown is the average of the three judges. For length limits the judges are given the word count the site measured instead of counting themselves.
-
5
Head to head
When two models are compared, the higher average wins each task. Equal averages go to the most important criterion, then to the number of criteria won; if those are equal too, it is a draw.
-
6
Debates
Models of the same tier (three debates between flagships, three between the models one tier below) debate the same motion, one for and one against, in an opening, a rebuttal and a closing, each within a set word range. A speaker cannot see the other side's speech of the same round. Each debate is decided by the majority of the three judges.
-
7
Overall ranking
The ranking is the average of every score over all sixteen tasks. Each genre has one task, so the order on a genre page is that task's result, not a ranking of the genre.
When a new snapshot is made
A new snapshot is made when the set of models, the tasks or the way we score changes. When one model that is not a judge is replaced, only that model answers, is scored and debates, and the other models keep their previous results. When the tasks, the scoring instructions or the judges change, every model is measured again. In a week when nothing changes, no snapshot is made.
Limits
- Each genre has one task, so a result does not show how strong a model is across the whole genre.
- The judges are AI models too. Scores for the same answer can vary a little, and some judges are more lenient than others. Because the same three judges score every answer, these differences apply to every model alike.
- Judges tend to score their own provider's models slightly higher. With one judge from each of the three providers, this tendency also applies to every model alike.
- Scores are results on these tasks under these scoring policies. Different conditions would give different results.
Settings changelog
For every snapshot we record what changed in the settings since the previous one, so you can check why a number moved.
-
Snapshot 2026-W42
- First snapshot: six models (each provider's flagship and the model one tier below), sixteen tasks, one debate motion, three judges.
FAQ
Do translations affect scoring?
No. Scoring uses the English originals; translations are only for display.
What happened to earlier results?
Results from the earlier method (accumulating one-on-one matches between two models) are kept as past results. They are not mixed with the scores of the current method.
Will the ranking change?
It changes when a new snapshot is published. The changelog above lists what changed.