The landscape of Large Language Models (LLMs) is shifting rapidly and in 2026 we have seen ever more powerful models released by different companies in a race for supremacy, but is newer always better? We took a step back and looked at the OpenAI models spanning 2024 and 2025 and asked ourselves if each newer model was better than its predecessors. To investigate this question, we built a small evaluation pipeline to assess and visualise how five OpenAI models (GPT-4o, GPT-4.1, GPT-5, GPT-5.1, and GPT-5.2) perform across 27 reasoning tasks.
Analysing the performance of these specific OpenAI models contributes to the broader LLM evaluation discourse, even without including models from other providers. This focused analysis serves as a case study in model evolution, the trade-offs of specialised tuning, and the technical hurdles in automated evaluation.
Rather than trying to analyse and rank every model on the market, we treated this as a case study in the evolution of one model family. Comparing consecutive OpenAI generations under the same evaluation makes it possible to see how the GPT lineage progresses and the task-specific differences they obtain. The results also challenge a simple “newer is always better” narrative.
The pipeline calculates accuracy, precision, recall, and F-score from the raw model responses for every task. The results reveal a clear overall leader but also show why model selection should depend on the work a model will perform.
BIG-Bench Hard
BIG-Bench Hard (https://github.com/suzgunmirac/BIG-Bench-Hard) is a collection of challenging reasoning tasks designed to test capabilities that have traditionally been difficult for language models. The evaluated tasks cover logical deduction, multistep arithmetic, object tracking, language understanding, navigation, causal judgement, date reasoning, and other forms of structured problem-solving. The use of BBG adds value because these tasks focus specifically on areas where LLMs previously failed to outperform an average human rater.
This makes BBH useful for looking beyond a single headline score and examining where each model is genuinely strong.
GPT-5: The Clear Leader
Across the BBH tasks, GPT-5 was the clear overall winner. Designed for advanced reasoning, coding, and multi-step problem solving, GPT-5 achieved the highest mean F-score on 24 of the 27 evaluated task variants, with an average task-level F-score of approximately 0.85.
Its strongest results included (near) perfect scores in
- Object Counting, for correctly identifying quantities of items within a text.
- Multistep Arithmetic for solving complex equations involving addition, subtraction, multiplication, and division
- Web of Lies, for identifying the truth value of Boolean functions expressed in natural language
Figure 1- Web of Lies scores showing the almost perfect results of GPT-5.
GPT-5 also dominated in Hyperbaton (adjective ordering), Salient Translation Error Detection, and Boolean Expressions. It achieved F-scores around 0.97-0.98 on the more demanding five- and seven-object Tracking Shuffled Objects and Logical Deduction tasks.
GPT-5.2 placed second overall with a mean task-level F-score of approximately 0.63 and was the strongest model on Sports Understanding. GPT-5.1 averaged approximately 0.57, GPT-4o approximately 0.56, and GPT-4.1 remained competitive on several individual tasks.
The GPT-5 Models
The GPT-5 family was introduced with distinct purposes, and their benchmark results reflect these specialisations:
- GPT-5: specialised in structured reasoning and precise final-answer selection.
- GPT-5.1: a more conversational version that is less stable overall. While it performed well in Snarks (sarcasm detection task) and Logical Deduction, it struggled with arithmetic and Boolean expressions.
- GPT-5.2: positioned for professional knowledge, it is often the second-best model overall. It showed significant improvements over its predecessors in Formal Fallacies, Sports Understanding, and Disambiguation QA.
The Performance Gap
This work provides empirical evidence that model progress is not always linear: the strongest overall model did not win everywhere, and older models outperform their successors on some tasks.
The Date Understanding task was the clearest outlier (see results in figure 2). On it, GPT-4.1 led with an F-score close to 0.78, followed closely by GPT-4o at approximately 0.77 and GPT-5.2 at approximately 0.73. GPT-5 scored near zero on this task. This is likely due to a systematic formatting mismatch rather than a lack of reasoning ability. That is, the model may have provided the correct reasoning or answers but failed to output them in the exact format required by the evaluation criteria.
Figure 2 – Scores for Date Understanding showing the lacklustre results of GPT-5.
GPT-4o also outperformed the GPT-5 models in Causal Judgement (determining how a person would answer a question about a story). These exceptions reinforce an important point that a newer model is not automatically stronger at every task.
Technical Hurdle: Recall vs Accuracy
Some tasks revealed a gap among precision, recall, and accuracy. Dyck Languages (which asks models to complete correctly nested bracket sequences) was the clearest example.
GPT-5 achieved perfect recall but only around 0.24 accuracy. GPT-4.1 also reached perfect recall while recording an accuracy of approximately 0.02, and GPT-5.2 combined a recall of approximately 0.85 with an accuracy of approximately 0.06. In these cases, the expected symbols often appeared in the response, but the complete sequence did not match the expected output. As seen in the figure 3 recall is relatively high, but accuracy is low.
Figure 3 – Scores obtained for the Dyck Languages dataset for the 5 models.
It points to the model’s understanding of the bracket-matching logic, but it doesn’t produce the output exactly as the scorer requires. The same scoring pattern appeared in tasks such as Temporal Sequences, Logical Deduction, Tracking Shuffled Objects, and Multistep Arithmetic.
This highlights a need for more nuanced evaluation metrics that can distinguish between outputs that are wrong and outputs that are unstructured.
Conclusions
Across this evaluation, GPT-5 was the strongest general-purpose reasoning model by a wide margin, leading on 24 of the 27 tasks. GPT-5.2 was usually the strongest alternative. The GPT-4 models remain competitive in specific areas like causal reasoning and date manipulation. while GPT-5.1, GPT-4o, and GPT-4.1 retained meaningful advantages on selected tasks.
The practical takeaway is straightforward: aggregate benchmark rankings are a useful starting point, but they are not a substitute for task-specific evaluation. The best model for a real application is the one tested on examples that resemble the work it will perform. For developers and users, the choice of model may depend less on newness and more on the specific reasoning demands of the task at hand.
