Building a benchmark for AI-generated React Native code

How well do AI models actually perform when writing React Native code?

Kevin was joined by Lech, Piotr, and Artur from Callstack’s R&D incubator to present React Native Evals. The project is an open-source benchmarking framework designed to evaluate large language models on real development tasks. Instead of measuring generic coding ability, the benchmark focuses on everyday React Native workflows such as animations, navigation, and asynchronous state management.

The project originated during an internal hackathon where the goal was to build a full evaluation framework within a short time. The result includes the benchmarking system itself, a dataset of real development tasks, and a white paper describing the methodology behind it.

How the evaluation framework works

The benchmark simulates a realistic development scenario. Each evaluation starts with a working React Native application and a task prompt that asks the AI to implement or modify a feature.

An AI agent then generates code changes based on the prompt. These changes are evaluated against a set of requirements that define what correct behavior should look like. Rather than enforcing one exact implementation, the framework checks whether the generated solution satisfies functional and architectural constraints.

To account for the non-deterministic nature of LLMs, every task is executed multiple times. The evaluation itself is performed by an LLM acting as a judge, analyzing the generated code and determining whether the requirements were satisfied.

Which models performed best

The benchmark included a wide range of models from different providers, including proprietary APIs and open-weight models.

Claude Opus 4.6 achieved the highest overall score in the benchmark, followed by GPT-5.4 and GPT-5.3 Codex. Performance varied depending on the type of task. Some models handled navigation scenarios well but struggled with animations, which turned out to be the most challenging category overall.

Another notable result was the performance of smaller open-weight models. While they did not reach the highest scores, several of them performed surprisingly well relative to their size and cost. This suggests that locally hosted models may still be useful for certain development workflows.

Why this matters for developers

AI coding assistants are becoming a regular part of the development workflow, but understanding their reliability in real scenarios is still difficult. React Native Evals introduces a transparent way to measure how these models behave when working on real application code.

The benchmark highlights differences in performance across models and task types, helping teams better understand where AI tools perform well and where they still struggle. It also opens the door to more targeted testing of specific frameworks, libraries, and development patterns.

Because the entire project is open source, developers can extend the benchmark with additional tasks. This allows the community to test AI performance across a broader range of real-world React Native use cases.

What’s next for the project

The initial release covers three categories: animations, navigation, and asynchronous state management. Additional categories are already planned to expand the benchmark.

The team also plans to extend the results dashboard with pricing comparisons and additional metrics, giving developers more insight into performance, reliability, and cost when selecting AI models for their workflows.

‍