---
title: "Meet react-native-ai: LLMs Running on Mobile, for Real"
description: "Run AI models on-device in React Native apps using the new react-native-ai library, powered by MLC and Vercel AI SDK. Learn how it works and why it matters."
url: https://www.callstack.com/blog/meet-react-native-ai-llms-running-on-mobile-for-real
updated: 2026-10-24T14:25:44.333Z
---

# Meet react-native-ai: LLMs Running on Mobile, for Real

Run AI models on-device in React Native apps using the new react-native-ai library, powered by MLC and Vercel AI SDK. Learn how it works and why it matters.

_By Szymon Rybczak, Software Developer @ Callstack · Published 2025-05-15 · Category: AI_

At the end of April, we announced the release of [react-native-ai](https://github.com/callstackincubator/ai)—a new React Native library that enables LLM execution on-device with React Native apps, powered by [MLC LLM Engine](https://github.com/mlc-ai/mlc-llm). In this blog post, we'll explain our motivation behind this module, our plans, and the challenges we faced during development.

> > Introducing react-native-ai 🔥  
> >  
> > Run LLMs on-device with React Native and Vercel [@aisdk](https://twitter.com/aisdk?ref%5Fsrc=twsrc%5Etfw) 😍  
> >  
> > $ 𝚗𝚙𝚖 𝚒𝚗𝚜𝚝𝚊𝚕𝚕 𝚛𝚎𝚊𝚌𝚝-𝚗𝚊𝚝𝚒𝚟𝚎-𝚊𝚒 [pic.twitter.com/MUGHjXx5vf](https://t.co/MUGHjXx5vf)
> > 
> > — Szymon Rybczak (@SzymonRybczak) [April 30, 2025](https://twitter.com/SzymonRybczak/status/1917540461037621554?ref%5Fsrc=twsrc%5Etfw)

[Post on X](https://twitter.com/SzymonRybczak/status/1917540461037621554)

## From `window.ai` demo to Callstack Incubator: How `react-native-ai` was born

During last summer, I saw a demo of [window.ai](http://window.ai/) running inside the latest Canary builds of Google Chrome, and I thought “Is it possible to run AI on-device?”, and the most important question “How hard is it?”.

After some research, I found out there’s a universal engine to run models **everywhere**_._ This is somewhat similar to the React mental model, where we also have the possibility of running code **everywhere**.

I spent a few long evenings creating the first PoC of running AI on-device with React Native, which I presented at the first edition of [thegeekconf in Berlin](https://www.callstack.com/events/bringing-ai-to-react-native). The demo on stage went great. I also wanted to hack streaming support, but unfortunately, the time between the speaker’s dinner and the next breakfast wasn’t enough 😅

I open-sourced the repo and posted a demo on X, which went pretty well. Then, this topic stayed in the drawer for a few months until I spoke with [Mike Grabowski](https://x.com/grabbou), who wanted to give this project a proper API and a proper release.

So we moved the project under Callstack Incubator, a place for new, exciting (not always fully stable!) stuff.

## The API: Plugging into Vercel AI SDK with just a few lines

We decided to expose API from the native side that is compatible with Vercel AI SDK, which Web-based developers might know well because of its flexibility and simplicity. With literally one-line changes, you can switch between models.

```javascript
import { generateText } from 'ai';

const { text } = await generateText({
    model: openai('gpt-4o-mini'),
    prompt: "why React Native is good choice?"
});
```

And when I learned that AI SDK has a public API for third-party providers, which can plug into `generateText()`, `streamText`, etc. I thought of this kind of API:

```javascript
import { prepareModel, getModel } from 'react-native-ai';
import { streamText } from 'ai';

const modelId = '';

prepareModel(modelId);

const { textStream } = streamText({
	model: getModel(modelId), // Passes reference into AI SDK
	prompt: "why React Native is a good choice?"
});
```

After a few weeks, together with Kewin, who wrote the whole implementation on Android and helped with API design a lot, we turned this API into reality.

[Local LLM Demo on Android](https://www.youtube.com/watch?v=xOgK37YfRpE)

[Local LLM Demo on iOS](https://www.youtube.com/watch?v=C3_zzeirULc)

## Under the hood: MLC LLM powering `react-native-ai` package

Under the hood module is leveraging [MLC LLM](https://github.com/mlc-ai/mlc-llm) universal LLM Deployment Engine with ML Compilation that after some research was the best fit for us!

> We’ll have a dedicated blog post coming soon regarding how MLC LLM engine works under the hood!

## Getting Started: Installing and running models on-device

To start using `react-native-ai` in your projects, you'll need to set up the environment and understand which models to use. Here's what you need to know:

### Installation Process

Setting up `react-native-ai` requires several steps to ensure the native modules are properly linked:

1. Install the package via npm: `npm install react-native-ai`
2. Clone the MLC LLM Engine repository and set the `MLC_LLM_SOURCE_DIR` environment variable
3. Install the `mlc_llm` CLI tool following the official guide
4. Create a `mlc-config.json` file in your project root to specify which models you want to use
5. For Android, set up the required NDK environment variables
6. For iOS, add the "Increased Memory Limit" capability in Xcode and install CocoaPods
7. Run `npx react-native-ai mlc-prepare` to generate all necessary binaries and libraries
8. Add the required polyfills to make Vercel AI SDK work with React Native

> To check what are the latest steps to get module working in your project we highly recommend checking out [Installation](https://github.com/callstackincubator/ai?tab=readme-ov-file#installation) section in README.

### Available Models

The library supports various quantized models that can run efficiently on mobile devices. You can specify these in your `mlc-config.json` file. Some popular options include:

- Mistral-7B-Instruct (various quantization levels)
- Llama-2-7B
- Phi-2
- Gemma-2B

Each model is available in different quantization levels (like q3f16\_1, q4f16\_1) which affect the model size and performance. To get started we recommend using some models from “Model” section in `mlc-ai` organization on [HuggingFace](https://huggingface.co/mlc-ai).

### Usage

After you’re done with package setup, and you chose your perfect model you can use API which we described before to download model from HuggingFace, then load it to the memory and generate (or stream) output from LLM 🔥 To learn more about usage check out [API](https://github.com/callstackincubator/ai?tab=readme-ov-file#api) section in the README file.

## Moving beyond AI SDK and integrating with more model architectures

Just like React allows you to write the code once and then “compiles” it to run on multiple devices, so does MLC with AI. Currently, we focus on integration with the AI SDK because we see great value in it and love using it in our own projects. But since MLC focuses on preparing the ideal runtime for the model, in the future, it should be possible to run other models as well.

Imagine a dedicated runtime for the models you have. Optimized and prepared to get the most out of the device it runs on, with a simple command, preparing the model created in any framework available to run on any device available. This is possible. And even though we know there’s a lot of work ahead, we are excited to start that journey!

## Challenges and things to improve

Currently, the setup is quite complex. It requires a lot of steps to correctly set up linking native static libraries, making everything compile correctly on both platforms, etc.

We’ve created a CLI that improves the experience and validates some required env variables, reads config and calls relevant CLI to properly prepare required pieces. We believe that there’s room for improvements to lower the number of steps required to make things work.

In our backlog, there’s also a place for more guides, tutorials, and content presenting what’s possible, what the limits are, and where it can be leveraged—so stay tuned.

## To wrap things up

There’s a lot of excitement around on-device LLMs, where we can have our “own” AI accessible from devices that we all carry in our pockets. To catch up with the excitement and align with the DevX on the web, we wanted to bring the best API possible.

We hope that this API will be future-proof and will fit the developers' needs. We’re more than happy to hear the feedback and hopefully implement some improvements!

## Thank you

Huge thanks to [Kewin](https://x.com/Q1w1N) for supporting me in development, for creating the whole Android module, and for helping design the user-facing API 🙏 Also thanks to [Jamon](https://x.com/jamonholmgren) for sharing `react-native-ai` package 🫡