Artur Morys-Magiera explored what it actually means to run LLM inference directly on mobile devices in React Native applications. Instead of treating “AI” as a buzzword, he narrowed the focus to LLMs and examined why teams might move inference on-device: reliability without network dependency, stronger privacy guarantees, and lower latency without cloud queues. He also highlighted real user-facing constraints, including model size, disk usage, and hardware variability across iOS and Android devices.

From there, the talk moved into the practical engineering layer: hardware acceleration (GPU, NPU, CPU), runtime fragmentation, debugging challenges with abstraction layers like OpenCL, and real-world performance issues traced to memory layout differences. Artur compared available libraries, explained their tradeoffs, and showed how a unified API approach can simplify integration while still supporting optimizations such as quantization, compilation-time improvements, and model selection based on device capability.

What you’ll walk away with:

• When running LLMs locally improves reliability, privacy, and latency
• How model size and OS-provisioned models impact real user experience
• Why hardware acceleration and device fragmentation shape performance decisions
• The tradeoffs between TF Lite, ONNX, ExecuTorch, MLC, llama.cpp, and Apple-based solutions
• How quantization, compilation optimizations, and unified APIs reduce integration risk

‍