On-device AI in mobile apps: benefits, limits and use cases
How mobile apps can run AI directly on the phone, why that helps privacy, speed and offline use, and when cloud AI is still the better choice.
What small language models are, where they match bigger models, where they don't, and how businesses use them for speed, cost and privacy.
Small language models are AI models with far fewer parameters than the largest frontier models. They're faster, cheaper and can run on a single server, a laptop or even a phone, which helps with cost and privacy. They do well on focused tasks like classification, extraction and short answers, but larger models still lead on complex reasoning, long documents and nuanced writing.
The headlines usually go to the biggest AI models. But many real business tasks don't need a giant model. Small language models, often called SLMs, can do focused jobs quickly and cheaply, sometimes without sending data to the cloud at all.
There's no strict definition. Generally, small models have from a few hundred million to a few billion parameters, compared with the much larger frontier models from leading AI labs. Many are released with open weights, meaning businesses can download and run them. Major providers also offer smaller, faster versions of their flagship models through their APIs.
Classification. Sorting emails, tickets or reviews into categories.
Extraction. Pulling names, dates, amounts and other fields from text.
Short, structured answers. Filling in templates, tagging content, simple Q&A from provided text.
Routing. Deciding which larger model, tool or team should handle a request.
On-device features. Text suggestions, summaries and smart replies inside apps, working offline. Our guide to on-device AI in mobile apps explores this.
High-volume pipelines. Processing thousands of items where cost per item matters.
Using an API for a small model is simplest: no infrastructure to manage, pay per use.
Self-hosting an open-weight model gives control over data and can be cheaper at high volume, but needs suitable hardware, often GPUs, plus engineering for deployment, scaling and security. Check each model's licence terms, as some open-weight models have usage restrictions. Our comparison of open-source and proprietary AI covers licensing in more detail.
Fine-tuning trains a model further on your own examples. For narrow, repetitive tasks with consistent formats, a fine-tuned small model can match or beat a much larger general model. You'll need good quality examples and a way to evaluate results.
Because small models are more sensitive to unusual inputs, testing matters even more. Include messy, real-world examples in your test set, such as typos, mixed languages and incomplete information. Our guide to AI evaluation and testing explains how to build a test set and track accuracy over time. If you're building for phones, read about on-device AI in mobile apps as well.
Our AI solutions team benchmarks models against your real data and picks the right size for each task. Our mobile app team builds on-device AI features, and our web app developers deploy self-hosted or API-based models into your software.
For focused tasks like classification, extraction and short answers, often yes. Test them on your own examples to be sure.
Yes. Many modern phones can run compact models for on-device features, and phone makers increasingly include AI capabilities.
It depends on the licence. Some are permissive; others have restrictions. Always check the licence terms.
This guide is general information, not legal, tax or financial advice, and details such as prices, features and rules can change. Product and company names mentioned are trademarks of their respective owners and are used only to identify those products; UK Technology Hub is not affiliated with or endorsed by them.
Tell us about your project and get a free, fixed quote.
Get a free quoteHow mobile apps can run AI directly on the phone, why that helps privacy, speed and offline use, and when cloud AI is still the better choice.
Practical ways to reduce the cost of AI features without hurting quality, from model choice and prompt design to caching, batching and monitoring.
How large language models like the ones behind ChatGPT, Claude and Gemini actually work, what they're good at, where they fall short and how businesses use them.