Small language models: when smaller AI is the smarter choice
What small language models are, where they match bigger models, where they don't, and how businesses use them for speed, cost and privacy.
How mobile apps can run AI directly on the phone, why that helps privacy, speed and offline use, and when cloud AI is still the better choice.
On-device AI runs models directly on a phone or tablet instead of sending data to a server. It keeps personal data on the device, responds instantly and works offline, which suits features like text suggestions, photo enhancement, transcription and smart sorting. Phones have limited memory and battery, so larger tasks still go to the cloud. Many good apps use a hybrid of both.
Modern smartphones contain dedicated chips for machine learning, and both Apple and Google now build AI capabilities into their operating systems. That means many AI features no longer need a round trip to the cloud. For app makers, on-device AI opens new possibilities, especially where privacy, speed or connectivity matter.
A trained model is packaged with the app or downloaded to the device, then runs on the phone's processor, graphics chip or neural engine. Developers use frameworks provided by the platforms, such as Apple's Core ML and related tools on iOS, and Google's options for Android, including ML Kit and on-device model runtimes. Platform-level AI features, where available, can also be accessed through system APIs.
Models must be compact. Techniques such as quantisation, which stores model values with less precision, shrink models so they fit in memory and run efficiently. Our guide to small language models explains why smaller models are often good enough for focused tasks.
Privacy. Personal data, such as photos, messages, health readings or voice, can be processed without leaving the device. This reduces data protection risk and is easier to explain to users.
Speed. No network round trip means near-instant responses, which matters for live camera features and typing suggestions.
Offline use. Features work on planes, underground and in areas with poor signal. This pairs naturally with offline-first app design.
Lower running costs. Processing on the user's device avoids per-request cloud AI fees.
Reliability. Features keep working even if your servers or an AI provider have problems.
Many apps combine both approaches:
For example, a note-taking app might transcribe speech on the device, then offer an optional cloud summary for long recordings.
On-device processing reduces risk, but it doesn't remove obligations. If results are synced to your servers, logged or used for analytics, data protection rules still apply. Explain clearly in your privacy notice what stays on the device.
Build a device test matrix covering recent flagship phones, mid-range models and older devices you still support. Measure how long each AI feature takes, how much memory it uses and whether it affects scrolling or animations. Our guide to app performance optimisation explains the wider performance picture.
Track feature usage, response times, battery impact, crash rates and user feedback across device types. Compare on-device and cloud versions where you offer both, to see which delivers better value for users.
Our mobile app development team builds on-device AI features for iOS and Android with sensible fallbacks. Our AI solutions team selects and optimises models for mobile hardware, and our wider app development service designs hybrid architectures that balance privacy, speed and capability.
Yes. Many AI features, such as text recognition, transcription and suggestions, can run fully on the device and work offline.
Generally yes, because data doesn't need to leave the device. Any data you sync or log afterwards still needs protecting.
Most modern phones support some on-device AI, but capability varies. Older and budget devices may need lighter models or cloud fallbacks.
This guide is general information, not legal, tax or financial advice, and details such as prices, features and rules can change. Product and company names mentioned are trademarks of their respective owners and are used only to identify those products; UK Technology Hub is not affiliated with or endorsed by them.
Tell us about your project and get a free, fixed quote.
Get a free quoteWhat small language models are, where they match bigger models, where they don't, and how businesses use them for speed, cost and privacy.
What offline-first design means, which apps need it, how data syncing and conflict resolution work, and the trade-offs to plan for.
How large language models like the ones behind ChatGPT, Claude and Gemini actually work, what they're good at, where they fall short and how businesses use them.