Multimodal AI: how text, image, audio and video AI work together
What multimodal AI models can do with images, documents, audio and video, real business uses, and what to watch for when building with them.
How voice AI works, what it's good for inside apps and websites, design principles for natural conversations, and the privacy and consent rules to respect.
Voice AI assistants combine speech recognition, a language model and speech synthesis so people can talk to an app or website naturally. They're useful for hands-busy situations, accessibility, field work, booking and support. Good voice experiences are fast, short and forgiving of mistakes. Be clear that users are talking to AI, get consent for recording and protect voice data, which can be sensitive.
Voice is the most natural interface people have. Advances in AI mean apps and websites can now hold flowing spoken conversations, not just respond to fixed commands. Here's how voice assistants work and how to build one people actually enjoy using.
A voice assistant typically has three stages:
Some newer systems process speech more directly, reducing delay and capturing tone. Our guide to multimodal AI explains how models handle audio alongside text and images.
Voice is less suited to complex comparisons, long lists or anything that's easier to scan visually.
Be fast. Delays feel much longer in conversation than on screen. Aim for responses that start within a second or so, and stream speech as it's generated.
Keep replies short. People can't skim audio. Offer the key answer first, then ask if they want more.
Handle interruptions. Users should be able to cut in, as they would with a person.
Confirm important actions. "Booking a table for four at 7 p.m. on Friday. Shall I confirm?"
Recover gracefully. When the assistant mishears, ask a clarifying question rather than guessing.
Offer a screen fallback in apps, showing text or options alongside speech.
Use a consistent voice that fits your brand.
Build a set of real spoken requests, including unclear, noisy and unexpected ones, and measure recognition accuracy, task completion and response time. Our guide to AI evaluation and testing explains how to structure test sets and track quality over time.
Voice recordings are personal data, and voice characteristics can be treated as biometric data in some jurisdictions. Good practice:
Voice should complement, not replace, other ways to use your product. Provide text alternatives and make sure everything voice can do is also possible by touch or keyboard.
If your users speak several languages, choose speech and language models that support them well, and let users pick their language rather than guessing. Test pronunciation of names, places and product terms in each language, and keep replies equally short and clear across all of them.
Track task completion rate, average conversation length, fallback rate (how often the assistant fails), user ratings and drop-off points. Listen to or read samples of conversations, with appropriate consent, to find patterns.
Our AI solutions team builds voice assistants connected to your systems, with fast responses and clear safeguards. Our mobile app team adds voice features to iOS and Android apps, and our app development service designs voice and screen experiences that work together.
Modern speech recognition handles many accents well, but accuracy varies. Test with speakers representative of your users.
Costs depend on minutes of speech and model choice. For many apps, they're modest, but high-volume phone use needs careful budgeting.
It's best practice and required by law in some places. Being transparent also builds trust.
This guide is general information, not legal, tax or financial advice, and details such as prices, features and rules can change. Product and company names mentioned are trademarks of their respective owners and are used only to identify those products; UK Technology Hub is not affiliated with or endorsed by them.
Tell us about your project and get a free, fixed quote.
Get a free quoteWhat multimodal AI models can do with images, documents, audio and video, real business uses, and what to watch for when building with them.
A practical approach to testing AI features: building test sets, measuring accuracy and safety, using automated checks and monitoring quality after launch.
How large language models like the ones behind ChatGPT, Claude and Gemini actually work, what they're good at, where they fall short and how businesses use them.