AI chatbot launch checklist: 30 checks before going live
A practical pre-launch checklist for AI assistants covering content, accuracy, safety, security, handoff, compliance, accessibility, performance and monitoring.
How attackers try to manipulate AI assistants through crafted messages and hidden instructions, the real-world risks, and practical defences for business chatbots.
Prompt injection is when someone crafts input, or hides instructions in content an AI reads, to make it ignore its rules, reveal information or take unintended actions. No single defence is perfect, so use layers: limit what the assistant can access and do, enforce permissions in your systems rather than in prompts, require approval for sensitive actions, check outputs, monitor conversations and test regularly.
AI assistants follow instructions written in natural language. That flexibility is also a weakness: attackers can try to override instructions with their own. As chatbots gain access to data and tools, prompt injection becomes a serious security issue.
A user types instructions designed to override the system, such as asking the bot to ignore its rules, reveal hidden instructions or pretend to be something else.
Hidden instructions sit inside content the AI reads: a web page, email, document or product review. When the assistant processes that content, it may follow the hidden instructions.
Give the assistant only the data and tools it needs. A public FAQ bot shouldn't have access to customer databases.
Don't rely on prompts to restrict access. If a user isn't authorised to see data, the underlying system should refuse, whatever the AI asks for.
Refunds, account changes and external messages should need confirmation from the user and, where appropriate, staff.
Content from websites, emails or documents shouldn't be able to change the assistant's instructions or trigger actions. Separate trusted instructions from untrusted data.
Filter obviously malicious inputs, and check outputs for sensitive data, policy violations and unexpected actions before they reach users or systems.
Assume instructions may be revealed, and never put secrets such as API keys or passwords in prompts.
Log conversations and actions, alert on suspicious patterns and review regularly.
Regularly try to break your own assistant with injection attempts, and include them in test suites. Our chatbot launch checklist includes security testing.
Financial services, healthcare and legal services face greater consequences. Our guide to AI chatbots in financial services covers sector-specific security.
Attack techniques evolve. Follow security guidance from AI providers and organisations such as OWASP, which publishes guidance on risks for large language model applications.
A software company's support assistant could search help articles and read customers' uploaded files. During testing, the security team uploaded a document containing hidden instructions telling the assistant to reveal its system prompt and share other customers' data. The assistant followed part of the instruction, revealing internal notes. The team restricted what the assistant could access, separated user content from instructions, added output filtering and limited actions to the logged-in customer's account. Further testing showed the attack no longer worked, and the company added regular red-team tests to its release process.
Log conversations securely, watch for patterns suggesting manipulation attempts and set alerts for unusual actions or data access.
Have a plan for suspected prompt injection incidents: who investigates, how to disable features quickly and when to inform affected customers or regulators.
Developers, support staff and content editors should understand prompt injection basics. Knowing how attacks work helps them spot suspicious conversations early.
Ask chatbot vendors how they defend against prompt injection, how often they test and how they handle reported issues.
Our AI chatbot development team designs assistants with layered security, our AI solutions team implements permissions and approval workflows and our web app developers build secure APIs that enforce access controls independently of the AI.
An attack where crafted input or hidden instructions make an AI ignore its rules, reveal information or take unintended actions.
Not completely with current technology, but layered defences greatly reduce risk.
Lower risk if they have no sensitive data or actions, but they can still be manipulated into embarrassing responses.
This guide is general information, not legal, tax or financial advice, and details such as prices, features and rules can change. Product and company names mentioned are trademarks of their respective owners and are used only to identify those products; UK Technology Hub is not affiliated with or endorsed by them.
Tell us about your project and get a free, fixed quote.
Get a free quoteA practical pre-launch checklist for AI assistants covering content, accuracy, safety, security, handoff, compliance, accessibility, performance and monitoring.
How banks, lenders, insurers, accountants and advisers use AI assistants for customer service, while managing regulatory duties, vulnerable customers and data security.
How business chatbots are changing, from agents that take actions and voice assistants to transparency rules, better knowledge grounding and new customer expectations.