AI vendor due diligence: questions to ask before you buy
A practical checklist for evaluating AI software vendors on data protection, security, accuracy, compliance, costs and long-term risk before signing contracts.
A step-by-step approach to choosing, scoping, running and evaluating an AI pilot, so you can decide with evidence whether to scale, change or stop.
A good AI pilot tests one focused use case with clear success measures, a baseline to compare against, realistic data, a small group of users, risk controls and a fixed timeframe. Measure time saved, quality, errors, user satisfaction and costs, then decide whether to scale, adjust or stop. Many failed AI projects skip baselines or success criteria.
Many businesses try AI with enthusiasm and end up unsure whether it worked. A structured pilot replaces opinions with evidence.
Good pilot candidates are:
Examples include drafting support replies, extracting invoice data, summarising documents or answering internal policy questions.
Agree measures before starting:
Set thresholds for success, such as "at least 30% time saved with no drop in quality."
Record current performance: time taken, error rates and costs. Without a baseline, you can't prove improvement.
Gather representative data, including difficult cases. Clean and organise it, and protect personal information.
Limit the pilot to one team, process or channel. Set a fixed duration, often four to eight weeks.
Use existing tools, vendor trials or a custom prototype. Our guide to AI vendor due diligence lists questions to ask vendors.
Train participants, gather feedback regularly and observe how they use the tool. Their experience often reveals practical issues.
Compare results against the baseline and success criteria. Consider:
Stopping a pilot that didn't work is a success of the process.
Plan for integration, training, support, monitoring and ongoing costs. Knowledge sources often need investment; our guide to AI knowledge management explains why.
An accounting firm piloted AI drafting for routine client emails. Baseline measurements showed staff spent an average of ten minutes per routine email. During a six-week pilot with five staff, AI drafts reduced time to about four minutes, with reviewers correcting minor errors in some drafts. Client satisfaction stayed stable, and costs were modest. The firm scaled the tool to all client-facing staff, with training and a review checklist, and set quarterly quality checks.
Share the charter with leaders and affected staff before starting. Explain that the pilot is a test, and that results, including negative ones, will inform decisions.
Record what worked, what didn't and why. These lessons speed up future pilots and help avoid repeating mistakes.
Production needs more than a successful pilot: security reviews, integration, support processes, training materials and monitoring. Plan these before announcing a full rollout.
Pick a process that is frequent, measurable and low risk, with a team keen to try something new. Avoid pilots that touch sensitive data, regulated decisions or customer-facing promises until you have experience. A modest, well-chosen first win builds confidence for bigger projects.
Staff whose work changes during a pilot should help design it. They understand the process, spot problems early and know what good results look like. Ask for their feedback weekly, take concerns seriously and share results openly. Pilots that feel imposed often fail, even when the technology works, while pilots shaped by the people doing the work tend to stick and scale more smoothly.
Our AI solutions team designs and runs AI pilots with clear measurement, our web app developers build prototypes quickly and our AI chatbot team pilots assistants with real users.
Often four to eight weeks, long enough to gather meaningful data.
Frequent, time-consuming, well-defined, lower-risk tasks with clear value.
Document lessons and decide whether to adjust or stop. A well-run pilot that shows something doesn't work still saves money.
This guide is general information, not legal, tax or financial advice, and details such as prices, features and rules can change. Product and company names mentioned are trademarks of their respective owners and are used only to identify those products; UK Technology Hub is not affiliated with or endorsed by them.
Tell us about your project and get a free, fixed quote.
Get a free quoteA practical checklist for evaluating AI software vendors on data protection, security, accuracy, compliance, costs and long-term risk before signing contracts.
How AI helps organisations capture, organise and find internal knowledge, from documents and wikis to meeting notes, with the governance needed to keep answers accurate.
How large language models like the ones behind ChatGPT, Claude and Gemini actually work, what they're good at, where they fall short and how businesses use them.