AI vendor due diligence: questions to ask before you buy

A practical checklist for evaluating AI software vendors on data protection, security, accuracy, compliance, costs and long-term risk before signing contracts.

By the UK Technology Hub teamPublished 4 minute read

The short answer

Before buying AI software, ask vendors how they use your data, whether it trains their models, where it's stored, how they secure it, how accurate the product is on tasks like yours, how they handle bias, what regulations they comply with, how pricing scales and how you'd leave. Get answers in writing, test with your own data and include key commitments in contracts.

AI vendors are everywhere, and many products look similar in demos. The differences that matter, such as data handling, accuracy and long-term costs, often only appear after signing. Due diligence protects your business.

Data use

  • What data does the product collect and process?
  • Is our data used to train or improve your models? Can we opt out?
  • Where is data stored and processed?
  • How long is data retained, and can we delete it?
  • Who at the vendor can access our data?
  • Do you use sub-processors, including AI model providers?

Security

  • What security certifications or audits do you have?
  • How is data encrypted?
  • How do you handle access control and authentication?
  • What's your incident response process and notification timeline?
  • How do you protect against prompt injection and AI-specific attacks?

Accuracy and quality

  • How accurate is the product on tasks like ours?
  • Can we test with our own data before committing?
  • How do you measure and monitor quality?
  • What happens when the AI is wrong?
  • How do model updates affect behaviour?

Bias and fairness

  • How do you test for bias?
  • What safeguards exist for decisions affecting people?

Compliance

  • Which regulations do you support, such as data protection laws and the EU AI Act?
  • Can you provide documentation for our compliance needs?
  • Do you provide data processing agreements?

Transparency

  • Can users be told they're interacting with AI?
  • Can outputs be explained or traced to sources?
  • What logs are available to us?

Contracts

  • Service levels and uptime commitments.
  • Data ownership and deletion on exit.
  • Liability and indemnities, including for intellectual property claims.
  • Notice of material changes.

Costs

  • Pricing model: seats, usage, conversations or outcomes.
  • Costs at expected scale in one to two years.
  • Extra fees for integrations, support or features.

Lock-in and exit

  • Can we export our data, configurations and content?
  • How portable are workflows to other tools?
  • What happens if the vendor is acquired or closes?

Support and roadmap

  • Support channels and response times.
  • Implementation help.
  • Product roadmap and stability.

Sustainability

  • Information on energy efficiency and data centre sources. Our guide to AI energy use explains why this matters.

Test before committing

Run a structured pilot with real data and success measures. Our guide to AI pilot projects explains how.

Red flags

  • Vague answers about data use.
  • No way to test with your data.
  • Unrealistic accuracy claims.
  • Difficult or costly exit terms.

A worked example

A professional services firm shortlisted three AI document review tools. All gave impressive demos. Due diligence revealed that one tool used customer data to improve its models by default, another stored data in a region the firm's clients wouldn't accept, and the third offered clear data terms, regional hosting and a trial with the firm's own documents. In the trial, accuracy on the firm's real documents differed from the demo, so the firm negotiated a phased contract with performance reviews.

Scoring vendors

Create a simple scoring sheet with weighted criteria, such as data protection, accuracy, integration, cost and support. Score each vendor after demos and trials to make comparisons objective.

Contract essentials

  • Data processing agreement.
  • No training on your data without consent.
  • Data deletion on termination.
  • Security commitments and breach notification.
  • Service levels and remedies.
  • Notice of significant model or product changes.

Ongoing vendor management

Due diligence isn't one-off. Review vendors annually, monitor changes to terms and products, and reassess if they change ownership or sub-processors.

Involving the right people

Include IT, data protection, legal, finance and the teams who'll use the tool. Each spots different risks.

Red flags

  • Vague answers about data use or storage.
  • No security documentation or certifications.
  • Refusal to sign a data processing agreement.
  • Claims of perfect accuracy.
  • No way to export your data.

Using a trial wisely

Test vendors with your own real, but non-sensitive, data and tasks rather than relying on demos. Measure accuracy, speed and ease of use with the people who will use the tool daily, and record results in your scoring sheet.

How we help

Our AI solutions team runs vendor evaluations and pilots, our web app developers assess integration effort and our AI chatbot team compares chatbot platforms objectively.

Frequently asked questions

What's the most important question to ask an AI vendor?

How they use your data, including whether it trains their models, is one of the most important.

Should I test AI software before buying?

Yes. Test with your own data and success measures in a structured pilot.

How do I avoid AI vendor lock-in?

Check data export options, portability and exit terms before signing.

This guide is general information, not legal, tax or financial advice, and details such as prices, features and rules can change. Product and company names mentioned are trademarks of their respective owners and are used only to identify those products; UK Technology Hub is not affiliated with or endorsed by them.

Related guides

All AI & automation guides