Questions to ask an AI agency before you hire
Ask an AI agency these ten questions before you sign: who will actually do the work, can they provide two contactable references, how your data is stored and whether it trains models, who owns the code and IP, how milestones and acceptance criteria are defined, what happens if you reject a milestone, how change requests are priced, whether they will sign a Statement of Work, whether they will accept escrow-protected payment, and what happened on a project that went wrong. The answers separate real engineering teams from resellers.
Most buyers evaluate agencies on the pitch. The pitch is the part an agency has practised most, so it tells you the least. These questions are ordered to move from easy to uncomfortable — the last three are where the real differences show up.
1. Who exactly will do the work, and what have they built?
Agencies often sell with a senior engineer and deliver with juniors. Ask for the names and backgrounds of the people who will actually be on your project, and ask what they have personally shipped in your problem class.
Good sign: named people, specific shipped projects. Red flag: 'our team' with no names.
2. Can you share two references from similar projects?
Past delivery predicts future delivery better than any pitch deck. Ask for references matched to your scope and industry — then contact them and ask whether the project actually shipped on time and on budget.
Good sign: contactable references offered readily. Red flag: only written testimonials, or 'clients are under NDA' for every single project.
3. How is our data stored, and is it used to train models?
This is the question that most often exposes an agency's real maturity. You want a specific answer about where data lives, who can access it, how long it is retained, and whether any of it feeds model training.
Good sign: a clear policy and a willingness to sign a data processing agreement. Red flag: vagueness, or being surprised by the question.
4. Who owns the code, the model, and the IP?
Ownership should be settled in writing before work starts, not negotiated after. Clarify who owns the source code, any fine-tuned models, the training data, and any derivative work.
Good sign: 'you own it, here's the clause'. Red flag: ownership is 'something we can discuss later'.
5. What does success look like, and how is each milestone defined?
Ambiguous scope is the most common cause of failed AI projects. Every milestone should have a written deliverable and a testable acceptance criterion agreed before work begins.
Good sign: measurable acceptance criteria per milestone. Red flag: milestones defined only by dates or hours.
6. What happens if we reject a milestone?
Ask the awkward question early. A serious agency has a defined process: what triggers a rework, how many revision rounds are included, and what happens if you still cannot accept the deliverable.
Good sign: a defined rework and dispute process. Red flag: 'that won't happen'.
7. How do you price change requests?
Scope will change — the question is whether that is priced transparently or used as a lever. Agree the mechanism for pricing changes before the project starts.
Good sign: a published rate and a written change process. Red flag: change pricing decided case by case, after the fact.
8. Will you sign a Statement of Work?
A Statement of Work turns a conversation into an enforceable agreement covering scope, milestones, acceptance criteria, timelines, and IP. Any agency unwilling to sign one is telling you something important.
Good sign: they have their own SOW template and welcome yours. Red flag: reluctance, or 'we work on trust'.
9. Will you accept milestone-based, escrow-protected payment?
Escrow means your money is held securely and released only when you approve each milestone. Agencies confident in their delivery rarely object.
Good sign: happy to work milestone-to-milestone. Red flag: pressure for a large upfront payment outside escrow.
10. Tell me about a project that went wrong.
The most revealing question on this list. Everyone who has shipped real work has a story. What you are testing is honesty and whether they learned something specific.
Good sign: a specific story with a specific lesson. Red flag: 'nothing has ever gone wrong'.
What to do with the answers
No agency will score perfectly, and they do not need to. What matters is the pattern: specific, documented answers on data, ownership and scope indicate a team that has delivered before. Vagueness clustered around money, ownership or references is the signal to walk away.
For the wider evaluation framework these questions sit inside, read how to vet an AI agency. If you want the warning signs on their own, see red flags when hiring an AI agency.
Where these questions are already answered for you
On NorthBridge AI, every listed agency has been through manual intake, portfolio review and company-registration verification. Every project ships with a signed Statement of Work, milestone-based escrow, and independent mediation if a dispute arises — so questions 8, 9 and 10 are handled by the platform rather than left to trust. See how it works.
Frequently asked questions
What questions should I ask an AI agency before hiring?
Ask who specifically will do the work and what their background is, for two contactable references from similar projects, how your data is stored and whether it is used for model training, what happens to the model and code if you stop working together, how scope and milestones are defined, what the acceptance criteria are for each deliverable, how change requests are priced, what happens if a milestone is rejected, whether they will sign a Statement of Work, and whether they will accept milestone-based escrow payment.
What is the single most revealing question to ask an AI agency?
Ask them to describe a project that went wrong and what they did about it. Agencies with real delivery history answer specifically and without defensiveness. Agencies without it deflect, claim nothing has ever gone wrong, or pivot back to a sales pitch.
Should I ask an AI agency for references?
Yes, and you should actually contact them. Ask for two references from projects similar to yours in scope and industry. A credible agency will provide them. When you speak to the reference, ask whether the project shipped, whether it hit the original timeline and budget, and whether they would hire the agency again.
Hire agencies that already answer these
Browse pre-vetted AI agencies on NorthBridge AI and hire with escrow protection.
Browse verified agencies