An insurer’s leadership team is choosing a vendor to build an assistant that will triage incoming claims and answer policyholders’ questions. Two companies reach the final round. The first runs a polished demo: the assistant speaks naturally, knows the product names and replies in under a second. The second brings no demo. It asks for 300 real, anonymised claims from the last quarter, proposes two weeks to build an evaluation set from them and only then wants to talk about the quality threshold that will go into the contract.
Leadership picks the first. Three months later the assistant is live, but nobody can say how good it is. A claims handler insists it gets one conversation in five wrong; the vendor says that is rare. Neither side has a number, because nobody agreed in advance what a “correct answer” meant. The acceptance dispute lasts longer than the build.
This article is for CEOs and CTOs choosing an AI development company, meaning a firm that will build a system for them on top of large language models or machine learning. Its thesis is this: when you commission an AI system, portfolio and demo matter very little; what decides the choice are the answers to questions about measurement, ownership and exit. A company that cannot explain before signing how it will prove the system works on your data, or that promises a specific accuracy before it has seen that data, should be eliminated regardless of price and references. Below are 12 questions that test this, and the answers that should end the conversation.
Why do an AI company’s portfolio and demo tell you less than they seem to?
Because in AI systems the gap between a version that looks good and a version that works well is wider than in conventional software. A form in a web application either works or it does not, and you can see which straight away. An assistant built on a language model always answers fluently, including when it is wrong. A demo prepared on a dozen hand-picked questions shows that the company can connect a model and write a decent prompt. It does not show how the system will behave on a thousand of your cases, with typos, attachments, out-of-scope questions and users deliberately trying to push it off balance.
A portfolio has the same limitation. A list of assistants, recommendation engines and document automation projects says the company has run such projects. It does not say whether the company knew how well they worked. The question that actually tests a reference is: “How did you measure the quality of that system, and what was the result?” A company that answers with the client’s impressions rather than numbers from an evaluation set is describing a project whose quality it never knew.
The fair counter-argument is that a demo does show the team’s skill, and measurement can be agreed later. That is partly true. A demo filters out firms that cannot work with models at all. The problem is that “later” usually means after the contract is signed, when the client has lost its negotiating leverage and the vendor has a natural interest in a quality measure that is easy to meet. Agreeing the measurement method before signing does not take much work. It only requires the company to have one.
The second reason a demo is a weak signal is the pace of model change. A demo built on a particular model version may behave differently within months, because that version will be retired. The value that survives the project is not a prompt tuned to one version but an evaluation set that lets you test every version that follows. That is why most of the 12 questions concern things that outlast a model change: measurement, data, cost, regulatory roles and exit terms.
How is commissioning an AI system different from buying an AI tool?
We have already covered how to assess vendors of off-the-shelf AI tools: problem fit, model performance, privacy, integration, pricing, lock-in and references. That article is about buying a product. This one is about a different situation: hiring an outside company to build a system that will be yours.
The difference has three practical consequences. First, with an off-the-shelf tool quality is a property of the product that you check in a pilot. With a custom system quality does not exist yet, so you must agree up front how it will be measured and who has the final word. Second, with a tool the intellectual property stays with the vendor and you get a licence. With a custom build, artefacts are created that can belong to you: code, prompts, evaluation sets, fine-tuning data, model weights, data pipelines. If the contract does not name them, some will stay with the developer. Third, a custom build brings a third party into the chain: the foundation model provider whose model the developer calls through an API. Its pricing and deprecation decisions affect your system, even though you may not be party to its contract with the developer, or you may be a party the developer never mentioned.
The general principles of a software development contract, such as IP assignment, acceptance, warranty and liquidated damages, are covered in our article on the contract clauses that protect the client. Here we focus on what a standard software contract does not anticipate but what, in an AI project, determines cost and risk.
How will the company prove that the system works?
This is the most important group of questions, because everything else depends on it. Without a measurement method you cannot accept the system, compare it with the previous version or check whether a model change broke something.
Question 1: How will you measure that the system works, and who will build the evaluation set?
A good answer sounds roughly like this: we will build an evaluation set from your real cases, with your subject-matter experts defining what a correct answer looks like. It will cover typical cases, edge cases and out-of-scope questions where the system should decline or hand over to a person. We will specify how answers are graded: exact match, a rule checked in code, grading by a second model against written criteria, or human review. The set will be versioned, and it belongs to you.
This is not an invention of this article. Anthropic’s documentation for developers building on Claude starts application development with defining success criteria that are specific, measurable, achievable and relevant, and only then designing evaluations that measure against them. It recommends automating grading wherever possible and states plainly that more questions with slightly lower-signal automated grading is better than fewer questions with high-quality human hand-graded evals. A company that works with language models professionally should talk this way without prompting. How to build such a set and grade a system that answers differently every time is covered in our article on testing LLM applications.
The disqualifying answer: “We will test the system with your team during UAT.” User acceptance testing is necessary, but it does not replace an evaluation set because it is not repeatable. A few people chatting to the assistant for a week produce an impression, not a measurement, and that impression cannot be reproduced after the model changes. The second disqualifying answer: “We will prepare the test set ourselves.” A set prepared by the developer measures what the developer considered important, and the developer has an interest in a high score.
Question 2: What quality threshold will go into acceptance, and what happens if it is not met?
A good answer: we will set the threshold after a short baseline phase, once we see how the simplest version performs on your set. It will be stated concretely, for example the share of correct answers broken down by case category, a maximum share of harmful or fabricated answers and a maximum response time. The contract will say what happens if the threshold is missed: another iteration at the developer’s cost, a change of scope, or ending the engagement after that stage without paying for the next.
The disqualifying answer is a specific number promised before the company has seen the data. “We guarantee 95% accuracy” reads well in a proposal, but nobody can honestly promise it without knowing how hard your cases are and what counts as success. Such a promise usually means one of two things: either the number will be measured on a set chosen by the developer, or it will not be measured at all. Either way you are paying for a number that means nothing.
The unit of the threshold matters too. Average accuracy across the whole set hides the categories where the system fails. If 80% of claims are simple status queries and 5% involve disputed liability, the system can score 92% overall and get half the disputed cases wrong. The threshold should be stated separately for the categories with the highest cost of error.
What happens to quality after go-live?
AI systems lose quality for reasons conventional software does not have. The mix of questions shifts because customers start asking about a new product. The documents the system draws knowledge from change. And the model itself changes, because the provider releases a new version or retires the old one. Code nobody has touched can start performing worse.
Question 3: How will you detect a drop in quality after go-live and after a model version change?
A good answer has two parts. The first is regression testing: the evaluation set from question 1 run automatically on every change to a prompt, source documents, retrieval configuration or model version, with the score compared against the previous run. The logic is the same as in conventional QA, except that answers are graded rather than form fields checked. The second part is production monitoring: a sample of real conversations graded regularly, indicators such as handover rate, refusal rate and user ratings, and a threshold that triggers an alert when crossed.
The disqualifying answer: “The model is stable, so there is no need.” Models offered through APIs are not stable in the sense that matters to a client, because providers retire them. Anthropic states in its documentation that it gives customers at least 60 days’ notice before retiring a publicly released model. An example from this year: customers were notified on April 14, 2026 that Claude Sonnet 4 would be retired, and from June 15, 2026 requests to it stopped working. A model released in May 2025 was therefore available for little more than a year. A team without an automated evaluation set then has two months to check by hand whether the new version behaves the same. A team with one runs the measurement and knows within an hour what has changed.
It is also worth asking who bears the cost of adapting the system to a new model version. If model retirement is not covered in the contract, the developer will price the migration as a change request, even though it was foreseeable from day one.
Who will own the prompts, the data and the weights?
An AI project produces more kinds of artefacts than a conventional software project, and not all of them are obviously protected works under copyright law. That means a standard clause assigning rights to “the software and documentation” may not cover them.
Question 4: Who will own the prompts, the evaluation set, the fine-tuning data and the model weights?
The artefacts worth naming in the contract are: application and integration code; system prompts and prompt templates; the evaluation set with expected answers and grading criteria; data prepared for fine-tuning (examples, labels, corrections); weights or adapters of any model fine-tuned in the project; retrieval and document-chunking configuration; data pipelines and the scripts to rebuild them; and architecture and design-decision documentation.
For code, the legal position is fairly clear: software is protected by copyright, and the rights must be transferred by contract. In the UK, for instance, an assignment is only effective if it is in writing and signed by the assignor (Copyright, Designs and Patents Act 1988, s. 90(3)). For prompts, evaluation sets and weights it is much less clear. A prompt may or may not be a protected work, depending on how much creativity it contains. Model weights are produced by a computational process, and whether copyright protects them is at best contested. The safest approach is not to rely on whether a given artefact is a protected work but to state in the contract that the developer hands these artefacts over, assigns whatever rights exist in them, grants an exclusive, perpetual licence for the rest, and treats them as the client’s confidential information and trade secrets.
The disqualifying answer: “Prompts and methodology are our know-how.” A developer can legitimately keep its own tools and libraries used across many projects, with a licence to you. But prompts and an evaluation set built on your data for your process are precisely the elements without which the system cannot be moved. If they stay with the developer, every subsequent developer starts from scratch.
Question 5: Where will our data go, and will anyone train on it?
Data in an AI project passes through more hands than in a conventional one: the developer, the model provider, often a vector database provider and monitoring tools. If personal data is involved, each should be listed as a processor or sub-processor in the data processing agreement under GDPR, and the transfer mechanism should be clear for any processing outside the EU.
The training question has a concrete answer at the major model providers, but you need to know it. OpenAI’s developer documentation states that since March 1, 2023, data sent to the API is not used to train or improve its models by default, and that abuse monitoring logs are retained for 30 days by default. Zero Data Retention is available only to customers approved by OpenAI. These are API terms, not consumer product terms, and they can vary by product and agreement. An AI development company should be able to tell you which service it will send your data through and on what terms, in which region it is processed, and whether retention-limiting terms are in place.
The disqualifying answer: “We use the API, so the data is safe.” It skips the questions of processing region, retention period, whether the developer itself stores conversations in its own logs, and whether its staff put your data into tools that are not in the contract. A second disqualifying answer: “We will use your data to improve our solution for other clients,” offered without separate consent and a price.
What will the system cost once it is running in production?
Build cost is easy to compare between proposals because it is expressed in person-days or a project price. Run cost is harder, because it depends on an architecture the client usually cannot see. In systems built on language models you pay for every model call, and one user question can mean several calls: intent classification, document retrieval, answer generation, a check by a second model. How the budget unfolds from pilot to production is covered in our article on AI implementation cost from PoC to production. Here the point is the question to ask before the architecture is chosen.
Question 6: What does one business event cost at current volume and at ten times current volume?
A good answer is a calculation, not a rate. The company should work out how many model calls one business event requires (one handled case, one processed document), how many input and output tokens each consumes on average, the price of the chosen model, the cost of retries on errors, vector search, log storage and monitoring. The result multiplied by current volume gives the monthly cost. Multiplied by ten times current volume, it shows whether the architecture survives success.
A simple example shows why this matters. Suppose the assistant handles 20,000 cases a month. Version one sends the full policy wording and the whole conversation history to the model with every question, about 15,000 input tokens in total. Version two retrieves only the three relevant policy clauses and trims the history, about 3,000 tokens. At the same price per token, version two’s input cost is five times lower, and at ten times the volume the difference stops being an IT budget line and becomes a P&L line. This is an illustration based on assumed values, not any specific model’s price list, but the ratio is typical of decisions taken in the first weeks of a project.
The disqualifying answer: “API costs are negligible.” They can be in a pilot with a hundred users. In production they depend on the architecture, and a company that cannot calculate them at proposal stage has designed a demo, not an architecture.
Question 7: Who pays the model API bills, and on what terms?
There are three common arrangements. In the first, the client holds its own account with the model provider and pays directly, and the developer works on it on the client’s behalf. In the second, the developer pays and re-invoices the cost, with or without a disclosed margin. In the third, the developer bundles model cost into a flat service fee.
The first gives the client the most control: it sees usage, sets limits, and keeps the account when it changes developer. The second and third are more convenient at the start, but they hide usage and tie you to the developer more tightly than the contract suggests, because keys, limits and usage history sit on its side. The disqualifying answer is not about the arrangement but about transparency: “Model cost is included in the subscription,” with no cap, no usage report and no rule for what happens when volume grows. Such a clause means that when traffic rises you either pay for a renegotiation or the developer starts saving on quality by switching the system to a cheaper model.
What if the model provider changes the rules?
The company you choose is almost certainly not the author of the foundation model. It uses a model from one of the large providers or an open-weight model running on its own or your infrastructure. Either way your system depends on decisions that neither you nor the developer controls.
Question 8: What happens when the model provider retires a version, changes its pricing or changes its terms?
A good answer has three elements. First, an abstraction layer: code written so the model can be swapped in one place without rewriting the application. Second, the evaluation set from question 1, which lets you check within hours whether another model or a new version gives comparable results. Third, a fallback: a named second model that has passed the same measurement and can take over traffic if the first is unavailable or becomes too expensive.
Changes are not limited to retiring whole models. Providers also remove request parameters and change API versions. Anthropic’s documentation records, for example, that API version 2026-10-21 removed the temperature, top_p and top_k parameters, and that requests using them on that version return an error. For an application that hard-coded those parameters, that is a change requiring a code fix and a fresh quality measurement. You cannot predict the date of such changes, but you can predict that they will happen.
The disqualifying answer: “We work exclusively with model X because it is the best.” Choosing one model as the default is reasonable. Building the system so that it cannot be moved is a decision whose cost you will bear. In an AI project lock-in has two floors: to the developer and to the model provider, and each has to be addressed separately.
Does the company start with the simplest solution?
Not every problem a board calls an AI problem needs a large language model. Some can be solved with classical machine learning on tabular data, some with rules, some with a well-written prompt and no extra infrastructure. An AI development company earns more from more complex projects, so it is worth checking whether its recommendation follows from the problem or from its offering.
Question 9: Where will you start — prompting, retrieval, fine-tuning or classical machine learning — and why?
A good answer starts with the simplest variant that could work and describes which measurement result would justify moving to something more complex. For language tasks the typical order is: first a model with a well-specified prompt and a few examples; if it lacks company knowledge, retrieval-augmented generation (RAG) over your documents; and only when measurement shows those two are not enough, fine-tuning on your own examples. For predictive tasks on numerical data, such as demand forecasting or churn risk, the starting point is often a simple statistical model that can be explained and maintained, not a language model.
The disqualifying answer: “We will build you a custom model,” offered before any measurement. Fine-tuning has its place, but it requires preparing training data, infrastructure, re-tuning whenever the base model changes, and MLOps capability that someone will have to maintain after the project ends. If a prompt with retrieval meets the agreed threshold, fine-tuning adds cost and lock-in without adding value. A company that proposes it at the outset is selling what it knows how to sell, not what you need. When it is worth hiring outside machine learning expertise at all, and how to judge the result, is covered in our article on machine learning consulting.
The second disqualifying answer: “Our proprietary AI engine.” If that phrase covers a model from one of the large providers with a prompt layer on top, that is not a problem in itself, but you need to know it, because the answers to questions 5, 7 and 8 depend on it. If the company will not say which model sits underneath, you do not know where your data goes or what happens when that model is retired.
Who is responsible for EU AI Act compliance?
Regulation (EU) 2024/1689, the AI Act, assigns obligations by role, and it applies to providers placing AI systems on the EU market regardless of where they are established, so it matters to US and UK companies selling into Europe too. Simplified, a provider is the party that develops an AI system, or has it developed, and places it on the market or puts it into service under its own name or trademark. A deployer is the party that uses an AI system in the course of its activities. For a custom-built system used under your brand, your company may be the provider and the development firm merely a contractor working on your behalf. That reverses the intuition of many boards, which assume the regulatory obligations sit with “the AI company”.
Question 10: Who will be the provider and who the deployer, and what documentation will we receive?
The question matters most for high-risk systems, which include the Annex III use cases such as recruitment and decisions about workers, creditworthiness assessment of individuals and access to essential services. Article 25 describes when a distributor, importer, deployer or other third party becomes the provider of a high-risk system: when it puts its own name or trademark on it, when it makes a substantial modification to a system that remains high-risk, or when it changes the intended purpose of a system so that it becomes high-risk. The original provider must then cooperate by sharing the necessary information and technical documentation.
The dates need checking, because they changed in 2026. Under the current wording of Article 113, the Regulation became generally applicable on August 2, 2026, but the requirements for Annex III high-risk systems apply from December 2, 2027, and those for AI embedded in products covered by Annex I from August 2, 2028. Obligations for general-purpose AI models have applied since August 2, 2025. The European Commission’s guidelines for providers of such models indicate that provider obligations apply only to those who make significant modifications to a model, not to everyone who fine-tunes one. For the client the conclusion is practical: a system you commission today will be in operation when the high-risk requirements apply, so documentation is best produced alongside the system rather than reconstructed later.
A good answer: a preliminary risk classification of your use case with reasoning, a proposed split of roles written into the contract, and a list of documents the developer will hand over: a description of the system’s intended purpose and limitations, evaluation results, a description of the data used to build it, and a description of the oversight and logging mechanism. The disqualifying answer: “The AI Act does not apply to us because we use an off-the-shelf model.” That confuses the obligations of a general-purpose model provider with those of the provider of a system that uses the model.
Question 11: Where does a person stay in the process, and what does the system log?
Even outside high-risk systems, the transparency obligations of Article 50 have applied since August 2, 2026: people must be told they are interacting with an AI system unless that is obvious from the context. For high-risk systems, Article 14 requires effective human oversight to be designed in, and Article 26 requires deployers to keep automatically generated logs for at least six months.
A good answer identifies the specific points in the process where a person makes or approves a decision, the criteria for handing a case to a person, and the scope of logging: input, output, model and prompt version, source documents used, and the human decision. Logs should be exportable without the developer’s involvement. How to design oversight that does not turn into clicking “approve” is covered in our article on human in the loop in AI processes. The disqualifying answer: “The system is fully autonomous; no human is needed,” offered for a process where an error affects a customer or an employee.
Who will work on the project, and how will you part ways?
The last question joins two things that go together in practice: people and exit. An AI system that nobody but its authors understands is a dependency, even if the contract assigns every right.
Question 12: Who exactly will work on the project, and what does handover look like when we end the engagement?
The first part is about the team. AI projects need a rare combination: an engineer who knows models and evaluation, a data engineer who will build and maintain the pipelines, a developer who will integrate the system with your applications, and an analyst who will translate the business process into test cases. Ask for the names of the people on the team, their time allocation and the substitution rule. A pitch led by a senior architect who disappears from the project after signing is a familiar pattern, and in an AI project its consequences are larger, because the reasoning behind why a prompt looks the way it does rarely makes it into documentation.
The second part is about exit. A good answer describes a handover package: the code repository, prompts, the evaluation set with the results of the latest run, fine-tuning data and weights if any were produced, infrastructure configuration as code, access to the model provider and monitoring accounts, instructions for rerunning the evaluation and a record of architectural decisions. Plus a support period for the handover and its price stated as a number. The best test is to ask for the table of contents of the handover package at proposal stage.
The disqualifying answer: “We will hand over the source code.” In an AI project, code without the evaluation set and the current prompts is like a car without its keys: it formally belongs to you, but the new team has no way of knowing whether the system still behaves the same after the first change.
Which answers should disqualify a company immediately?
The table below collects the 12 questions in a form you can use in vendor meetings or an RFP. The disqualifying column does not list weak answers; it lists answers after which further conversation is pointless, whatever the price.
| No. | Question | A good answer includes | Disqualifying answer |
|---|---|---|---|
| 1 | How will you measure that the system works? | An evaluation set from your cases, grading criteria, versioning, ownership on your side | ”We’ll check it in UAT” or “we’ll build the test set ourselves” |
| 2 | What quality threshold goes into acceptance? | A threshold set after a baseline, by category, with consequences if missed | Guaranteed accuracy before seeing the data |
| 3 | How will you detect a quality drop? | Automated regression evaluation on every change, production sampling, alerts | ”The model is stable, there’s no need” |
| 4 | Who owns prompts, test set, weights? | Artefacts named in the contract, assignment or exclusive licence, handover in usable form | ”Prompts and methodology are our know-how” |
| 5 | Where will our data go? | List of processors, region, retention, no training without consent | ”We use the API, so the data is safe” |
| 6 | What does one business event cost? | Cost per event at current and 10x volume | ”API costs are negligible” |
| 7 | Who pays for the model API? | A transparent arrangement, usage reporting, caps, a rule for volume growth | Flat fee with no cap and no usage report |
| 8 | What if the model provider retires a version? | Abstraction layer, measured replacement, fallback model, migration cost in the contract | ”We only work with model X” with no portability plan |
| 9 | Where will you start? | The simplest variant and a measurement threshold that justifies the next step | A custom model or fine-tuning before any measurement; undisclosed base model |
| 10 | Who is the provider under the AI Act? | Preliminary risk classification, roles in the contract, list of documents | ”The AI Act doesn’t apply because we use an off-the-shelf model” |
| 11 | Where does a person stay, what is logged? | Human decision points, handover criteria, scope and export of logs | ”Fully autonomous” for a process with costly errors |
| 12 | Who works on it, and what is the exit? | Names and allocation, handover package contents, price of exit support | ”We’ll hand over the source code” as the whole exit plan |
The first three rows and row nine carry particular weight, because they test whether the company has a method rather than just a proposal. A company may lack a ready answer on the AI Act if your use case is unusual, and that can be resolved with a lawyer. A company without a measurement method will not acquire one mid-project, because every technical decision will be made by feel.
How should you ask these questions so the answers can be compared?
Questions asked aloud in a pitch produce answers that are hard to compare, because each company answers in a different order and at a different level of detail. It is better to put them in writing as part of the RFP and ask for responses in a fixed format: a short description, a sample artefact (an anonymised excerpt of an evaluation set from another project, a template evaluation report, the contents of a handover package) and a proposed contract clause.
The request for an artefact is the best filter. A company that genuinely measures quality has an evaluation report template and can show it within the hour. A company that measures by feel will say it “will prepare one once the project starts”. The difference shows faster than in any demo.
The second step is a paid initial stage instead of a full project. Rather than choosing a developer for the whole project on the strength of proposals, commission a short stage in which the company builds the evaluation set with your experts, measures the simplest variant and proposes a threshold for the contract. The stage ends with a document and an evaluation set that belong to you and that you can use with another developer too. If the company refuses such a stage, or will only price it as part of a full project you cannot exit, that is also an answer. The same rules apply as to any proof of concept: success criteria written down before the start and an explicit decision on the next step at the end.
The third step is a reference call focused on measurement. Instead of asking a former client whether they were satisfied, ask how they measured the system’s quality, whether they received the evaluation set, what happened at the first model version change and what a month of running the system costs today compared with what the proposal assumed. Those four questions show whether the reference concerns a system that works or a system that was accepted once.
The fourth step is staffing your own side. None of the good answers will work if nobody on the client side owns the evaluation set and the quality threshold. Usually this is someone from the business function whose process the system serves, supported by someone from IT. Without that person, even the best measurement method stays with the developer, because only the developer will know how to use it.
What are the key takeaways before you sign?
Choosing an AI development company is decided by answers to questions a demo cannot test. Demo and portfolio show that the company can build something that looks good. Questions about measurement, ownership, run cost and exit show whether, when the contract ends, you will have a system you can assess, maintain and move.
Remember five things. First, an evaluation set built from your cases and owned by you is the most valuable artefact of the project, because it survives a change of model, developer and architecture. Second, a company that promises a specific accuracy before seeing your data has no measurement method and should be eliminated. Third, ask for the cost of one business event at current and ten times current volume, because that, not build cost, decides whether the system pays off. Fourth, name prompts, test sets, fine-tuning data and weights in the contract, because a standard software clause may not cover them. Fifth, settle the AI Act roles before the project starts, because for a system under your brand the provider may be you.
ARDURA Consulting supports companies building AI systems in two ways. We can add engineers to your team to build the evaluation set, data pipelines and integrations, through Python developers with AI and machine learning expertise, with a two-week specialist replacement guarantee and a 30-day notice period. Or we can run the project as custom software development, starting with a discovery phase of one to two weeks. If you want to work through the 12 questions for a specific use case before you send out an RFP, book a call.