Adding AI to your product: what it really costs

Start with the job, not the technology

AI earns its place when it does a specific job better or faster than the alternative: matching, summarising, classifying, drafting, or guiding a user through a task. In our own work, that has meant matching care seekers with culturally aligned care workers at CareAlgo, and an AI coach in Coded that guides learners rather than doing the work for them. "Add a chatbot" is not a job.

The two kinds of cost

• Build cost: designing the feature, connecting it to your data, building the screens around it, and testing that the answers are good enough. This is a one-off, and for a well-defined feature it is a modest part of a first release.
• Running cost: what you pay the model provider every time the feature is used. It grows with usage, which is good news if usage means revenue, and bad news if it does not.

Estimating running costs before you build

You can estimate this on the back of an envelope:

1. How many times a day will a typical customer use the feature?
2. How much text goes in and comes out each time? Model providers charge by volume of text, measured in tokens.
3. Multiply by the provider's current price per token, and by your expected number of customers.

Prices change often and vary widely between models, so check the provider's current pricing rather than an old article. Then compare the result with what each customer pays you. If the AI cost per customer is a large share of the subscription, you need a cheaper model, fewer calls, or a different price.

Ways to keep running costs down

• Use the smallest model that does the job well enough, and save larger models for the hard cases.
• Cache answers to repeated questions.
• Send only the data the model needs, not whole records.
• Set usage limits per customer so one account cannot run up an unexpected bill.

The costs founders forget

• Evaluation. A set of real examples to test the feature against, re-run every time you change a prompt or a model.
• Guardrails and human review. In care, health, education and finance, someone accountable must be able to check and override what the AI suggests.
• Data protection. Where personal data goes, whether the provider stores or trains on it, and a data protection impact assessment if the data is sensitive. Your buyers will ask.
• Explaining it. Users and regulators increasingly expect to know when AI is involved and how a decision was reached.

Start small

Pick one feature where AI clearly saves your users time, ship it to pilot customers with a human in the loop, and measure whether they use it. It is far cheaper to learn that from one feature than from an "AI-powered" product built on assumptions.

If AI is central to your product, we cover it in the architecture and cost range of the Fundraising Tech Pack. More on our AI product development page.