Consider an accounts payable team fielding questions from suppliers: Has the invoice arrived? Why is the amount disputed? When will payment be made? An AI assistant could read the enquiry, find the relevant records and draft a reply for a member of staff to approve. The proposal is to reduce the work enough to handle growing demand without another hire. This is an illustrative case, not a customer result.
A demonstration can show whether the assistant produces a useful reply. The person funding it needs a different answer: how much work would it remove, and would that change the team's need for another person? A fast answer to a small part of the queue may leave the staffing decision untouched.
The assessment should connect those two questions. Start with why suppliers need to ask, establish what a simpler change could resolve, then examine the work left for AI. The result should tell the sponsor what must happen for the investment to pay, how plausible that is, and which missing evidence is worth buying before committing to the build.
Give AI the right comparison
Ask the team to walk through recent enquiries before counting them. If suppliers chase invoices because they receive no acknowledgement, an automatic receipt may remove some of those messages. If nobody owns disputed amounts, a faster draft will still wait for someone to make the decision. The first improvement may be a notification or a clearer responsibility, not an assistant.
These alternatives have costs too. Compare the current process, the simpler improvement and the improvement with AI added. Count the benefit of each against the same starting point, then ask what the extra AI spending buys. Otherwise the assistant can receive credit for work that a cheaper change would have removed anyway.
Amazon's Java upgrades illustrate the distinction at a very different scale. In December 2024, Amazon reported that it had used Amazon Q Developer to migrate tens of thousands of applications to Java 17, with performance improvements delivering $260 million in annual cost savings.[1] That is company-reported evidence of a valuable change. To assess the AI contribution, we also need to ask how much more slowly or expensively the upgrades would have happened without it. Bringing those savings forward can itself be valuable. The full annual saving is not automatically the incremental return on the AI tool.
For the supplier assistant, the opportunity left after the simpler changes might be enquiries that require someone to read a long correspondence and reconcile it with several records. That is a candidate to investigate, not a reason to assume the build will pay.
Find the work that could support the investment
A total enquiry count conceals the difference between a routine status check and an hour spent resolving a discrepancy. Separate the recurring types and find where the effort sits. Look at who does the work, how often it recurs, and what still needs human judgement after the assistant has drafted its answer.
This changes both the build and the economics. One repeated task within a busy team may offer a manageable scope and enough released time to avoid planned recruitment. A few minutes spread across many roles may still be useful, but cannot be valued as an avoided hire unless the work can actually be reorganised. In that situation, access to an existing AI tool may be more proportionate than a dedicated integration.
Before investigating every detail, give the proposed scope a deliberately generous test. Suppose the assistant handles every suitable enquiry successfully. How much effort could it remove beyond the simpler fix, and what could that capacity be worth? Retain any work the design still requires, such as approval. Include other material benefits where there is a credible basis for them, and compare benefits and costs over the same period.
If even that ceiling cannot cover a defensible minimum implementation and operating cost, a more accurate model will not rescue this financial case. Change the scope or stop. Passing the ceiling means only that the idea deserves closer examination; it does not make the optimistic assumptions a forecast. A service or resilience objective may justify a different case, but it should be stated rather than disguised as a staffing saving.
Work backwards from the cost
For a proposal that survives, replace the generous assumptions with ranges the business can defend. How many enquiries would remain after the simpler change? On how many would staff actually use the assistant? How much effort would remain for checking, correcting and escalating its replies?
A small calculation makes the requirement tangible. Suppose, purely for illustration, the AI addition costs £60,000 in its first year and is used on 12,000 enquiries during that year. It needs an average benefit of £5 per enquiry just to cover that cost. The cost estimate must include implementation, data preparation, integration, training and operation, including the human work the new process requires. Count each cost once. The benefit must be incremental to the simpler process, not the value of all the work the team already does.
The next question is whether that benefit is achievable. If the claim is avoided recruitment, the operations leader needs to show that the released capacity is sufficient, in the right roles and at the right time, to cope without the planned hire. Multiplying minutes by a salary rate does not establish that. If the benefit is better service, define the improvement and the amount the business is willing to spend on it instead.
First-year break-even is only one view. A longer-lived investment needs costs and benefits over its expected life, including rollout delays, maintenance and eventual replacement. Finance should apply the company's required return and account for when the money is spent and received.[2]
Keep disagreements visible in the estimate. The supplier may expect little review; the team may expect to check every answer. Model both before averaging them away. Where there is enough evidence to justify probabilities, Monte Carlo simulation can combine uncertain inputs into a distribution of possible returns and estimate the chance of falling short of cost. That probability depends on the assumptions, including how inputs move together. A plausible range alone does not establish it.
Investigate the unknown that could change the answer
The assessment now has a purpose for collecting more data. If the case depends on reducing time spent finding invoice records, check whether the assistant can obtain the right records with the access it would actually have. If it depends on avoiding a hire, establish how much end-to-end effort it removes. Do not commission a broad data exercise when one of these questions could settle the decision.
A system log can help identify volumes and recurring paths. It cannot tell you that the three days between an enquiry and a reply were three days of work. The gap may contain waiting, searching, phone calls or a decision made elsewhere. Ask the people doing the work and observe or sample the relevant tasks. A useful early deliverable is a precise data request: what is missing, how to obtain it, and which investment assumption it would test.
Historical enquiries can test whether the assistant retrieves the right evidence, produces an acceptable answer and recognises cases it should escalate. Keep test cases separate from development examples, and give the assistant only information that would have been available at the time. This can establish technical performance on those cases. It cannot show whether staff will use the assistant, how much checking they will do in practice, or whether suppliers will send fewer follow-up messages.
Those questions may warrant a limited live trial against the simpler process, with comparable work and agreed quality checks. Set its budget, duration and decision threshold before starting. Confidentiality and acceptable error levels remain conditions of use, even if the average financial result looks attractive. Fund the trial only if its answer could change the investment decision enough to justify the cost and delay.
Keep the estimate after the decision
Before implementation, date the forecast and record the assumptions that made the investment worthwhile. In this example, the operations owner would track enquiry mix, uptake, total handling effort and service quality. Finance would check the claimed change in recruitment or spending, alongside actual implementation and running costs. Agree when to compare these with the forecast and what would justify continuing, changing course or stopping.
If the assistant writes good replies but the team still needs the planned hire, that is a discrepancy to explain. Perhaps review consumes the expected saving. Perhaps demand rose beyond the forecast. Perhaps the time was released in roles that could not absorb the extra work. Compare with what would have happened under the alternative process, rather than assigning every change after launch to AI.
The useful output of the assessment is a decision the business can revisit: build this scope because these conditions are supported, investigate this unknown because it could change the choice, or take the simpler option. Keep the original estimate beside the observed result. It is how the next investment learns from this one.
Sources and notes
[1] Amazon, “New Amazon Q Developer Capabilities Accelerate Large-Scale Transformations of Legacy Workloads”, 3 December 2024. The Java-migration figures are Amazon's own report. The release does not provide a full programme cost or a comparison with every alternative migration method. The distinction between upgrade value and incremental AI contribution is our analysis.
[2] HM Treasury, The Green Book (2026), chapters 1–2: options, lifetime costs and benefits, uncertainty, sensitivity analysis and planning evaluation. This is public-sector appraisal guidance, not a private-sector requirement or an endorsement of this assessment framework. The supplier-enquiry case and its first-year arithmetic are illustrative assumptions, not measured performance or a quoted implementation price.