How to evaluate an AI project before you build
Assess the workflow, data, expected benefit, and risks of an AI project. Define the evidence you need before committing to a build.
By OlypsisUpdated 4 minute read
Overview
Start with a process your team wants to improve. Look at real examples, measure the time it takes, and identify a small part you can test. The audit should leave you with a clear recommendation, an estimate of the effort involved, and a way to judge the result.
Choose a process to examine
Ask your team which tasks take too long, require repeated copying between systems, or create a backlog. Useful starting points include handling supplier invoices, answering support questions, and researching prospective customers.
Choose one process and speak to the people who do it. Find out what they would like to change and what already works well. That gives the project a purpose before you decide which tools it needs.
Walk through real examples
Gather a small set of recent examples, including routine work and difficult cases. Ask someone to walk you through each one: where it arrived, what they checked, which tools they used, and what happened next.
For an invoice process, for example, you might find that someone reads a PDF, copies the details into accounting software, checks them against a purchase order, and emails a colleague about a mismatch. Each step has a different requirement. Reading the PDF, checking a total, and approving a payment should be considered separately.
- What information does each step need?
- What does a person have to read, copy, look up, or decide?
- Which unusual cases need extra attention?
- Where does the work stop, and what is it waiting for?
For a concrete example of narrowing the scope, see how to choose a first AI workflow.

Measure the current process
Record how often the task happens and how much time people spend on it. Keep time spent working separate from time spent waiting. An invoice waiting overnight for approval does not represent a full night of staff time.
Use those figures to estimate the potential benefit. Include the time people would still spend reviewing outputs and handling exceptions. Record assumptions where data is missing, and check them with the team before relying on the estimate.
Choose a manageable first build
Compare the benefit with the effort: access to data, connections to existing software, the variety of inputs, and the consequences of a mistake. Some steps may only need a rule or a standard integration. Others may benefit from AI that can interpret documents or questions.
In the invoice example, the first build could extract fields and prepare a draft for review. Your team could compare those drafts with the original invoices before deciding whether to automate more of the process.
Agree on how to test it
Decide what a useful result looks like before building. Depending on the task, that could mean less time per document, more accurate answers, or faster research. Include difficult examples and track errors as well as successful outputs.
Ask the team to try the whole process, including review and corrections. A fast first draft is only useful if it reduces the total effort needed to finish the task.
Include the people who will check the output when you plan human review.
What you receive from our audit
- A description of the current process and where the main problems occur.
- A recommendation for the first build, including its limits and the software it needs to connect to.
- An estimate of the benefit, with assumptions and missing information identified.
- A proposed scope, timeline, and approach to testing, plus the expected costs of building and running it.
| Decision | Evidence to capture |
|---|---|
| What is the task? | A starting event, the inputs, and a result the team can accept. |
| Can it be built? | Available data, permitted access, integrations, and a technical test. |
| Is it worth building? | Current effort, remaining review work, likely benefit, and cost assumptions. |
| How will we judge it? | Representative examples, acceptance criteria, and the response to failures. |
The audit’s timeline depends on the scope and access to your team and data. Agree on it before starting. If the work does not justify a build, the recommendation should explain why.
Have a project in mind? Tell us what you’d like to improve, and we’ll help you work out where to start.
Book a call