Systems & Execution
The short answer: pick one workflow, collect 20 to 50 real examples of that work, and test every candidate against them. Compare what your existing software already does, what plain automation could do, and what a new AI tool would add. Score them against the same five things (workflow fit, data access, permissions, total cost and switching effort) and choose the option that wins on your work, not on the demo.
That sounds obvious. It rarely happens. Most small firms start from the tool, usually the one a peer mentioned or the one with the best launch video, and work backwards to a use for it. When we coded 105 first-hand discussions from service businesses, tool and partner selection came up more than any other challenge. One eight-person digital agency was stuck choosing between Gemini and Claude for its core platform. Another firm had, in one poster's words, bought six subscriptions and hoped.
This guide gives you a decision method you can run in a few weeks: a weighted scorecard, a test-set template, a full-cost model and one worked agency example with the maths shown.
Three things distort most comparisons.
The demo is built for the demo. Vendors show clean inputs and a happy path. Your work arrives as a forwarded email chain, a half-filled brief and a spreadsheet with three versions of the client's name. A tool that shines on tidy data can stall on yours, which is why Gartner found 63% of organisations lack, or are unsure they have, the data management practices AI needs, and predicts that 60% of AI projects without AI-ready data will be abandoned through 2026.
Public benchmarks measure someone else's work. Leaderboards test models on standard question sets. They say little about whether a tool can turn your call notes into a scoping document your senior consultant would sign off.
Labels run ahead of products. Gartner calls this "agent washing", where chatbots and older automation tools are rebranded as agents, and estimates only about 130 of the thousands of vendors selling agentic AI are the real thing.
The fix for all three is the same. Test on your own work and price your own costs.
A tool can only be judged against a job. Write the job down first, in one page, before anyone opens a pricing page:
If you can't fill this in, you aren't ready to choose a tool. You are ready to map the process, which is where most of the value sits anyway (we cover why in Your Business Context Is Your Biggest AI Asset).
The whole selection method then runs in this order:
This is how you compare AI platforms using your actual work. Pull past examples of the workflow, with the input as it arrived and the output your team finally sent. Anthropic's guidance on evaluating AI agents is that 20 to 50 simple tasks drawn from real failures is a great start, and that turning reported failures into test cases keeps the set close to real usage.
Pick deliberately. Include the easy ones, the messy ones and two or three that went wrong last year. A test set made only of clean examples will flatter every tool equally.
Each test case needs more than a prompt and an answer:
Then have the person who reviews that work today score each tool's output against the checks, blind to which tool produced it. If you use another AI model to grade outputs at scale, calibrate it against human scores first. Researchers testing this method found AI judges show position bias, verbosity bias and a tendency to favour their own answers, even though the strongest models agreed with human preferences about as often as humans agree with each other.
Long vendor lists are where decisions go to die. Group the choices into three types instead, and only shortlist one or two candidates within each.
The middle row is the one people skip. In our research, one agency solved client reporting with direct platform integrations and no AI at all. If your bottleneck is copying numbers from one system into another, a model adds cost and a new way to be wrong.
Often, yes, as the first thing you test. Vendors have pushed AI into the subscriptions small firms already pay for. Google folded Gemini into its Business and Enterprise Workspace plans in January 2025, raising Business Standard from $12 to $14 per user per month on annual plans. Microsoft launched Microsoft 365 Copilot Business for organisations of up to 300 users at a list price of $21 per user per month, against $30 for the enterprise licence. (Prices change often; check them on the day you decide.)
Built-in AI starts with two advantages: it already sits next to your files, and it inherits the access rules you already have. The answer changes when:
That last point came up in our research too: a two-person business didn't want to pay for seats for staff it didn't have. Per-seat pricing is designed for headcount, and your saving may come from a single workflow.
Here is the weighted scorecard we use as a starting point. The weights are Orbflo's default for a typical service workflow, not an industry standard; move them if your situation calls for it. A recruitment firm handling candidate data might raise permissions to 25%. A five-person studio with no IT support might raise switching effort.
Score each option from 1 to 5 on each criterion, multiply by the weight, and add up. Do the scoring with the person who does the work and the person who pays for it in the same room. They will disagree, and the disagreement is useful.
On permissions, remember that if a tool processes client or candidate personal data on your behalf, UK GDPR requires a written contract with specific terms. The ICO sets out what a controller-processor contract must include. Ask for the vendor's version before the pilot, not after.
The licence is usually the smallest number in the decision. The biggest is almost always your team's time: setting up, checking outputs and fixing what breaks. Usage dashboards show logins, and our research found that review time is rarely counted when firms judge whether a tool paid off.
The clean way to compare is cost per finished output: everything above, divided by the number of reports, proposals or placements the workflow produces. We set out the method in Cost per Outcome: How to Measure the Real Economics of AI Workflows.
Be realistic about timing too. Deloitte's survey of 1,854 executives found most organisations reported satisfactory ROI on a typical AI use case within two to four years, far slower than the seven to twelve months usually expected of technology spend. A narrow, well-chosen workflow can pay back faster than that. A vague one won't pay back at all.
Here is the method applied to one decision. The agency and every number below are illustrative assumptions so you can follow the maths; swap in your own.
The workflow. A 22-person performance marketing agency sends 35 retained clients a monthly report. Today an account manager spends 3.5 hours per report pulling numbers from ad platforms and analytics, building charts and writing commentary. A senior reviewer spends another 30 minutes. That is 4 hours per report, 1,680 hours a year. At a loaded internal cost of £45 an hour, the workflow costs about £75,600 a year.
The test set. The head of client services picks 30 past reports: 20 ordinary months, six with a tracking problem, and four where the client complained. Each test case holds the raw platform exports and the final report as sent.
The four options.
The scores. After running the 30 test cases through each option, the team scored them like this:
The full cost. Here is where the scores come from on cost. Time is the number that moves.
Assumptions: 420 reports a year at £45 per internal hour. Team hours include setup in year one (A 20, B 40, C 30, D 20) and maintenance (B and C 2 hours a month, D 6 hours a month). D's cash cost includes the £9,000 build in year one only. All prices are illustrative, not vendor quotes.
The decision. The built-in assistant loses, even though it is the cheapest to buy, because it doesn't touch the slowest part of the job: pulling the numbers. Automation and the custom build tie. The specialist tool writes good commentary but locks the agency's templates inside a product it can't easily leave, and its per-client price grows with every new account.
The agency ships option B first. It cuts the workflow's cost roughly in half in year one with the lowest risk, and needs nobody new to maintain it. Three months later, once someone owns the connectors and the AI-drafted commentary has passed the 30-case test set, it adds the commentary step and becomes option D. That sequence is common: get the data flowing with plain automation, then put AI on the part that needs judgement or writing. Our No-Code AI Agency Playbook walks through the reporting build in more detail.
Default to buying, and build only where the workflow is part of what clients pay you for. MIT NANDA's 2025 research found that tools bought from specialist vendors and partnerships succeeded about 67% of the time, while internal builds succeeded about a third as often. Building has become cheaper with no-code tools and coding assistants, but the upkeep hasn't gone away. Our research found one agency that replaced its subscriptions with in-house applications, which only cuts cost if someone owns the maintenance.
If you bring in a partner, run them through the same method. A good one will welcome it. Ask:
Be wary of any partner whose answer to every workflow is the same platform. Gartner's warning that escalating costs, unclear business value and weak risk controls will sink a large share of agentic projects applies to the projects partners sell as much as to the ones you build yourself.
Once the scorecard has narrowed things to one or two options, run a short pilot on live work with a clear finish line. Four to six weeks is usually enough for a monthly or weekly workflow. Before it starts, write down the number that means yes (for example, under two hours per report with no more than one correction per review) and the date you will decide. Then decide on that date. Pilots that drift into permanent trials are how firms end up paying for six subscriptions.
The scorecard holds across service businesses. What changes is the usual first workflow and the criterion that tends to decide it.
The Orbflo findings cited here come from our study of 105 first-hand discussions from service-business communities, collected and coded between October 2025 and October 2026. Each discussion was counted at most once per theme. It is a qualitative sample of public posts, mostly from agencies and consultancies, so the counts show how often a problem came up, not how common it is across UK SMEs. Where we describe what firms did in response, that is what posters reported, not an outcome we measured.
Pick one workflow that felt harder than it should last month. In a free 30-minute session, we’ll map it with you and sketch an AI-native redesign, before you spend anything on tools.
Start with one workflow, not a tool. Write down its trigger, inputs, output and reviewer, collect 20 to 50 real past examples, and test each candidate on them. Compare the AI in your existing software, ordinary automation and a new tool against the same weighted criteria, and pick the one with the lowest full cost per finished output that passes your quality checks.
As few as the work requires. Most small firms do best with the assistant built into their main software, plus one or two tools aimed at specific workflows that clearly pay for themselves. Every extra tool adds a login, a data flow and a maintenance job, so add one only when it wins on your scorecard.
Long enough to cover several real cycles of the workflow, which for weekly or monthly work usually means four to six weeks. Set the success measure and the decision date before the pilot starts, and stop or commit on that date.
The AI Operating System Scorecard is a diagnostic tool that measures whether your business is structurally built to make AI compound, across nine dimensions including how decisions get made, how clearly your processes are defined and how your team is using and integrating AI.
The output is a clear view of where your biggest leverage gaps are and where to focus first.
One practical AI operating-system insight every week.
No fluff, no spam.
© 2026 AV Ventures Limited trading as Orbflo | All Rights Reserved
