Implementing AI in Small to Midsized Businesses →

What Mid-Market Companies Get Wrong When They Try to Implement AI

Upwork's 2026 SMB data shows 74% of companies say AI improved productivity — but most gains are under 25%. The bottleneck is not the AI.

Upwork’s Q1 2026 survey of small and mid-market business leaders found that 74% of respondents say AI improved their productivity — but for most, those gains have not yet crossed the 25% mark. That gap between adoption and impact is not a technology failure. It is a sequencing failure.

At H&R Block, I was brought in for a short, laser-focused engagement — re-architect the SOA for TaxCut 2007, the company’s flagship consumer tax software product. The scope was precise. The expected output was precise. I absorbed the existing rules engine and complex tax-quality-assurance logic, then built a multi-layered solution architecture that supported the next wave of product development. Because the scope was defined before I started, the work was measurable and the outcome was clear.

That level of precision — specific problem, specific outcome, defined success criteria — is what most mid-market AI implementations are missing.

quadrantChart
title Mid-Market AI Use Cases: Effort vs. Business Impact
x-axis Low Implementation Effort --> High Implementation Effort
y-axis Low Business Impact --> High Business Impact
quadrant-1 Start here
quadrant-2 Plan for 6-12 months
quadrant-3 Skip for now
quadrant-4 Automate quickly
Document extraction: [0.3, 0.72]
Customer support routing: [0.25, 0.55]
Sales forecasting: [0.72, 0.85]
Invoice processing: [0.35, 0.65]
Meeting summaries: [0.15, 0.3]
Custom model training: [0.88, 0.72]

The 74%/25% Gap Is a Process Signal, Not a Technology Signal

When three-quarters of companies say AI helped but most gains are modest, the common interpretation is that AI needs to improve. The more accurate interpretation is that most companies have not yet created the conditions for AI to perform well.

AI performs well when the input data is consistent and clean, when the task is well-defined enough that the output can be evaluated without manual review, and when the workflow around the AI is designed to capture the efficiency gain rather than absorb it into a new verification step.

Most mid-market AI implementations violate at least two of those conditions from the start. Companies deploy AI against document processing workflows where the documents are inconsistent and poorly formatted. They deploy AI to generate customer communications without defining what good output looks like. They deploy AI in sales workflows without the clean CRM data that the models need to produce useful forecasts.

The AI runs. It produces output. Someone reviews the output manually to see if it is right. The productivity gain disappears into the review step.

The Right Scoping Question

The correct question before any AI implementation is not “what can we automate?” It is “what do we have that is ready to be automated?”

That distinction matters because readiness is not obvious. A process that seems like a good candidate for AI — repetitive, data-heavy, time-consuming — may have hidden complexity that makes it a poor near-term implementation target. Legal document review looks repetitive but requires judgment calls that current AI handles inconsistently. Customer support routing looks simple but depends on taxonomy and ticket classification that most mid-market companies have not standardized.

A quick readiness assessment across potential use cases reveals a shorter list — but a list where implementation will actually succeed. That shorter list is where to start.

What “Start Small” Actually Means

Every consultant says “start small.” In practice that advice is too vague to act on. What it should mean, concretely:

One use case. Not a category of use cases. Not a department. One specific task, with specific input data, a specific expected output, and a defined threshold for what counts as successful performance.

One team. A point of contact who owns the use case and is accountable for the implementation. Not a committee. One person with authority to make the decisions required to get the implementation to production.

A 90-day scope. If the use case cannot reach a meaningful production state in 90 days, it is not small enough. Break it down until it is.

The organizations that break this pattern — that start with a broad AI strategy, multiple initiatives, and a 12-month roadmap — consistently deliver results that do not match the investment. The ones that pick one thing and do it completely are the ones with concrete ROI to point to six months later.

The Data Problem No One Wants to Fix First

The most common thing I see in mid-market AI evaluations is this: the data required to support the intended AI use case is not in the state the implementation requires. It is in the wrong format, spread across too many systems, missing key fields, or not clean enough to produce consistent outputs.

The temptation is to proceed anyway — to implement the AI against messy data and plan to clean the data later. That approach produces a system that gives inconsistent results, erodes trust in AI among the users who interact with it, and ultimately gets abandoned or replaced.

Clean the data first. It is slower. It is less visible. It is the right sequence.

The Competitive Window Is Real — and Shorter Than It Looks

A TechRadar Pro analysis of mid-market AI adoption makes the point that mid-market companies are in a useful position right now: the tools are mature, but most competitors have not yet deployed them effectively. That window is real. It is also shorter than the articles describing it tend to suggest, because adoption in this segment is accelerating.

The way to take advantage of it is not to start as many AI initiatives as possible. It is to pick the right one — the one where the data is ready, the process is defined, and the ROI can be measured — and execute it well enough that the internal credibility for the next initiative is already established by the time you are ready to start it.

At H&R Block, the focused scope of the SOA re-architecture was not a limitation. It was the reason the engagement worked. A defined target, executed completely, produces results that a broad initiative never does. That principle is the same whether the project is a tax-software rebuild or a mid-market AI implementation in 2026.

Frequently Asked Questions

What is the most common AI implementation mistake in mid-market companies?

Adopting AI tools before addressing the data and process foundation they require. AI works best on clean, structured, consistent data — and on well-defined processes where the inputs and expected outputs are clear. Most mid-market organizations have neither in the areas they most want to automate. The result is AI that runs but produces outputs that require manual review to validate, eliminating the efficiency gain the adoption was supposed to create.

How long does it realistically take a mid-market company to implement AI effectively?

For a well-scoped, single-use-case implementation with adequate data preparation, three to six months from scoping to production is realistic. Organizations that try to run AI broadly across multiple business areas simultaneously consistently take longer and achieve less. The companies getting real, measurable ROI from AI in 2026 are the ones that picked one use case, prepared it properly, and implemented it completely before expanding.

Do mid-market companies need to hire AI expertise internally to implement AI?

For most mid-market companies, no — not initially. The evaluation, scoping, vendor selection, and initial implementation of a well-defined AI use case can be run with fractional technology leadership and existing staff who understand the business process. Internal AI expertise becomes necessary when the organization is running multiple AI programs simultaneously, when governance requirements are complex, or when custom model development is on the roadmap. Most mid-market companies are not there yet.

Shawn Livermore — Fractional CTO & Chief AI Officer
About the Author

Shawn Livermore

Fractional CTO and Chief AI Officer with nearly 3 decades of enterprise architecture experience. Clients include Kelley Blue Book, LERETA ($18B property tax processor), First American Financial, Carvana, WellPoint/Anthem, and PacifiCare. 92 client reviews, 5-star average.

View full background →

Need a fractional CTO or CAIO?

Technology leadership without the full-time headcount. Engagements start with a conversation.

Man writing a flowchart diagram on a whiteboard with a blue marker.