AI for CIOs · Playbook v1.0

Building AI in the Enterprise: A CIO's Playbook

A working checklist for the CIO who has been asked to “do something with AI”, written from the CIO's side of the table. Each step says what to do, how you know it is done, and which research it rests on. When new research changes a step, the step and the changelog say so.

I am not writing this as an AI expert. The tools change every few months, and I am learning them together with everyone else. What I bring is 18 years of running IT, on the client side and in consulting, and the last year building with AI myself, hands on.

So the checklist is built on the questions that stay when models change: why we use AI here, who owns it, how we check it works and what it costs. It should still hold when the next model comes out, and it works for any size of company. In a small company one person answers all of it; in a large group each phase has its own owner. If a step doesn't match what you see, tell me.

By Pavel Veselov · 23 steps in 6 phases · Built on 11 sources · Updated

Map of the playbook: six phases in order, Diagnose, Choose, Redesign, Govern, Measure, Fund and scale, with dashed links where an earlier step feeds a later one.
How the steps connect. The retention curve from 1.3 feeds the renewal check in 2.4 and is tracked again in 5.3; the baseline set in 2.1 and 3.1 is what 5.1 measures against; pilots that survive 1.4 fold into one program in 6.1; an agent that 3.3 justifies gets autonomy only after the six controls in 4.1; and cost per transaction (5.2) and token budgets (5.4) answer the scale gate in 6.2.

Phase 1. Diagnose before you build

In the first 30 days, find out what AI the company already pays for, uses and quietly abandons, before proposing anything new.

1.1Find all AI spend, including what sits outside IT

Do:
Pull AI-related invoices, cloud and model bills, SaaS add-ons and business-unit contracts into one view, with an owner for every line.
Done when:
Every AI cost line has a named owner and a stated business purpose.
Why:
BCG's Applied AI Index 2026 puts AI spending at 3.3% of revenue, with about 80% of it outside the enterprise IT budget. A CIO who only sees the IT budget sees a fifth of the picture.

Sources: BCG Applied AI Index 2026

1.2Map shadow AI before buying anything

Do:
Find out which personal AI tools employees already use, for which tasks, and why they prefer them to the official tools.
Done when:
You can name the top five unofficial use cases and the official alternative for each, or the gap.
Why:
MIT's GenAI Divide report found personal AI use at more than 90% of surveyed companies, while only 40% had bought an official subscription. One lawyer kept using ChatGPT instead of her firm's $50,000 tool because it gave better output.

Sources: MIT “GenAI Divide” Report (the 95% Study)

1.3Run a retention curve on your largest AI licenses

Do:
For the three biggest AI contracts, group users by activation month and check how many are still active at weeks 1, 4 and 12.
Done when:
Each contract is tagged keep, fix or cut well before its renewal date.
Why:
A steep week-one drop points to onboarding or role fit; a slow bleed to week twelve points to workflow friction. Define “active user” in the contract and require event-level usage exports, or you are reading the vendor's definition of success.

Sources: Measure AI Abandonment, Not Adoption

1.4Inventory pilots and stop most of them

Do:
List every live pilot with its sponsor, business metric and cost. Stop the ones with no outcome metric or no owner.
Done when:
Only a handful of initiatives remain, each tied to a business outcome.
Why:
In HBR's manufacturer case, 14 use cases from a workshop became 2 rebuilt workflows, and much of the value came from the 12 that were dropped. BCG's AI lead Nicolas de Bellefonds puts it bluntly: “If you see a roadmap with 200 use cases, walk away.”

Sources: HBR: From AI Experimentation to Transformation · BCG Applied AI Index 2026

Phase 2. Choose where to play

Concentrate on a few domains where AI output can be checked cheaply and the organization can actually absorb the result.

2.1Pick three to five domains, not a use-case list

Do:
Choose three to five business domains with a measurable P&L line and map their end-to-end workflows together with frontline people.
Done when:
Each domain has a C-suite owner and a baseline metric.
Why:
HBR's authors find that companies stuck in the “micro-productivity trap” start from long use-case lists; those that escape narrow AI to a few strategic domains. BCG finds future-built companies 3.5× more likely to fund AI as one multiyear program.

Sources: HBR: From AI Experimentation to Transformation · BCG Applied AI Index 2026

2.2Apply the verifiability test

Do:
Score candidate workflows on five traits: text-based, repetitive, with natural human review, limited regulation, verifiable output. Start with the highest scores.
Done when:
Your first wave contains only workflows where a wrong answer is caught cheaply.
Why:
a16z's revenue data shows adoption following verifiability rather than model capability: coding leads because code runs or it doesn't, and support works because the worst case is a hand-off to a human.

Sources: a16z: Where Enterprises Actually Adopt AI

2.3Check that the organization can absorb the result

Do:
For each domain, ask how much of the value chain you control end to end and how many technologies must be combined. Run a pre-mortem against the typical failure for that position.
Done when:
Each initiative names who will put the output into production, not just who builds it.
Why:
GM's generatively designed seat bracket was 40% lighter and 20% stronger, and never reached production because its factories were built for stamped steel. A better AI output is worth zero if the organization cannot use it.

Sources: HBR: Match Your AI Strategy to Your Organization

2.4Decide buy, partner or build for each workflow

Do:
Default to buying or partnering for commodity capabilities. Build where the workflow differentiates you, or where agentic coding makes an internal tool cheaper than a license.
Done when:
Every significant SaaS renewal gets an explicit build-versus-buy check.
Why:
MIT's report found external partnerships reaching deployment about twice as often as internal builds (self-reported). McKinsey found 32% of respondents had dropped at least one software purchase because agentic coding let them build it in-house. See “Where the sources disagree”.

Sources: MIT “GenAI Divide” Report (the 95% Study) · McKinsey State of AI 2026

Phase 3. Redesign the work, not the task

Rebuild whole workflows around AI instead of bolting AI onto existing steps.

3.1Rebuild workflows end to end, led by frontline experts

Do:
Redesign each chosen workflow from a clean sheet with the people who do the work, and remove steps rather than only speeding them up.
Done when:
The new workflow has fewer hand-offs and a non-AI baseline to compare against.
Why:
In HBR's manufacturer case, early quotes went from four to six hours of engineering time to a 20-minute estimate by non-designers, and win rate rose 10 points in three months. Bain's clients call the alternative “paving the goat trails”.

Sources: HBR: From AI Experimentation to Transformation

3.2Capture judgment, not documentation

Do:
Run moderated panels of experienced practitioners through real edge cases. Where they agree, write a rule; where they disagree, record the reasoning as context for agents.
Done when:
Agent context files contain escalation thresholds, exceptions and tone, reviewed by the experts themselves.
Why:
Every company has the same models; the difference is written-down judgment. At ITA Group the first six to seven months of codifying expertise were slow, then software timelines shrank from months to weeks.

Sources: HBR: Teach Your AI How You Make Decisions

3.3Use the simplest architecture that works

Do:
Start with one model call plus retrieval and examples. Move to a scripted workflow, and only then to an autonomous agent, when the simpler step measurably fails.
Done when:
Every agent in production has a written reason why a workflow was not enough.
Why:
Anthropic's engineers found the most successful implementations used simple, composable patterns, and warn that the right answer “might mean not building agentic systems at all”.

Sources: Anthropic: Building Effective AI Agents

3.4Invest in tools more than prompts

Do:
Document every tool an agent uses as you would for a junior developer, and test how the model actually calls it.
Done when:
Tool definitions have examples, edge cases and clear boundaries, and failures get fixed in the interface, not only in the prompt.
Why:
Building its coding agent, Anthropic spent more time on tools than on the prompt; switching to absolute file paths made the model use its tool “flawlessly”.

Sources: Anthropic: Building Effective AI Agents

3.5Budget for people, not just models

Do:
Plan effort at roughly 10% algorithms, 20% technology and data, and 70% people, organization and process change.
Done when:
The AI budget has explicit lines for training, change management and workflow redesign.
Why:
BCG's 10-20-70 rule. If 70% of the effort is organizational, running AI is mostly change leadership, and an AI team staffed only with engineers is set up to stall.

Sources: BCG Applied AI Index 2026

Phase 4. Govern agents like a workforce

Put controls, ownership and decision rights in place before agents get autonomy.

4.1Install the six controls before granting autonomy

Do:
Governed data and memory access; human-in-the-loop for high-risk decisions; testing with shut-off and rollback; clear hand-off interfaces; a named owner for every agent; ongoing security.
Done when:
No agent acts autonomously in production without all six in place.
Why:
42% of companies expect to give agents real autonomy by 2030, but only 5% have all six controls today. Companies with all six report three times the agentic value of those with one.

Sources: BCG Applied AI Index 2026

4.2Give every agent a human owner and a lifecycle

Do:
Assign an owner to each agent and manage it through onboarding, evaluation, retraining and retirement, the way you manage staff.
Done when:
There is an agent register with owner, scope, permissions and last review date.
Why:
MIT SMR and BCG describe agentic AI as “a tool that learns and a worker that is owned” and recommend an “HR for agents” function; 76% of respondents already see agents as more like coworkers than tools.

Sources: MIT SMR–BCG: The Emerging Agentic Enterprise

4.3Share decision rights across business, HR and IT

Do:
Business units own outcomes, HR owns how agents change roles and skills, IT owns platforms and security, and they decide together.
Done when:
Agent decisions no longer sit with IT alone.
Why:
HBR's authors argue that agents are digital labor governed jointly by business units, HR and IT; MIT SMR and BCG reach the same conclusion from their survey and interviews.

Sources: HBR: Teach Your AI How You Make Decisions · MIT SMR–BCG: The Emerging Agentic Enterprise

4.4Give whoever runs AI real authority

Do:
Whether the CIO runs AI directly or through an AI lead, the role needs budget, decision rights over the AI portfolio and a CEO-level sponsor.
Done when:
The owner of AI can stop a project and move its budget without asking permission line by line.
Why:
Gene Rapoport, one of the HBR article's authors, on AI leaders without power: “If they just have a title, then they end up being a mascot.”

Sources: HBR: From AI Experimentation to Transformation

Phase 5. Measure what the P&L sees

Replace adoption dashboards with outcome, cost and retention metrics that finance recognizes.

5.1Measure outcomes against a non-AI baseline

Do:
Track the business metric each workflow is meant to move (win rate, conversion, cycle time, cost to serve) against a comparable process without AI, with finance involved from day one.
Done when:
Each initiative has a P&L line, a baseline and an owner in finance.
Why:
BCG finds companies that track AI value in the P&L realize three times the value (3.6% vs 1.2%). McKinsey's gap between 80% feeling more productive and 37% seeing EBIT impact is what happens without this step.

Sources: BCG Applied AI Index 2026 · McKinsey State of AI 2026

5.2Track fully loaded cost per completed transaction

Do:
Include model fees, compute, data lookups and the human time spent on exceptions, and compare with the manual baseline every month.
Done when:
Cost per transaction sits on the same dashboard as speed.
Why:
Richard Ewing audited an award-winning workflow that was 90% faster and cost $12–14 per document against about $0.80 manually; 40% of files still needed a human, who took twice as long as before.

Sources: The AI Case Study That Left Out the Cloud Bill

5.3Track retention, and don't mandate usage

Do:
Report cohort retention at weeks 1, 4 and 12 for every major AI tool, and resist making usage mandatory.
Done when:
Retention is reported next to delivery metrics, not instead of them.
Why:
“The correct abandonment target is not zero.” Mandated usage destroys the only honest signal of whether a tool fits the work.

Sources: Measure AI Abandonment, Not Adoption

5.4Treat tokens as a design constraint

Do:
Budget horizontal tools such as chat assistants as a cost of doing business, and manage agentic and coding workloads for ROI with a token budget per workflow.
Done when:
Every agentic workflow has a cost ceiling and an owner who watches it.
Why:
About one in five McKinsey respondents say AI operating costs already constrain them, and high performers hit cost limits on coding agents three times as often (18% vs 6%). Per-token prices fall; consumption grows faster.

Sources: McKinsey State of AI 2026

Phase 6. Fund and scale

Scale only what holds up at production volume, under one program with one owner.

6.1Run AI as one multiyear program

Do:
Fold scattered pilots into a single enterprise program focused on a few high-value workflows, with a C-suite owner and short review cycles.
Done when:
There is one AI portfolio, one owner and one review rhythm.
Why:
61% of BCG's future-built companies fund AI as a single multiyear program, against 17% of laggards.

Sources: BCG Applied AI Index 2026

6.2Pass a scale gate with three questions

Do:
Before scaling, answer: what does one completed transaction cost against the manual baseline; how often does a human intervene, and at what cost; do the economics hold at production volume?
Done when:
No pilot moves to production without written answers signed off by finance.
Why:
Pilot costs are often paid from central innovation funds, so the business unit never sees the bill until the project code closes. In Ewing's audit, the economics turned negative within 60 days of full production.

Sources: The AI Case Study That Left Out the Cloud Bill

Where the sources disagree

Good research contradicts itself. These are the open questions and how the playbook resolves them for now.

Is enterprise AI failing or working?

Playbook position: They measure different things: P&L impact, paid deployment and self-reported EBIT. For a CIO, the industry rate matters less than whether your own workflows beat their baselines (step 5.1).

Should you buy or build?

Playbook position: Buy for commodity capabilities and speed; build where the workflow differentiates you or the vendor product is a thin wrapper. Re-check at every renewal (step 2.4).

Does governance slow AI down?

Playbook position: Controls are what let you grant autonomy at all. Without them, agents stay stuck in low-risk pilots (step 4.1).

How this playbook stays current

I read new research on enterprise AI every week. A piece joins the library only if it should change a decision. Then every step is checked against it: does it confirm the step, contradict it, or add a new one? Changes are logged here with the source.

v1.0 ·

  • First version: 23 steps in six phases, built from the 11 sources in the CIO's AI Library.
  • Cost per transaction (5.2) and the scale gate (6.2) come from Richard Ewing's CIO.com audit column of 8 October 2026.
  • Retention metrics (1.3, 5.3) come from Nishant Virmani's CIO.com column of 9 October 2026.

Need a second set of hands to run this?

I work as a right hand to the CIO: I can own the AI workstream (spend audit, portfolio, vendor claims, governance, the business case) while the CIO stays on strategy and stakeholders.

Discuss your AI agendaRead the 11 sources