CIO.com · Column

You're Measuring AI Adoption. You Should Be Measuring AI Abandonment

Adoption dashboards show who showed up. Cohort retention shows which AI tools people quietly stopped using.

Published
Reading time
About 8 min
Authors
Nishant Virmani
Read the original

What is it about?

Virmani argues that licenses, activated seats and prompt counts measure distribution, not value, and are usually defined by the vendor. The signal that matters is abandonment: who tried a tool and walked away, how fast, and where the workflow broke.

His method borrows cohort retention from consumer apps: group users by activation month and check how many are still active at weeks 1, 4 and 12. A steep week-one drop points to onboarding or role fit; a slow bleed to week twelve points to workflow friction. At renewal, drop tools nobody adopted, repair decaying ones and concentrate spend on tools that hold their cohort.

Key numbers

  • Worked example: on a 1,000-seat deal, 20% who never adopt plus 20% who drop off means 400 paid licenses nobody uses.
  • Contract asks: define “active user” before signing and require event-level usage export.
  • “The correct abandonment target is not zero.” Mandating usage destroys the signal.

Why read it

  • It is the most immediately actionable piece here: a retention curve for your three largest AI contracts takes an afternoon.
  • It gives you a defensible way to cut or expand AI licenses at renewal.
  • It warns against treating retention as proof of value; pair it with delivery metrics.

Who should read it

  • CIOs with enterprise AI licenses up for renewal
  • IT procurement and vendor managers
  • Owners of internal AI platforms

About the author: Nishant Virmani

Staff Technical Program Manager, Google

Virmani has spent more than 15 years in engineering and program leadership across Google, YouTube and Apple. He started at Google in 2012 as a test engineer, moved to Apple, where by his account he managed iOS and macOS releases, and returned to Google in 2018.

His work there has covered Gmail release management, Workspace data governance and AI adoption. This column is his first byline on CIO.com.

Read it carefully: common misquotes

  • The 400-license figure is a hypothetical, not survey data.
  • The column cites METR's early-2025 finding that experienced developers were 19% slower with AI; METR has since labelled that result out of date.
  • The “12,000 shadow AI apps” figure comes from a security vendor's own telemetry.

How it shapes the CIO's AI Playbook

Notes by Pavel Veselov, last reviewed . Summaries are my own; quotes and figures belong to the original authors.

All 11 sources in the CIO's AI Library