As delivered — a presentation by Elijah Schwartz at InstructureCon 2026 in Louisville, Kentucky.
Case Study
Getting 50,000 Students to Actually Use AI
Kaplan built an AI “Explainer” into the moment students struggle most — and iterated relentlessly on how students actually adopt it. The power was in the customization: a platform Kaplan fully owns, runs on any model, at a fraction of the cost of any alternative.
“We partnered with a great company: ibl.ai.”
Elijah Schwartz
Executive Director, Innovation, Insights, & AI Strategy — Kaplan
~50,000
Students reached
100% owned
Kaplan owns the code and data
Any LLM
Model-agnostic by design
A fraction
of the cost of any alternative
A method and a bet, shipped to ~50,000 students
At last year’s InstructureCon, Kaplan shared the idea: put an AI Explainer right at the question-review moment — where the friction (and the learning) actually lives — so a student who just missed a question can get it explained on the spot. The design was grounded in learning science: Bloom’s 2-sigma, effortful retrieval, Dunlosky, Fogg, cognitive load theory, and the zone of proximal development. And Kaplan shipped it, to about 50,000 students.
It “worked” in the pilot — and 84% never found it
The quality was there: roughly 95% of responses landed, and students who used it rated it 4.7 / 5. But 84% of the students Kaplan could measure never used the feature. They just never clicked. As Elijah put it: a feature nobody finds has a flatline efficacy. The problem was never the AI. It was discoverability and adoption — a design problem hiding behind good pilot numbers.
Six mistakes about surfacing it — and they apply to any feature
None of these are AI problems. They are the quiet ways a genuinely good feature fails to reach the people who need it most.
We counted impressions, not intentions
96.5% of the “messages” were the chatbot saying hi — 206,328 system “hellos” and 6,227 launches that produced zero questions. The dashboard counted every greeting as “usage,” so the numbers looked on fire while the room was cold. The fix: count intentional actions, not impressions.
We didn’t patch the leaky funnel
From everyone given the feature, the drop-off was brutal — down to single digits who came back a second time. The fix: log the fall-off flows and plan interventions where students actually leave.
We thought “present” meant “found”
The icon was right there on the page, every single visit — but median usage was one session. Being on the screen is not the same as being used. The fix: teach students how to use it, right away.
We named it for us, not for them
“AI Tutor” is a marketing name (“I don’t need a tutor — why would I tap that?”). The job students actually wanted was “Explain This” — make the question I just missed make sense. The fix: name it for the job it does.
We handed them an empty box
Open a blank chat and students freeze — “…now what?” Kaplan knew the tricky topics but didn’t surface them. The fix: always show the first move.
We added friction on purpose — then fixed it with one copy edit
A wall of rigid legal consent text got 64% of students through the gate. The policy was locked, so Kaplan just rewrote the attestation so a human could read it — and acceptance went to 100%. The fix: every click is a barrier; be strategic.
A human pointing at it
The single biggest jump didn’t come from a new button. When an instructor pointed at the tool and said “here’s how to use this,” students answered 253 questions — versus 148 without. Same feature, same page. Just a human pointing it out. Kaplan’s takeaway:
Adoption is a social act before it’s a UI act.
What moved it, what helped, what didn’t
Kaplan tested the whole flow — new copy, welcome nudges, relaunch emails, instructor intros — and scored each lever. The pattern is clear: the levers that won all added a reason, a first move, or a human.
| Lever | Verdict | What happened |
|---|---|---|
| A human points at it | Moved it | 253 vs 148 questions asked |
| Plain-language consent | Moved it | 64% → 100% past the gate |
| Show the first prompts | Helped | Fewer closed opens, 2× more first questions |
| Icon, placement, naming | Helped | Necessary, but not sufficient on its own |
| In-app nudge at likely friction | Maybe | Promising signal, still measuring |
| More announcements | Didn’t | A second email is still an email |
Kaplan owns the platform — so it could iterate relentlessly
Every fix above — rewriting the consent copy, changing the first-move prompts, renaming the feature, wiring in instructor intros, testing the whole flow with pilot-level rigor — required changing the product, fast, and on Kaplan’s terms. That is only possible with full ownership of the code and data — no tickets to a vendor’s roadmap, no waiting on someone else’s release cycle.
A rented SaaS tool
- Copy, prompts, and flows gated by the vendor
- Locked to one vendor's model and pricing
- Per-seat pricing punishes reaching every student
- You wait on the vendor's release cycle to iterate
ibl.ai at Kaplan
- Full source-code and data ownership — iterate at will
- Run any language model; swap freely as the market moves
- Exponentially lower pricing than any per-seat alternative
- Ship changes on Kaplan's schedule, not a vendor's
Full ownership
Kaplan owns the code and the data — iterate at will, on Kaplan's terms.
Any language model
Model-agnostic by design — run and switch any LLM as capability and price change.
A fraction of the cost
Exponentially lower pricing than any per-seat alternative — so reaching 50,000 students doesn't blow up the budget.
Elijah Schwartz at InstructureCon 2026
The complete deck as delivered on stage in Louisville, Kentucky. Click any slide to enlarge.
Where does a student in your program most need a nudge?
Kaplan’s closing question to the room: where in your course does a student most need help — and nobody’s there to give it? If you can point to it, ibl.ai can build the support for it — on a platform you own, on any model, at a fraction of the cost.