Designing for usage-based AI pricing
Five weeks getting a product ready for a new way of charging for it.
- Problem
Icertis was moving from a flat rate to charging for AI by the amount used, and wanted to know whether the tools they had would hold up. I tested the one they used for reading usage, and four of the nine people could not finish a task on it.
- Approach
I measured the tool they already had before designing anything, traced the failure back to the pricing change, then built one view per audience rather than one dashboard with filters.
- Impact
Task success went from 5 of 9 to 9 of 9, and average time from 5 minutes 36 to 1 minute 50.
Icertis was about to start charging for AI by the amount used, and wanted the software ready before that happened.
Their current screens answered the questions a flat rate asks. Under the new model, people would be asking different ones.
- 9 colleagues tested, twice each
- 4 gave up on the tool they already had
- 3 prototypes, one per audience
- 50+ screens in five weeks
So what was actually wrong?
Moving from a flat rate to usage-based pricing sounds fairer, and it is. It also means the bill is different every month, which changes what three different groups need to be able to see.
Revenue stops being a subscription line and becomes a variable one, swinging with how much customers happen to use. Forecasting and cash flow get much harder to plan against.
Invoices can arrive far bigger than expected. Finance teams either over-buy credits and waste budget, or under-buy and get cut off partway through work.
Nobody on either side of this contract had worked this way before. Under a flat rate you never needed a mental model for consumption, so nobody had built one.
All three come down to the same thing.
A price that holds still and a price that moves are not the same product to the people paying for it.
One asks whether you are getting value from something you already bought. The other asks whether money is leaking right now.
There was already a tool. Was it working?
My instinct was to look at the tool, list what was wrong with it, and design something better.
Then I realized it had never been tested against what the new model would ask of it. It worked well for the old one, and they wanted to know if it could work for this one as well.
So I ran it as a study before designing anything. Two tasks, straight from what the team does every week, with .
Can you find X usage for company Y between date A and date B?
Can you find X usage for company Y's use of product Z between date A and date B?
Four of the nine gave up before finding an answer.
Why was it so hard?
Apart from the layout, there were two functional reasons.
The first is that it was built for the pricing model they were leaving, not the one they were moving to.
Answers: how many seats are we paying for, and when does the contract renew?
Answers: what should we stop doing today, before it costs us another $4,000?
An empty seat costs you nothing extra. A job left running keeps charging you by the hour.
The second reason is that it was built for one reader, and three groups needed it in different ways.
Acme Corp · March · 11,400 sessions · $386K metered
Sorted by which accounts are behaving oddly, not which ones are biggest. A big number tells you nothing on its own. You need to reach the session behind it.
The same spend expressed against what was bought. A percentage is actionable in a way a dollar figure is not, because it implies a date.
No dashboard, just one estimate of what this job will cost, shown before they run it. Afterwards there is nothing left to decide.
One dashboard with filters doesn't fix this. Each group needs a different thing to be the biggest thing on screen.
So I made three of them.
I can't show the screens, but I can tell you what each side needed to stop being afraid of.
Customers weren't afraid of the price. They were afraid of the surprise. So most of the work was about letting them see the number coming.
- A soft landing. Three months where they saw what the new bill would have been while still paying the old rate, so they could learn what their usage costs before it cost them anything.
- A ceiling they set. Throttle or stop at a number they chose, so they know the worst case before it happens.
- A warning while it still helps. An alert when spend runs well above their own normal, while there's still time to do something about it.
- Words that explain the model. A lot of this was UX copy, to move people onto the newer pricing model without bombarding them with everything at once.
Internally the ask was the opposite: a space where every number on screen could be explained. I did that by cutting the data three ways.
- By customer. Is this account accelerating past what they committed to, or has it stopped? One tells you about growth and the other about churn.
- By product. Which parts people actually consume, and whether a feature earns more than it costs to run.
- By time. What the month looks like before the month is over, and where the peaks are.
Any two of them can be held still while the third moves. Fix the company and the month, and you see which product drove the bill. Fix the product and let time run, and you see whether a spike is a habit or a one-off. Every one of those used to be a data request and a few days of waiting. Now it is a few clicks.
Each group got one view built around their question, instead of one dashboard trying to cover all three.
Did it help?
I ran the same nine people through the same two tasks again.
5 min 36 on average
1 min 50 on average
The time saving matters less than the finishing. Slow is annoying. Giving up means someone else has to find the number instead, usually by asking a colleague to pull it for them.
What happened after I left is confidential. What I can say is that the questions people asked got better.
Can someone pull the usage for this account?
Why did this account spike on the 14th?
Three things I took from five weeks.
-
I measured the old tool instead of critiquing it.
I nearly skipped this. Listing problems with the existing tool would have taken an afternoon and I would have been mostly right. Measuring it took a week and gave me a number nobody could argue with, which is why the redesign got built instead of debated. I ask what the current thing scores now before I argue for a new one.
-
I looked upstream of the screen for the cause.
The tool was answering a flat-rate question in a usage-based world. I only got there by asking what the business had changed, which I had not thought of as a design question before this.
-
I let engineering change what I drew.
Meeting them every two weeks meant I learned early what the data could and could not do. A lot of ideas died in those rooms. The ones that survived were worth prototyping, and I would rather lose an idea in week two than in week five.
Nine colleagues, two tasks, and four of them gave up on a tool they used every day.
Three prototypes later all nine finished, in a third of the time, and the question in the room changed from asking for the number to asking about it.
I couldn't have done this without the help of my colleagues and seniors at Icertis. My manager guided me throughout the internship, a design senior showed me how design actually gets made inside a company this size, and the engineering team were patient enough to explain the data model and answer my questions.
Making invisible work visible
The planning behind every household chore has no name, so couples cannot discuss it or divide it. Five design guidelines, from three studies with nine couples.