How Do We Measure The Things That Don't Happen and Why That Matters in an AI World and to the Economy
That is a fundamental problem inside companies and a bigger problem in times of innovation and innovation-driven change, like the 4th Industrial Revolution that is upon us.
I have seen smart, innovative bets killed by senior execs whose best skill is knowing how to stay in their jobs. I am not saying these are bad people or leaders; in fact, some were actually great, but over 20 - 30 years at the same firm were ninja black belts at keeping their jobs/position, which is kind of a tautology. A part of that is knowing that risky, innovative program bets that fail get you negative marks while there is no ledger, no repercussions for making 'the hard choice' to kill a program idea before it gets out of the barn. The easiest way is to kill it during the gestation period, which at most firms is the funding cycle.
This is really the board and CEOs' fault for having an environment where innovative bets are only rewarded if they pay off. More on that in the book Anna Catalano and I are writing. You could also go deep on that by reading genius Annie Duke
At the end of this article, there is a paragraph about another amazing example of AI outperforming humans from the super useful Substack from Azeem Azhar.
But ...
A lot of what I see about AI is about hitting and beating benchmarks. That is important and informs our understanding of the trajectory and intensity of this change wave.
Understanding that is a key input to the strategy refresh moment all boards should have now.
But...
The economic impact and disruption at all levels isn't about AI performing ahead of the top performers.
For example, in Q4, the NYC area usually recruits 100,000 new grads for marketing analysis, consulting, broker positions, and lawyer positions.
AI can make all of those 30% more productive, and stand-alone AI actually outperforms the bottom half (by imputed performance). Ironically, the output quality seems to improve more when teamed with AI for the better performers.
Lump all of that together and 100,000 new hires may turn into this:
Top performer (25k openings) improvements from AI reduces that to 12.5k
Based on output and quality, the bottom half performers total 50k, which becomes 15k. Their impact is more significant because they have more human overhead (training, mentoring, fixing), but less output.
Another 25% or 25k for smarter process and resource leverage redesign because we are interspersing it all with AI (re: Agentic AI). Having spent decades untangling long-evolved, often idiotic, undocumented horizontal process flows at four large, not dumb, companies, my guess is that 25% is very low.
So maybe we hire just 27.5k of the 100k.
Back in the Real World, How Does that Look
In the real world, a smart CFO will be looking at a 2 -3 year horizon and either pause, freeze or shrink hiring in anticipation of this. This will be even more draconian/cautious if they have lost visibility to the top line over the next six quarters or so. I'll talk about that later.
This will be deflationary in three ways:
impacts to employment, which reduce demand/spend,
wage/salary pressure
increased unemployment which also takes energy out of the economic system beyond its demand side deflationary impact.
It's good that these things aren't also happening:
The cusp of the biggest advances in history in robotics.
A healthcare coverage reduction. Reduction in healthcare coverage also pulls energy out of the economic system. Healthcare spend does not have the same primary input impact as, say, consumer durables spend. It is also usually concurrent with a reduction in primary inputs into production (eg, when people are sick).
Spend reduction in foundational research just as China increases theirs, and the cycle time between the research and market impacts continues to shrink to record levels.
Oh wait..yes all of those things are happening.
Worth noting healthcare deterioration has a negative economic multpiler because of the energy it sucks out of the economic engine and increased defense spending does that also simply because it is decidedly not labor intensive.
You then have to be very careful about supply side price shocks to avoid creating a new lower equilibrium point for the economic engine, as beyond a certain point, it is not self-stabilizing like a chemical system; it can actually reverse the flywheel itself on a downward path for some time. The worst thing to create more reduced economic activity is reduced economic activity.
Two sets of choices drive economic activity in a consumer-anchored nation.
Economic Choices Drive the Economy (Precisely Two Choices)
Yes I know how that sounds.
I. Consumer
Consumer spending is a function of consumer choices, which is a function of consumer sentiment. If consumers worry about the economic future, they treat discretionary spending differently and defensively. Of course, this results in stagnating and deflationary spending over a long period of time. Consumer choices shouldn't just be measures of what they do, they should be measures of what they chose not to do. Again, the difficulty of measuring what didn't happen.
II. CFOs
Every company runs off a business plan/model. The top line is the basis for anything else you do with that model. When CFOs lose visibility to that top line, the rest of the model is circumspect and requires caution. Then the type of spend constraints, freezes, pauses I mentioned above happen. Again, the difficulty of measuring what didn't happen.
That is stagnating and deflationary if it persists. It also feeds the consumer side in two negative ways. One is purely on sentiment, which is what really drives choice, and the other is in real aggregate purchasing power, which is salary/wage based.
Many people talk about uncertainty and I think that definitely plays a part in the portfolio theory-infused roulette, which is the stock market, but I think the economy runs off visibility, not uncertainty. If you studied the history of economic thought, especially from an epistemological perspective, you would feel deep uncertainty about anything connected to our economic modeling. (Yes, I studied that, yes I am a dork - on authority from my Gen Z daughter. I think she means that in a good way).
So, in these times of accelerated AI-anchored change, managing the talent supply chain against the job demand side is more important than you think. Ensuring longer-term economic visibility and doing more things that improve it and less that do the opposite is important.
Prologue Singapore
My most awesome Gen Z daughter, who lives horribly far away in Singapore because she fell in love with and married a wonderful Singaporean chap, sent me a picture of a giant poster advertisement at a local mall. The poster features a greying man who is 50-ish. It pitches a service that promises to retrain you for the AI world and move you into a new tech-intensive job pool. (NJ to Singapore is 9952 miles.)
This is what sparked this rant from very useful Azeem Azhar
Can AI innovate?
Sakana’s AI Scientist produced the first fully AI-generated paper to pass the same peer-review process as human scientists at an ICLR workshop. The system submitted three papers and – organizers knew, but reviewers were blind to their origins in a double-blind setup. One scored a 6.33 average, beating the acceptance threshold and outshining many human entries. It’s important to note that workshop acceptance rates (60-70%) are much higher than main conference tracks (20-30%) – but it’s still a sign of progress in the right direction.
