"Told You So Tuesday" in New Jersey (not a real thing). China , AI and the distant future, like Q3, 2025
July 2024
On this platform six months ago, I said:
"In the distant future, like next year, we will move into the rocket ship curve for GenAI. On the hardware side, the architecture and 'software' that runs it will become 200X+ more efficient."
The Distant Future - Q3 2025
So, I think three things about DeepSeek's near future, my opinion:
This space is moving fast, so I would bet only a portion of the clever development that the team has done is in this release.
The data exhaust and analysis that is possible once this is live in the real world at scale (which is now) is hugely valuable in helping engineers, designers, and architects make further advancements.
Typically, advances of this sort are significantly faster, cheaper/more efficient, and functionally equal or better and also require less work/cost to advance and upgrade, i.e., increased agility.
So what that means is this is just the beginning. This will have a self-improving flywheel effect further fueled by it being open source.
This also means that, in some ways, chip design/architecture has missed a fork in the road. While the H100 is clearly superior in the ordinary sense to the H800, it is a progression anchored in the chip and its capability. DeepSeek used the 'lesser' H800 chips in part because the advanced H100s were embargoed. Chips are a component of larger 'solutions'. Improving that more extensive solution isn't always about brute force computational power at the chip level. Doing an end-to-end optimization of the larger solution has historically allowed innovators to be leapfrogged.
Making a next-gen 'descendant' of H800s that is three times more efficient per unit dollar and per dollar/watt is the next best lucrative step. Someone will do that.
What Does History Tell Us?
History is full of examples of impressive Wave One innovation companies that were leapfrogged (often destroyed) by Wave Two.
This is done by taking a hard look back and saying what we would/could we do differently today from a model, architecture, design, and technology perspective to leap from Wave One to Two, not to make an incremental gain but a game-changer.
There is a brilliant invention/innovation like Ford or Standard Oil a hundred years ago. Technological advances are often available to Wave Two, like in-memory processing that was adopted (plus a complete rethink), leapfrogging Peoplesoft (by the way, same founders with an 18-year gap).
Ocient playing at the software/architecture level used readily available hardware (NVMe drives) to create a hyperscale data platform for analytics (petabytes to exabytes) that was 500x more efficient (five years ago). Yes, 500x. The redesign of the steel industry around innovations like the Electric Arc Furnace made the US and Ruhr Valley global steel leaders, killing British Steel slowly. That is partly because it's so hard to marshall the investment to rearchitect existing stuff that is working perfectly well despite evidence that a Wave Two is on the horizon. To some extent, Docker, Uber, Google, Shopify, early AWS, and maybe early Palantir and Salesforce are examples.
Why is this so Hard:
Product Management/Development is hard work. A tit-for-tat market battle delivering competitive feature function. It's a hard job.
The actual winning recipe, though, is, at the right time, redesigning your product factory for efficiency, utility, and innovation with a clean sheet view on model, architecture, financial structure, and output/throughput optimization.
In poker terms, this would be like betting the last card on the board would be a king so you can complete your straight and win the pot. Sounds like a dumb bet. It is a dumb metaphor, but not a dumb bet. What if I told you the bet was X and the pot was 1000x? Theoretically, you would make that bet every time (52 cards, 4 kings). The problem with the metaphor is that while the cost /return might be sized accurately, the probability isn't because this is so very hard to do. You can calculate the cost (bet) and return (pot), but you have to factor in, in the real world, that this is the very deep end of the pool and requires the right leadership, vision, culture, and talent, which is rare, very rare.
Often, the ideas seem stupid. For example, a hub-and-spoke system of jets for packages was assessed by smart people to be dumb in 1971. FedEx handled about 5 billion shipments in 2024.
In the real world, diverting attention from feature function improvement to this sort of reset is hard to sell and harder to implement. Typically, existing teams aren't capable of doing this, and leadership shies away from resetting and replacing teams that aren't clearly failing. That is why, most often, Wave Two companies win.
The clean sheet/reset work requires not just great technologists/engineers but also architects, design, and cost structure ingenuity. That last one is particularly hard to find in successful tech teams.
You also have to find the funding and be a little crazy. On paper the challenges are not sexy or attractive, they are a combo of astronomically hard tech drudgery and wishful thinking.. They are a gamble. My experience is that it takes an exceptional CEO /CFO and board, especially at a firm not in crisis, to entertain an expensive program pitch that includes the combo of hard tech drudgery, a gamble, and wishful thinking.
These sorts of efforts fail a lot. The part of the David versus Goliath story that people miss is that Goliath killed 300 'Davids' of various kinds before he met the final one.
Ocient is a great example. I remember walking on a street in Chicago with the genius CTO before it was really a company discussing everything that would need rethinking/rengineering for increased speed/efficiency,. It included things like rewriting the Intel drivers for NVME drives, how chips communicate on a board, insanely fast code and a brand new database architected from scratch. These are all things the average technologist thinks are crazy. I remember having to stop walking to really grok what was being discussed. They architected a platform that is two orders of magnitude and sometimes better in the scale analytics space. One order of magnitude was the impact Henry Ford had. Luckily, the Ocient crew had just sold their last start-up for a shed load of money and already had a world-class handful of tech talent. They also had patience, which is, of course, related to the balance sheet.
VANTIQ is another example. They upgraded and combined what is typically four different technologies, plus new tech (GenAI/Agentic). Then, they made it all work in real-time inside a design that not only internalizes the complexity to make ease of use/adoption astonishingly good but also makes future change cheap/easy. They gave it a construction environment that lets users build complex solutions in 50x less time/effort. Yep, 50x. The first time I saw a PPT about it, I was sure it was not real. Again, it was a mountain of person-years of top tech talent to build.
Okay, switching to decaf.
