How Jensen Huang's thinking on GPU & AI evolved
the evolution of GPU & AI thinking · 1993–2026
Open the interactive atlas →How the thinking evolved
1993→97
Founding & survival
NVIDIA is founded at a Denny's (1993). The first chip flops — ~249k of 250k cards come back; SEGA bails them out, and the RIVA 128 bet (shipped without a prototype) keeps them alive. Hence the motto 'thirty days from going out of business'.
„Our company is thirty days from going out of business.”
1999
GPU — the engine of imagination
GeForce 256 — NVIDIA coins the term 'GPU'. Games are the deliberate killer app: the hardest compute problem plus huge volume that funds R&D and seeds the platform.
„The GPU started out as the engine for simulating human imagination. Now, NVIDIA's GPU runs deep learning algorithms, simulating human intelligence.”
2006
CUDA — the big bet
CUDA turns the GPU into a universal parallel processor. The bet 'consumed all the gross profit', the market cap fell to ~$1.5B — a decade in 'zero-billion-dollar markets'.
„CUDA increased our cost of that GPU, which is a consumer product, so tremendously, it completely consumed all of the company's gross profit dollars.”
2012
The big bang of AI
AlexNet wins ImageNet on TWO consumer GeForce GTX 580s. Huang: 'the big bang of modern AI'. NVIDIA bets the company on deep learning — gaming cards ignite the AI revolution.
„That was the moment that was the big bang of modern AI.”
2023
AI factory / 'iPhone moment'
ChatGPT = the 'iPhone moment of AI'. The data center stops being a cost — it becomes a factory turning energy into tokens. 'General-purpose computing has run out of steam'.
„We are at the iPhone moment of AI.”
2026
The inference economy
The closure: reasoning/agentic AI needs '100× more compute'; the token is a commodity and compute is revenue. The next wave — physical AI: 'everything that moves will be robotic'.
„Tokens are the new commodity.”
Key concepts
Graphics
GPU — the engine of imagination
The GPU as a 3D graphics accelerator for games — the starting point; graphics cards fund R&D and seed the platform.
“30 days from going out of business”
Existential paranoia as culture — from founder fragility (1993) through the near-death of RIVA 128 (1997) to 'pain and suffering'/'run, don't walk'. Character forged by adversity.
Coining the term “GPU”
GeForce 256 (1999): hardware T&L, 'the world's first GPU' (a marketing claim). IPO, Xbox contract.
Neural rendering (ray tracing + DLSS)
RTX/Turing: real-time ray tracing ('holy grail') + RT/Tensor cores; DLSS = AI in graphics; the thesis 'graphics = AI'.
Parallel computing
Accelerated computing (CUDA)
CUDA turns the GPU into a universal parallel processor; accelerated computing replaces general-purpose.
CUDA as a moat
A moat through the ecosystem: installed base (GeForce) + libraries + backward compatibility. 'The house that GeForce built'.
Zero-billion-dollar markets
Targeting markets that don't yet exist (market making, not share taking) — where there are no customers, there is no competition.
"Moore's Law is dead"
The end of CPU scaling → accelerated computing as the answer ('Huang's Law'). A polemic against Intel; justifies demand for GPUs.
Deep learning
Deep learning — the engine of intelligence
AlexNet, the 'big bang of AI,' on 2× GTX 580; the GPU turns out to be the computer for training networks. The company's pivot in 2012.
DGX — "AI supercomputer"
'AI supercomputer in a box'; the first DGX-1 was delivered in person to OpenAI (2016) — the spark of modern AI.
Tensor Cores / Transformer Engine
Dedicated silicon for DL (Volta 2017) → Transformer Engine (Hopper 2022) → FP8/FP4 (Blackwell).
AI factory
AI factory
The data center = a factory turning data + energy into tokens/intelligence; generative AI as a new industry.
“iPhone moment of AI”
ChatGPT = the moment of mass AI adoption (GTC March 2023, NOT Computex); generative AI reaches hundreds of millions of people.
Token economy
The token = the new commodity ('tokenomics'); 'compute is revenues'; tokens-per-watt as an AI factory KPI.
Blackwell / annual roadmap
B200/GB200 NVL72 (dual-die, 208 billion transistors) as an 'AI factory' product; annual cadence: Blackwell→Rubin→Feynman.
Sovereign AI
Every country its own infrastructure/AI; data = a national resource. (Aligned with GPU demand — a conflict of interest.)
Physical AI / robotics
'ChatGPT moment for robotics'; Cosmos/Isaac GR00T/Newton; 'everything that moves will be robotic'.
Inference economy (reasoning)
Test-time/reasoning scaling: 'inference 100×'; agentic AI as the largest consumer of compute; 'demand parabolic'.
Selected quotes
„Our company is thirty days from going out of business.”
„The GPU started out as the engine for simulating human imagination. Now, NVIDIA's GPU runs deep learning algorithms, simulating human intelligence.”
„CUDA increased our cost of that GPU, which is a consumer product, so tremendously, it completely consumed all of the company's gross profit dollars.”
„That was the moment that was the big bang of modern AI.”
„We are at the iPhone moment of AI.”
„Tokens are the new commodity.”
„I wouldn't do it.”
„Building Nvidia turned out to have been a million times harder than I expected it to be — than any of us expected it to be.”
„If we realized the pain and suffering, and just how vulnerable you're going to feel, the challenges, the embarrassment and the shame — nobody in their right mind would do it.”
„My first job before CEO was a dishwasher. And I did that very well.”
„I know it's going to be perfect, because if it's not, we'll be out of business.”
„At one point, I had to fly to Japan and explain to Sega's CEO that the technology they contracted us to build would not work. Asked to be released from a contract we could not complete. And then asked that they still pay us. Without the money, NVIDIA would vaporize.”
Key events
- 05.04.1993 Founding of NVIDIA (Denny's, San Jose)
- 08.11.2006 CUDA — a bet on parallel computing
- 05.04.2016 GTC 2016 — Pascal P100, DGX-1, 'AI'
- 04.01.2017 CES 2017 — Xavier, AI Co-Pilot, 'AI is the future'
- 13.08.2018 SIGGRAPH 2018 — Turing/RTX, real-time ray tracing
- 14.05.2020 GTC 2020 — Ampere A100 ('kitchen keynote')
- 22.03.2022 GTC 2022 — Hopper H100, Transformer Engine, 'AI factories'
- 24.05.2023 Earnings FY24 Q1 — guidance shock, the '$1T data center' thesis
- 23.08.2023 Earnings — explosion of demand for the H100
- 15.03.2024 Stanford SIEPR/GSB — 'pain and suffering', low expectations
- 22.05.2024 Earnings — sovereign AI, 'AI generation factories'
- 20.11.2024 Earnings FY25 Q3 — 'Blackwell demand is staggering'
- 18.03.2025 GTC 2025 — Vera Rubin roadmap, reasoning, physical AI
- 09.07.2025 Valuation milestones $1T → $5T
- 15.04.2026 Dwarkesh/Stratechery — the CUDA moat, 'the whole data center'