Executive Summary
NVIDIA announced that leading AI inference providers, including Baseten, DeepInfra, Fireworks AI, and Together AI, are using its Blackwell platform to significantly reduce operational costs. By combining the Blackwell architecture with optimized software stacks and open-source models, these partners are achieving up to a 10x reduction in cost per token compared to the previous Hopper platform. This improvement in "tokenomics" makes scalable, real-time AI applications more economically viable for customers across industries like healthcare, gaming, and enterprise support.
Key Takeaways
* Massive Cost Reduction: The NVIDIA Blackwell platform enables a reduction in the cost per token for AI inference by up to 10x compared to the NVIDIA Hopper platform.
* Key Partners & Use Cases:
* Baseten (Healthcare): Delivered a 10x inference cost reduction and 65% faster response times for Sully.ai's medical AI agents.
* DeepInfra (Gaming): Achieved a 4x reduction in cost per token for Latitude's AI-native games by leveraging Blackwell's NVFP4 format.
* Fireworks AI (Agentic AI): Provided 25-50% better cost efficiency for Sentient Chat, supporting a launch to 1.8 million users.
* Together AI (Customer Service): Enabled a 6x drop in cost per query for Decagon's AI voice agents, with response times under 400ms.
* Technical Enablers: Cost savings are driven by a combination of Blackwell's hardware, low-precision data formats (NVFP4), and optimized software libraries like NVIDIA TensorRT-LLM.
* Future Roadmap: NVIDIA signals continued focus on cost reduction with its next-generation Rubin platform, which aims for another 10x improvement over Blackwell.
Strategic Importance
This announcement solidifies NVIDIA's dominance in the AI inference market, demonstrating that its platform is critical for making large-scale AI deployment economically feasible. By drastically lowering the primary operational cost of AI, NVIDIA accelerates enterprise adoption and positions its hardware as the essential infrastructure for the next wave of AI applications.