Artificial Intelligence (AI) is transforming industries, and d-Matrix Inc., a Santa Clara-based startup, is leading innovation with its Corsair AI processor. Backed by Microsoft, this groundbreaking technology delivers high-performance AI inference without GPUs or high-bandwidth memory (HBM). Corsair’s revolutionary features, unmatched benchmarks, and strategic partnerships position it as a game-changer in the AI market.
What Makes Corsair Unique?
Corsair redefines AI inference by replacing traditional GPU architecture with Digital In-Memory Computation (DIMC). This cutting-edge approach achieves a memory bandwidth of 150 TB/s, making it ideal for generative AI tasks while significantly reducing costs.
Key Features:
- DIMC Technology: Enhances memory and computational efficiency.
- Chiplet Packaging: Integrates memory and computation in a compact form factor.
- Scalability: Supports up to 256GB off-chip memory using DMX Bridge cards.
- PCIe Gen5 Compatibility: Ensures seamless integration with existing systems.
Performance Benchmarks
Corsair excels in AI inference speed and energy efficiency. For example, it processes 60,000 tokens per second at 1ms per token using Llama3 8B models on a single server. Larger models, such as Llama3 70B, achieve 30,000 tokens per second at 2ms per token within a single rack.
Performance Highlights:
| Model | Tokens/Second | Latency per Token | Environment |
|---|---|---|---|
| Llama3 8B | 60,000 | 1ms | Single Server |
| Llama3 70B | 30,000 | 2ms | Single Rack |
These metrics demonstrate Corsair’s ability to handle resource-intensive AI tasks, offering substantial savings in energy and operational costs over GPU-based systems.
Advanced Architecture
The Corsair processor employs Nighthawk and Jayhawk II tiles, leveraging a 6nm manufacturing process. Each Nighthawk tile integrates four neural cores and a RISC-V CPU, optimized for large-model AI inference. Additionally, versatile datatype support, including block floating point (BFP), enhances computational efficiency.
Architectural Details:
- 6nm Manufacturing Process: Ensures compact and efficient design.
- Neural Cores: Four per tile for parallel data processing.
- Integrated Memory: 2GB of performance memory embedded on-chip.
- Peak Processing Power: 2400 TFLOPs at 8-bit precision.
Strategic Partnerships
Microsoft’s investment underscores the transformative potential of Corsair. Additionally, Micron Technology, a key Nvidia partner, collaborates with d-Matrix to refine the processor’s capabilities. These alliances highlight Corsair’s industry relevance and scalability.
Industry Impact:
“We saw transformers and generative AI coming, and founded d-Matrix to address inference challenges around the largest computing opportunity of our time,” said Sid Sheth, co-founder and CEO of d-Matrix. “The Corsair compute platform enables blazing-fast token generation for high interactivity applications with multiple users, making Gen AI commercially viable.”
Applications and Market Impact
Corsair’s architecture is designed for diverse generative AI and interactive applications, including:
- Transformers: Critical for natural language processing and AI model generation.
- Agentic AI: Powers autonomous decision-making systems.
- Interactive Video Generation: Drives real-time AI-driven content for enhanced user engagement.
This efficiency and versatility make Corsair a cost-effective solution for businesses across various sectors.
Future Availability
Corsair is currently available to early-access customers, with a broader rollout planned for Q2 2025. This phased launch allows d-Matrix to refine its technology based on user feedback and evolving industry needs.
Final Thoughts
Corsair introduces a new era in AI inference, offering scalable and efficient solutions without relying on traditional GPUs. Backed by strong industry partnerships and innovative architecture, d-Matrix positions itself as a leader in AI hardware. As generative AI demand surges, Corsair’s impact will likely resonate across technology sectors and enterprises alike.
