Nvidia's newest competitor: the Groq blazing fast, game-changing LPU
let's start with a quote by groq ceo, jonathan ross, from a recent venturebeat article:
"In a bold claim, Ross told me that “we are probably going to be the infrastructure that most startups are using by the end of the year.”
this is a bold claim indeed, but i wouldn't bet against him. why would any startup go with more expensive chips that don't perform nearly as well and are not as customizable? groqchips are the first chips to be built with ai in mind. this is probably the biggest development in ai since ChatGPT premiered in November '22.
if you want to see how blazing fast the chip is, check out this 4-minute video:
https://youtu.be/QE-JoCg98iU?si=wMRAT9rrzQ26LXVz
here are some of groqchip's main features:
it's deterministic, meaning that the same input will always yield the same output, making it much easier to understand and control what's going on.
it's 10x to 100x faster (very low latency) than other systems, handling large language models more efficiently with a record-breaking speed of more than 100 tokens per second on a 70-billion-parameter model. [1][2].
groq's approach of developing software first also results in lower power consumption and lower costs [2].
this 22-minute video goes into the technical details:
https://youtu.be/TjCDyiUVZL0?si=hmgaFCs2nLo1P6i0
all of this of course gets us a lot closer to agi. the question now is whether nvidia and other ai developers and chip giants will collaborate with groq or be forced by the market to develop their own competing lpus.
here are the references:
[1] Groq's Record-Breaking Language Processor Hits 100 Tokens Per Second On A Massive AI Model https://www.forbes.com/sites/moorinsights/2023/08/11/groqs-record-breaking-language-processor-hits-100-tokens-per-second-on-a-massive-ai-model/?sh=58d9905f358f
[2] Inside Groq’s LPU and its impact on generative AI - SiliconANGLE https://siliconangle.com/2023/11/17/inside-groqs-lpu-impact-generative-ai-sc23/