AI investors and startups are beginning to realize bigger may not always be better.
Small language models—pared-down versions of the large language models that underpin generative AI technology—are gaining traction and opening investment opportunities in an area dominated by OpenAI and Anthropic.
These models are everything LLMs like ChatGPT and Claude aren’t. They’re considered “small” because they’re trained using less data. They’re cheaper to run and can be deployed on local devices. They don’t have access to an internet’s worth of knowledge, but what they lack in breadth they make up for in depth.
Large language models are struggling to gain adoption among large enterprise customers concerned with privacy, speed and cost. SLMs are finding a footing by offering more secure and specialized generative AI applications.
In June, I reported on how startups can survive the new AI age. Nearly everyone I spoke to urged startups to fill the gaps the larger players can’t. Investors, founders and executives now say SLMs are key to hastening enterprise AI adoption and paving a path forward for survival against incumbents.
The golden ticket
The problem with large language models, according to some investors, is that they’ve become too generic and are stuffed with information not applicable to corporate and other enterprise customers. Jesse Middleton, general partner at Flybridge Capital, whose portfolio includes the small language model developer Arcee.AI, said that smaller models can be more effective at solving specific problems.
“The difference between an SLM and LLM, it’s [that the SLM is] like a PhD student: super deeply knowledgeable about a specific problem area, with a lot less knowledge of other things,” he said. “I don’t want my self-driving car to know about philosophy. I want it to know everything there is about driving appropriately down a highway and saving my life in the case of an oncoming incident.”
Without all the baggage, SLMs can be something LLMs often aren’t: useful.
John Bodrozic, co-founder of HomeZada, a digital home management platform, said that AI would be useless to his company without small language models. The company uses them to create a personalized AI assistant that knows its user’s home. LLMs alone aren’t specific enough to the task to justify their costs, he said.
“AI in my world has to be unique, it has to know about the boundaries of each home. It can’t be a standalone product—it has to be integrated with something,” Bodrozic said. “Figuring out how to make it small and unique is paramount. Otherwise, it’s just useless.”
Victor Botev, CTO and co-founder of Iris.AI, a startup creating a small language model for use in the material sciences field, said that many companies are now realizing that with SLMs, they can get more for less.
“In the enterprise, I would say the motto is ‘the best quality for the money we have,’” he said. “It’s a golden environment for small language models.”
The local beat
A key benefit to small language models is their ability to run locally on devices like computers and phones. LLMs often have to send queries and information to data centers due to their sheer size.
Another complaint is that large models have become too big to implement for enterprise customers concerned with privacy and security. Amol Ajgaonkar, CTO at IT technology company Insight, said that companies in sensitive areas like healthcare and fintech don’t want to risk transmitting data off-site.
“Everyone kind of overlooks, whether you’re running a large language in the cloud or with some private hosting capability, [that] your data is still leaving and being sent away,” he said.
By running locally, SLMs can also solve for one of LLM’s biggest complaints: speed. In fields that require the quick processing of data, using LLMs that have to maintain a connection to a data center has made deployment a non-starter. Locally run models don’t require internet connections and can process requests and data much faster than LLMs.
Middleton of Flybridge again pointed to self-driving cars as an example of how locally deployed models can make a difference. “I want the model to run locally, not be beholden to a cloud somewhere when there’s an oncoming vehicle and it’s got to ping out and get an answer back.”
At what cost?
Perhaps generative AI’s biggest hurdle has been how cost-prohibitive it is to implement and run.
Grace Isford, partner at Lux Capital whose portfolio includes Hugging Face, a platform for machine learning development, and the Japanese small language model developer Sakana.AI, said that while large language models can be alluring, smaller models can offer the same performance for cheaper.
“If you have a Porsche and a Prius and you need to get from point A to point B, [and] you want to spend money and look fancy, do the extra bells and whistles,” she said. “But the Prius is going to do the job.”
In addition to being cheaper to run, small models are less costly for founders to build.
Thomas Wolf, chief science officer and co-founder of Hugging Face, said that smaller models can level the playing field between startups and large companies.
“It gives a shot to startups that don’t necessarily get the big firepower,” he said
Wolf also said that smaller models are quickly gaining ground on Hugging Face’s online repository of downloadable models. He said enterprise users are hungry for personalized, programable AI to serve their needs.
“You don’t need large frontier models for every task,” he said. “It’s perfectly fine if your marketing brief summarization model cannot solve Math Olympiad problems.”
Featured image by Julia Midkiff/PitchBook News
Learn about our editorial standards.