There are two key requirements for the success of any digital product — ease of access and a user-friendly interface and experience.
If this wasn’t the case, Linux-based OS systems would trump MacOS and Windows.
This is also the reason why GPT, Claude, and Grok models are triumphing over open-source LLMs like the Llama and Mistral series, even when the latter provide some incredible utility with great customizations.
I think OpenAI and DeepSeek present interesting case studies here.
DeepSeek is making open-source foundational models, but if it didn’t also provide services directly through its web interface and the iPhone app — there is fat chance it could disrupt the U.S. markets the way it did.
Like DeepSeek was doing before, it would have remained a product that nerdy devs talked about as the “real deal” in their small circles, as was happening in months preceding the mania that surrounded the Chinese company.
OpenAI similarly built open-source models in shadows for roughly seven years between 2015 and 2022 — until it launched ChatGPT. I’m sure you remember what that was like.
To Llama, Or Not To Llama
One constraint in the more widespread utility of open-source models is technical skills — that one is more obvious.
But when it comes to LLMs — it’s actually also money. 💸
You see, running open-source models requires GPUs. Some small open-source models can run on consumer GPUs like the one I have in my top-of-the-line M3 Max Macbook Pro with 36 GB of memory.
But others require dedicated ones.
Meta this weekend dropped the Llama 4 series of models, including the Maverick and Scout LLMs, and announced plans to release Behemoth at a later date.
There are no updates yet on the reasoning model from the fourth series, except for a nerdy-looking Llama telling us it is “coming soon.”
Here is the most eyebrow-raising bit about Llama 4: As Meta keeps chasing more and more parameters in its LLMs, the cost to acquire the hardware to run these models gets ridiculous.
It’s a bit early and I haven’t analyzed every information available on developers trying to run the models on different devices but it seems that the minimum requirement to comfortably run the lower-end Scout model is a single Nvidia H100 GPU, which costs roughly $40,000 — provided you can manage to get your hands on one.
If Sam Altman, with his hundreds of billions of dollars, struggles to find GPUs, so does this poverty-struck startup founder.
Mixture of Experts
Having said that, there is one interesting thing that makes it possible to run the Llama 4 line on Apple products — the possibility of the Mac Studio with 128 GB memory or above.
That is a Mixture of Experts.
Some of the earlier LLMs were actually a single model trained on a whole swath of data from across domains like GPT-3 or the original Llama. But companies are rapidly switching to a mixture of experts concept.
This means that even though we see Llama 4 Scout as a single model that we are talking to, it is actually deciding between 16 separate trained models on which one will respond to the query, based on whether we asked it a math question or asked it to spark creativity.
This is different from the traditional dense models that operated on single monolithic networks, where all of the parameters of the LLMs were activated for every query. So, even if you asked it “what’s 2+2,” it would active all of its knowledge on Socrates and Plato’s philosophies.
Dear Zuck, Size Doesn’t Matter
Setting aside the difficulties of running the Llama 4 series, even the ones who have tried it (mostly through Groq/OpenRouter) are less than impressed.
The Llama 4 series isn’t doing great at coding or deep questions — but seems to love emojis (and me ❤️).
So here goes, even as companies keep obsessing over increasing the parameters in the training of foundational LLMs, which doesn’t seem to be improving things.
In fact, it may have opened a key business opportunity that we thought of as closed so far. That of training more domain-specific niche models.
As noted by AI researcher Andriy Burkov, if your business idea isn't in the math and coding or factual question-answering domains, there is a great opportunity to build your business-specific dataset.
The potential increase in generalist models' skills will no longer be a threat.
So, is now the time we make our own LLM at Dzambhala Finance? Perhaps, but we need enough revenue to sustain a bigger database.
This post is republished from the Artificially Boosted newsletter that goes out every week.
