My desire on how the AI race should go?
We need more energy-efficient, sustainable models, not just the most capable models now. While all the companies are trying to be the best in the race, the real winner will be the one who can run longer with less resources in a balanced speed and capabilities.
Google's focus on Flash models is actually good in general; they're the fastest models with very good quality. Distillation [transferring knowledge from a large model into a smaller, faster one] at its best, imo.
Even if the models stopped improving, I think Google and OpenAI models are in a good space now.
1B downloads on Gemma is saying real news here: most people started using their own hardware for models. It means we should focus on open models that actually can do more with less hardware requirements.
Real breakthroughs are techniques like: Quantization [reducing the precision of model weights to save memory], MoE [Mixture of Experts: routing inputs through specialized sub-networks], and Distillation. We should focus more research on these, which will lead to better understanding and real breakthroughs without burning our limited resources.
We need a deliberate AI winter. But who is going to listen to a random person posting on X at 1 PM thinking too much about where we are heading?
Ultimately, we need to leave some resources for future generations (our children) to at least survive, not just a dying earth full of cutting-edge, resource-hungry solutions.
Share this article
Generate a story card or copy link to clipboard
