Is it truly hopeless for anyone but pioneers to build an #AI foundation model? The mounting evidence suggests that unlike semis or operating systems, these models may not remain in the exclusive realm of a privileged few.
As explained in the post (https://bit.ly/45QZrhZ), the pivotal moment of late 2022 was not an "iPhone"-like product or Einsteinian innovation in methods/hardware; it was simply that for some, persistence paid! Their transformer models, being trained for years without sufficiently satisfactory results, crossed an unknown threshold of complexity to give human-like results.
With the validation of transformer neural net logic, all boosted the training pace, and contenders emerged to challenge the frontrunners in weeks. (read earlier post https://bit.ly/3qlKUe2).
A graphic analogy to drive home the point: 2022 became the birth year of the first machine brains with input-output conversions mimicked those of humans. Interestingly, the creators of these early machine brains didn't pursue significant patents. Their neighboring peers recognized the quantity game for own machine-birthing endeavors. While many pioneers may genuinely believe their achievements to be unreplicable by anyone globally, the author of at least one internal memo was less confident, suggesting the exact opposite (https://bit.ly/42sPdC2).
Even if Alpaca was developed in days over the open-source model from Meta at USD600, it came from a team from the same nation. For distant observers, it looked like the best the rest of the world could hope for was to join the queue to access the APIs of the earliest creators and develop use cases atop - similar to their fate in other technology segments.
Falcon 40B shatters all this. Abu Dhabi's TII unleashed a trillion-token foundation model that leads the Open LLM leaderboard in performance (https://bit.ly/3NwHVJ7). The single example is enough for everyone in countries from India to Singapore, Japan to the UK, or corporates from even non-technology sectors that to try developing models without backbreaking resource or time requirements. Many will finally note that the sizes of teams behind almost all foundation models are relatively small, and so are the length of their codes - unlike teams that built the first smartphones or the length of code in their OSs.
In short, Sam Altman could be wrong about India's chances of the building.
Does one swallow make a summer? While theoretically promising, we must gather more information on the practical use cases of all models before passing a verdict on whether the world at the same time two years later will have ten or thousand useable foundation models. Many foundation models may prove to be mere namesakes with limited use.
The takeaway for policymakers and corporate leaders worldwide is not to resign to being perpetual end-users burdened with exorbitant payments. Abu Dhabi's endeavors deserve much greater media attention than they have received thus far.
