Meta incidentally filed take-down orders to sites such as GitHub and open-source AI group Hugging Face to corral the purloined program. Eventually, faced with the code being easily available across the web, Meta gave up trying to order the tide to go back. Instead, it embraced the release.
Both versions of Llama have been trained on Common Crawl, GitHub, Wikipedia, Project Gutenberg, ArXiv, Stack Exchange, and other open test websites. While Microsoft and OpenAI's ChatGPT got the headlines, many open-source developers turned to Llama.
Besides having access to Llama, Meta also shared its weights. The other major LLMs haven't. With weights, the parameters learned by a model during training, it's much easier to create and run custom AI programs. The other big LLMs, such as GPT, are usually only accessible through application programming interfaces (API).
While AI is built on open-source foundations, Llama is the first major open-source LLM. Its pre-trained models have been trained on 2 trillion tokens, and have to double Llama 1's context length. Its fine-tuned models have been trained on over 1 million human annotations. Its model size parameters range from 7 to 70 billion parameters.
So, that's why open-source developers welcomed Llama, but why did Meta open Llama-v2 up? According to its researchers, "While many companies have opted to build AI behind closed doors, we are releasing Llama 2 openly to encourage responsible AI innovation. Based on our experience, an open approach draws upon the collective wisdom, diversity, and ingenuity of the AI practitioner community to realize the benefits of this technology. Collaboration will make these models better and safer."
But, is Llama-v2 actually open source? While Meta says the right things, the Llama 2 Community License Agreement has not been approved by the gold standard group of open-source licensing, the Open Source Initiative (OSI).
Strictly open-source or not, Llama 2 is certainly open enough for most practical purposes. And, as Amanda Brock, CEO of OpenUK, put it, it's "Not an OSI approved license but a significant release of Open Technology … This is a step to moving AI from the hands of the few to the many, democratizing technology and building trust in its use and future through transparency. No, it's not perfect, and yes, there is more work to be done, but this bold move sets the tone for AI-open innovation with a responsible but light-touch principles-based approach to regulating the use and development of AI."
Meta isn't just playing nice with the open-source community. Meta also declared Microsoft is its preferred Llama 2 partner. So, Llama 2 is available in the Azure AI model catalog, enabling developers using Microsoft Azure to build with it. The LMA is also optimized to run locally on Windows, giving developers a seamless workflow as they bring generative AI experiences to customers across different platforms. Llama 2 is available through Amazon Web Services (AWS), Hugging Face, and other providers.
Besides opening up the code, in this release, Meta is also trying to make Llama safer, more well-behaved, and less prone to hallucinations than the other models.
Meta has done this by Red-Teaming (Security Testing) Exercises designed to clean up safety weaknesses. Meta has also released a Developer's Guide for Safe and Responsible Use to help developers understand and apply the best practices for developing and responsible model testing. Finally, Meta's provided a Llama Acceptable Use Policy to prohibit certain use cases to help ensure that these models are being used fairly and responsibly.