The BigCode initiative’s purpose is to construct state-of-the-art giant language studying fashions (LLMs) to construct code in an open and accountable means.
Code LLMs allow the completion and synthesis of code from different code and pure language descriptions, and permits customers to work throughout a variety of domains, duties, and programming languages.
The initiative is led by ServiceNow Analysis, which does analysis to futureproof AI-powered experiences, and Hugging Face, a neighborhood and knowledge platform that gives instruments to allow customers to construct, practice, and deploy ML fashions based mostly on open-source code and applied sciences.
BigCode is inviting AI researchers to collaborate on a consultant analysis suite for code LLMs masking a various set of duties and programming languages, accountable growth and governance of information units for code LLMs, and quicker coaching and inference strategies for LLMs.
“The primary objective of BigCode is to develop and launch an information set giant sufficient to coach a state-of-the-art language mannequin for code. We’ll be certain that solely information from repositories with permissive licenses go into the information set,” ServiceNow Analysis wrote in a weblog publish.
“With that knowledge set, we’ll practice a 15-billion-parameter language mannequin for code utilizing ServiceNow’s in-house GPU cluster. With an tailored model of Megatron-LM, we’ll practice the LLM on the distributed infrastructure.”
Further particulars concerning the mission can be found right here.
