How can I find a local coding LLM that wasn’t trained on copyleft code?
It’s an ethical concern, not a legal one, so if the data collection doesn’t entirely rule out copyleft, that might be OK
How can I find a local coding LLM that wasn’t trained on copyleft code?
It’s an ethical concern, not a legal one, so if the data collection doesn’t entirely rule out copyleft, that might be OK
Olmo might be the closest you can get
https://allenai.org/olmo (website straight up crashes my phone)
https://huggingface.co/allenai/OLMo-7B
Olmo is trained on the Dolma dataset, “a diverse mixture of web content, scientific papers, code, public-domain books, social media, and encyclopedic materials”, so this might be a more ethical source, but I’m not sure if it’s copyleft free.
https://huggingface.co/papers/2402.00159
You could rent a cluster of GPUs and remove copyleft material and retrain
Dolma uses starcoder https://huggingface.co/blog/starcoder, which
so this might do. Thanks.
Only correct answer so far. Here’s the licenses included in the dataset.