I think there is a very narrow, specific implementation of LLMs for NPCs that could be used for games that I personally would deem as both ethical and acceptable:
The model would need to be trained exclusively on a deep write-up of source materials authored by human beings (something like what Patrick Stewart was given for his role in Oblivion); and human voice actors hired for all characters, who explicitly approve their use towards generative characterisation solely for this one role/game.
This is just my personal line in the sand, and I know not everyone would agree - and that’s fair enough.
Every time I read something like this I realize just how little people understand how these things work.
The tiniest LLMs you can run on your machine are 1B, those take around 10TB of unrepeatable and well varied text to train. Let me put this into perspective, if you downloaded the whole Wikipedia, you would get 1% of the amount of data needed to train the tiniest LLM you can run. Do you think you can ethically source the remaining 99%?
10 Terabytes. 1 letter is 2 bytes in unicode. 10e12 bytes. 5e12 letters. 1 word is about 5 letters. 1e12 words. Assuming 500 words per page, that would be 2e9 pages. Page thickness assumed at 1e-4. 2e5 meters, if you stack all the pages. 200 kilometers high tower of paper.
<pedantry>For English language text letters are 1 byte when encoded in UTF-8 (which is how essentially all text is stored and transmitted these days).</pedantry>
Though it probably balances out in your final answer since you’re not counting spaces, punctuation, formatting, or other markup.
I think there is a very narrow, specific implementation of LLMs for NPCs that could be used for games that I personally would deem as both ethical and acceptable:
The model would need to be trained exclusively on a deep write-up of source materials authored by human beings (something like what Patrick Stewart was given for his role in Oblivion); and human voice actors hired for all characters, who explicitly approve their use towards generative characterisation solely for this one role/game.
This is just my personal line in the sand, and I know not everyone would agree - and that’s fair enough.
Every time I read something like this I realize just how little people understand how these things work.
The tiniest LLMs you can run on your machine are 1B, those take around 10TB of unrepeatable and well varied text to train. Let me put this into perspective, if you downloaded the whole Wikipedia, you would get 1% of the amount of data needed to train the tiniest LLM you can run. Do you think you can ethically source the remaining 99%?
10 Terabytes. 1 letter is 2 bytes in unicode. 10e12 bytes. 5e12 letters. 1 word is about 5 letters. 1e12 words. Assuming 500 words per page, that would be 2e9 pages. Page thickness assumed at 1e-4. 2e5 meters, if you stack all the pages. 200 kilometers high tower of paper.
<pedantry>For English language text letters are 1 byte when encoded in UTF-8 (which is how essentially all text is stored and transmitted these days).</pedantry>
Though it probably balances out in your final answer since you’re not counting spaces, punctuation, formatting, or other markup.
Meaning that to train even the tiniest LLM you can get you need a stack of paper so tall it could get knocked by satellites.
This is the scale that people miss when they talk about “writing some stuff to train an AI model”.