Isn’t parameters just the number of virtual dials that can be changed, essentially the number of neurons, I don’t think it’s got anything to do with the actual amount of data.
After all distilling works (somehow), by taking all the knowledge of the large AI with a lot of parameters and putting it into a smaller AI with fewer parameters. The amount of data hasn’t changed only the number of parameters. Presumably the inverse is also true, you can have a lot of parameters but not a lot of data.
Yeah 1 trillion params has to be pretty much all of human written creation
Isn’t parameters just the number of virtual dials that can be changed, essentially the number of neurons, I don’t think it’s got anything to do with the actual amount of data.
After all distilling works (somehow), by taking all the knowledge of the large AI with a lot of parameters and putting it into a smaller AI with fewer parameters. The amount of data hasn’t changed only the number of parameters. Presumably the inverse is also true, you can have a lot of parameters but not a lot of data.