• Echo Dot@feddit.uk
    link
    fedilink
    English
    arrow-up
    1
    ·
    edit-2
    16 minutes ago

    Isn’t parameters just the number of virtual dials that can be changed, essentially the number of neurons, I don’t think it’s got anything to do with the actual amount of data.

    After all distilling works (somehow), by taking all the knowledge of the large AI with a lot of parameters and putting it into a smaller AI with fewer parameters. The amount of data hasn’t changed only the number of parameters. Presumably the inverse is also true, you can have a lot of parameters but not a lot of data.