A lot of people have deep misunderstandings on what the massive data centers are for. While they do use a decent portion of them to run requests, as they’re trying to service a lot of people, a big chunk is training and data processing. Not for the end user, but for their R&D, model production, and the endless churn of data processing as they find and create new data to incorporate in models.
Which is how you can run pre-trained LLMs on your computer, if it’s a good one.
that still doesn’t mean inference is profitable. (which isn’t to say i give a shit about profits, but it’s a good indicator of economic viability even if socialized). inference still requires far more compute to complete a task than most previous means of doing so.
A lot of people have deep misunderstandings on what the massive data centers are for. While they do use a decent portion of them to run requests, as they’re trying to service a lot of people, a big chunk is training and data processing. Not for the end user, but for their R&D, model production, and the endless churn of data processing as they find and create new data to incorporate in models.
Which is how you can run pre-trained LLMs on your computer, if it’s a good one.
I mean, China is training comparable models with a tiny fraction of compute. And it’s not just them distilling US models.
that still doesn’t mean inference is profitable. (which isn’t to say i give a shit about profits, but it’s a good indicator of economic viability even if socialized). inference still requires far more compute to complete a task than most previous means of doing so.