The human-car gestalt, for example, experiences life and has opinions that are quite different from the human-bicycle gestalt, the human-train gestalt or even the bare human with no mode of transportation. What’s possible and what’s desirable for a driver is pretty radically different even to what’s possible or desirable for the same human outside of a car: in individual instances of this happening, we call it road rage, and on a policy scale it leads to things like people getting extremely angry about bike lanes even when said bike lanes would benefit them at all times when they’re not in their car. The human-car gestalt has interests and opinions that are separate from the human in the gestalt, often opposed to them and able to assert them over the human part of the gestalt.
It is an unfortunate fact that, when faced with such an analysis of technology, tech people will often think that for whatever reason the considerations that apply to literally the entire rest of the world of technology somehow don’t apply to them. This kind of overweening arrogance is, of course, why we find ourselves in so much of the shit that we’re in, but nonetheless, to really get my point across it’s probably a good idea to spell out how these dynamics operate in the tech world specifically.
Tools change how we think and what our motivations are. The very use of morphine sociologically creates recreational morphine users, motivates us to farm poppies and guard the fields with violence and shit. You can’t have one without the other.
A human-stove gestalt isn’t motivated to put their face on it. But they are motivated to desire fire codes, home kitchens, exhaust fans, and in older times a right to absolutely fuck air quality. Or consider how the human-gas stove gestalt is motivated to push back against air quality safety regulations given the academic evidence that they cause harm. It should be a no brainer to just not hurt yourself and your family and change technologies, but habituation has changed motivations to the extend that the human-gas stove gestalt wants things that actively harm the human.
It’s not some abstract point. A human-llm gestalt will want different things, many of which may be harmful and if you use these things you need to be aware and avoid it. Like avoiding becoming carbrained when using a car. Idealistic vibes aren’t enough, we need systems and rules and politics or it will be the motor vehicle again.