Before Big Tech researchers warned of a superintelligence doomsday, 2028 hopeful Gov. Gavin Newsom (D-Calif.) vetoed a bill that would have made AI “kill switches” mandatory.
The bill would prohibit a developer from using a covered model commercially or publicly, or making a covered model or a covered model derivative available for commercial or public use, if there is an unreasonable risk that the covered model or covered model derivative can cause or enable a critical harm, as defined. The bill would require a developer, beginning January 1, 2028, to annually retain a third-party auditor to perform an independent audit of compliance with the requirements of the bill, as provided.
This bill would require a developer of a covered model to submit to the Frontier Model Division, which the bill would create within the Government Operations Agency, a certification under penalty of perjury of compliance with these provisions, as specified. By expanding the scope of the crime of perjury, this bill would impose a state-mandated local program. The bill would also require a developer of a covered model to report each artificial intelligence safety incident affecting the covered model or any covered model derivative controlled by the developer, as specified, to the Frontier Model Division.
This bill would require a person that operates a computing cluster, as defined, to implement written policies and procedures to do certain things when a customer utilizes compute resources that would be sufficient to train a covered model, including assess whether a prospective customer intends to utilize the computing cluster to train a covered model.
It wasn’t just a “kill switch for ASI” bill, which is a pretty questionable strategy for preventing rogue superintelligence anyway, even if you accept that ASI is a prominent threat.
Ion Stoica is a professor at the University of California, Berkeley and cofounder of Databricks, an AI company built on Apache Spark. If SB 1047 passes, he predicts that within a year open source models from overseas, likely China, will overtake those made in the United States.
It happened anyway even without this bill, it would not have accomplished anything because they don’t actually have the leverage to lock down development.
SB 1047 also contained provisions designed to deter development and release of open weights models:
It wasn’t just a “kill switch for ASI” bill, which is a pretty questionable strategy for preventing rogue superintelligence anyway, even if you accept that ASI is a prominent threat.
I though this bit from one of the linked source articles was funny though:
It happened anyway even without this bill, it would not have accomplished anything because they don’t actually have the leverage to lock down development.