Musk vs. Altman: AI safety cannot be one man’s job

Strip away the feud, the damages claim, the dueling charisma. What these two men were offering, each in his own way, was a promise that their personal stewardship would keep artificial intelligence safe for the rest of us. Musk said Altman had stolen a charity. Altman said Musk was a wounded co-founder who could not stand losing control. Both arguments rested on the same hidden assumption. The future of AI depends on having the right billionaire in the room.

That assumption is the real problem, and the court in Oakland cannot solve it.

Consider what the trial obscured. OpenAI began as a nonprofit. It layered on a capped-profit subsidiary. It converted to a public benefit corporation. It may soon test public markets. Each form was sold as a guarantee that mission would constrain capital. Each form collapsed under the weight of the capital it needed to function. The clearest evidence is the November 2023 firing of Altman by his own nonprofit board. The board fired him on a Friday. By Monday, Microsoft and seven hundred employees had reversed the decision. The board’s charter said one thing. The capital said another. The capital won.

This is the world we live in now. The most consequential decisions about how artificial intelligence is developed, deployed, and constrained are being made inside a handful of private companies. Congress and the executive branch cannot keep pace with this technology. Even at peak form, neither could write laws fast enough to govern it. And neither is at peak form. Regulators are years behind. Into that vacuum, corporations have moved. They write safety policies, publish model cards, build red teams and disclosure frameworks. This is where the real rules of AI are being drafted.

The trial offered us a choice between two versions of the same flawed answer. Musk wanted to dismantle OpenAI’s for-profit arm and remove Altman, restoring a nonprofit purity the technology’s capital requirements had already rendered impossible. Altman wanted vindication for the path that produced the most valuable AI company on earth and also produced the very governance failures the nonprofit was designed to prevent. Pick one, and you still have a system where the safety of a transformative technology turns on whether the right individual has good judgment on a particular Tuesday. Good intentions are not a governance structure.

We can do better, and the answer is not to pretend Congress will arrive in time. Corporations are now the primary forum in which the rules of AI are being written. The task is to demand they write those rules properly.

That is what AI governance needs. Three things, specifically. First, policy choice. AI companies should set their safety policies through a structured, transparent process, with documented expert input, identified decision-makers, and reasoned justifications, not through unbridled discretion exercised case by case. Second, organized action. Those policies should be implemented through clearly assigned operational responsibility, with senior officers accountable when capability thresholds are crossed. Third, monitoring. Companies should audit their own compliance and report results to a board committee with the independence and expertise to act.

None of this requires Congress to decide what AI should or should not do. It requires the law to ensure that the people deciding inside the company do so through a process worthy of the stakes. Call it a social business judgment standard. Courts already evaluate whether a board took its job seriously when it sold the company or fired the CEO. They can learn to do the same for the decisions that will define the next generation of technology.

Corporate form, on its own, will not save us. A nonprofit charter did not save OpenAI’s mission. A public benefit corporation will not save the next one. The Oakland jury asked whether Elon Musk had been wronged. The better question is whether the rest of us have been. Two billionaires offered themselves as the guarantors of AI safety. We were never going to be safer that way.

Corporations are writing the rules of AI. The law’s job is to discipline how they write.

The opinions expressed in Fortune.com commentary pieces are solely the views of their authors and do not necessarily reflect the opinions and beliefs of Fortune.