Approach

What we mean by catastrophic risk, where we build across a catastrophe, and how we choose what to build.

What we mean by catastrophic risk

A catastrophe is an event, or connected series of events, that results in millions of deaths, loss of a free society, or severe impairment of the progress of civilization.

Catastrophic risks can be anticipated, mitigated, and, often, prevented. Many beneficial emerging technologies are also extremely dangerous. Technological change increasingly outpaces societal ability to anticipate and respond to dangers, leaving traditional organizations like governments unable to organize the relevant resources. Typical product companies are incentivized to create dangerous technologies and promote them; self-regulation is insufficient.

Scope and severity

We sort risks by breadth of people affected (scope) and resulting harm (severity). Ultra concentrates on the upper right: local or larger in scope, and endurable or worse in severity.

Example risks by scope and severity. Ultra focuses on local, global, and trans-generational risks that are endurable or terminal.
Scope ↓ · Severity → ImperceptibleEndurableTerminal
Trans-generational Loss of one beetle species Counterforce nuclear war Unfriendly AI takeover
Global Global warming by 0.01 °C Destruction of satellite constellations Mirror bacteria release
Local Congestion from one extra vehicle AI attack on one country's infrastructure Genetically targeted bioweapon
Personal Loss of a single hair Theft of one's car Fatal car crash

Ultra's focus. Adapted from Nick Bostrom and Milan Ćirković (eds.), Global Catastrophic Risks (Oxford University Press, 2008). Examples are illustrative.

The catastrophic-risk lifecycle

Every stage of a catastrophe needs products, not just policy. Ultra builds across the lifecycle.

  1. Detect

    Know that a catastrophe may be imminent, characterize it, and initiate a warning with recommended action.

  2. Prevent

    Stop a catastrophe from happening, including by well-controlled, autonomous response.

  3. Mitigate

    Reduce the consequences by increasing the speed of decisions and actions.

  4. Respond

    Deploy the right resources with a real-time feedback loop that monitors effect and recalibrates appropriately.

  5. Sustain

    Keep successful responses resilient to hostile action from adversaries and from misaligned systems.

Product development pillars

How we decide what to build

Ultra builds a product only when it passes five tests:

  1. It addresses a catastrophic risk.
  2. We have an ambitious yet plausible engineering approach to build an impactful product.
  3. There is a large addressable market for the solution.
  4. We can model a path to high profitability with the product.
  5. Competitive approaches are exceptionally difficult to create and deploy.

“Somebody really should be doing something about this.”

The problems Ultra tackles usually suffer from a coordination failure that keeps governments and private actors from organizing the necessary resources to solve them. Ultra is a new kind of organization, designed to get past those coordination problems and mobilize the resources needed to avert and mitigate catastrophes.

Six members of the Ultra team talking over dinner
Next Products Space & nuclear security, societal resilience, AI safety, and what comes next.