Approach
What we mean by catastrophic risk, where we build across a catastrophe, and how we choose what to build.
What we mean by catastrophic risk
A catastrophe is an event, or connected series of events, that results in millions of deaths, loss of a free society, or severe impairment of the progress of civilization.
Catastrophic risks can be anticipated, mitigated, and, often, prevented. Many beneficial emerging technologies are also extremely dangerous. Technological change increasingly outpaces societal ability to anticipate and respond to dangers, leaving traditional organizations like governments unable to organize the relevant resources. Typical product companies are incentivized to create dangerous technologies and promote them; self-regulation is insufficient.
Scope and severity
We sort risks by breadth of people affected (scope) and resulting harm (severity). Ultra concentrates on the upper right: local or larger in scope, and endurable or worse in severity.
| Scope ↓ · Severity → | Imperceptible | Endurable | Terminal |
|---|---|---|---|
| Trans-generational | Loss of one beetle species | Counterforce nuclear war | Unfriendly AI takeover |
| Global | Global warming by 0.01 °C | Destruction of satellite constellations | Mirror bacteria release |
| Local | Congestion from one extra vehicle | AI attack on one country's infrastructure | Genetically targeted bioweapon |
| Personal | Loss of a single hair | Theft of one's car | Fatal car crash |
Ultra's focus. Adapted from Nick Bostrom and Milan Ćirković (eds.), Global Catastrophic Risks (Oxford University Press, 2008). Examples are illustrative.
The catastrophic-risk lifecycle
Every stage of a catastrophe needs products, not just policy. Ultra builds across the lifecycle.
-
Detect
Know that a catastrophe may be imminent, characterize it, and initiate a warning with recommended action.
-
Prevent
Stop a catastrophe from happening, including by well-controlled, autonomous response.
-
Mitigate
Reduce the consequences by increasing the speed of decisions and actions.
-
Respond
Deploy the right resources with a real-time feedback loop that monitors effect and recalibrates appropriately.
-
Sustain
Keep successful responses resilient to hostile action from adversaries and from misaligned systems.
Product development pillars
- Preservation of liberty and privacy.
- Vertical integration. Hardware, software, and the human workflows enabled by them, designed together.
- A preference for unclassified capabilities and systems.
How we decide what to build
Ultra builds a product only when it passes five tests:
- It addresses a catastrophic risk.
- We have an ambitious yet plausible engineering approach to build an impactful product.
- There is a large addressable market for the solution.
- We can model a path to high profitability with the product.
- Competitive approaches are exceptionally difficult to create and deploy.
“Somebody really should be doing something about this.”
The problems Ultra tackles usually suffer from a coordination failure that keeps governments and private actors from organizing the necessary resources to solve them. Ultra is a new kind of organization, designed to get past those coordination problems and mobilize the resources needed to avert and mitigate catastrophes.