Jade Leung: How Can We See the Impact of AI Strategy Research?

The Talk

Technology shapes civilizations. Technology has enabled us to hunt, gather, settle, and communicate. Technology today powers our cities, extends our lifespans, connects our ideas, and pushes the frontier of what it means to be human.

Technology has also fueled wars over power, ideology, prestige, history, and memories. Indeed, technology has pushed us to the precipice of risk in less than a decade — a fleeting moment in the timespan of human civilization. [With just a few years of research,] we equipped ourselves with the ability to wipe out the vast majority of the human population with the atomic bomb.

If we rewind to the emergent stage of these transformative technologies, we have to remember that we are far from being clear-eyed and prescient. Instead, we're some combination of greedy, clueless, confused, and reckless. But ultimately, how we choose to navigate between the opportunities and risks of transformative technologies will define what we gain from these transformative technologies — and also what risks we expose ourselves to in the process.

This is the canonical challenge of governance of these transformative technologies. Today, we're in the early stages of navigating a particular technology: artificial intelligence (AI). It may be one of the most consequential technologies of our time and the most important one for us to get right. But getting it right requires us to do something that we've never done before: Formulate a navigation strategy with deliberate caution and explicit altruistic intention. It requires us to have foresight and to orient [ourselves] toward the long-term future. This is the challenge of AI governance.

If we think about our history and track record, our baseline is pretty far from optimal. That’s a very kind way of saying that it sucks. We're not very good at governing transformative technologies. Sometimes we go down this path and [the journey] is somewhat safe and prosperous. Sometimes, we falter. We pursue the benefits of synthetic biology without thinking about how that affects biological weapons. Sometimes we stop ourselves at the starting line because of fear, failing to pursue opportunities like atomically precise manufacturing or STEM cell research. And sometimes we just fall into valleys.

During the Cuban Missile Crisis, President John F. Kennedy estimated that the chance of nuclear war was one in three. One in three.

The reality is we've been pretty damn lucky. We deserve no credit for [avoiding any of these catastrophes]. But as my swimming coach once said, “If you're really, really, really bad at something, you only need to try a little bit to become slightly better at it.” So here's to being slightly better at navigating these transformative technologies.

I think there are three goals in the AI strategy and governance space [that can help us rise] slightly [above] our currently awful baseline.

Goal number one: Gain a better understanding of what this landscape looks like. Where are the mountains? Where are the valleys, the slippery slopes, the utopias? This is super-hard to do. It's very speculative and uncertain, so we need to be humble. But we should try anyway.

The second thing we can try to do is equip ourselves with good heuristics for navigation. If uncertainty is an occupational hazard of working in this space, then we can try to figure out, in general terms, what might be good and bad [to pursue]. How should we orient ourselves? Which directions do we want to go in?

The last goal is to translate these heuristics into actual navigation strategies. How do we ensure that our heuristics make it into the hands of the people who are turning this boat in certain directions?

If you'll stick with my navigation metaphor for a bit longer, we can think of the first goal as a mapping exercise to determine where the mountains, valleys, water sources, and cafes with good wifi are. The second goal is about equipping ourselves with a compass. If we know that there are aggressive rhinos to the south and good vegan restaurants in the north, we’ll go north instead of south.

This metaphor is kind of falling apart, but the third goal is the steering wheel. You can’t use it if you’re in the back of the car. That's ultimately what I want to focus on today: How do we make sure that [our map and compass will be used to steer — i.e., to make real-world decisions about AI]?

[I have two reasons for focusing] on this. First, AI strategy and governance research is effective when it happens upstream of actionable, real-world tactics and strategies. They can be relatively far upstream. I think we would lose a lot of good research questions if [we were always motivated by] whether something could inform a decision today. But I think it would be a mistake for anyone who does AI strategy and governance research to [avoid] thinking about how they expect their research to [play out] in relevant, real-world decisions.

That leads me to the second reason for focusing: I don't think we know how to do this [make our research actionable] well. I think we invest far more effort into understanding how to do good research than we do into understanding how to [come up with] good tactics. Don't get me wrong: I don't think we know how to do good research yet. We're still trying to figure that out. And I find it hilarious that people think that I know how to do good research; if only you knew how little I know! But I think we need to invest far more proportional effort into [asking ourselves]: Once we’ve done our research and have some insights in place, what do we do to [apply] them and [influence] the direction in which we're going?

With that in mind, let's start at the end. What are the decisions that we want to influence in the real world? Another way to ask this question is: Who is making the decisions that we want to change?

They fall into two broad categories: (1) those developing and deploying AI and (2) those shaping the environment in which AI is developed and deployed.

Those developing and deploying AI include researchers, research labs, companies, and governments. those shaping the environment.

In terms of [the second group], there are a number of different environments to shape:

The research environment can be shaped by lab leaders, funders, universities, and CEOs. They shape the kind of research that is being invested in — i.e., the research considered within the Overton window.
The legislative environment, which constrains what can be deployed and how, can be shaped by legislators, regulators, states, and the people [being governed].
The market environment, which can be shaped by investors, funders, consumers, and employees. They create incentives that drive certain forms of development and deployment, because of supply and demand.

Now, you can either become one of these decision-makers or you can become a person who influences them. This is in no way a commentary on your brilliance as human beings. But none of you will become important. I'm unlikely to become important. The reality is that’s how the world works. If you do end up becoming an important person, the recording of this talk is your voucher for a free drink on me. But if you assume that I'm right, most of you are going to fall into the category of people who influence decisions as opposed to making them.

Therefore, I’m going to [spend the rest of this talk] focusing on this question: How do we increase our ability to influence the decisions being made [about AI]?

There are many steps, but I see them falling into two broad areas. The first step is having good things to say. The second step is making sure that the people who matter [are made aware of] these good things.

A quick note on what I mean by “good”: I'm broadly conceiving of all of us as good in the normative sense of steering our world in a direction that we want, and good in the pragmatic sense, in that a decision-maker will be likely to actually go in that direction because it's reasonable and falls within their timeframe.

Oftentimes these two definitions of good conflict. For example, things you think will be good for the long-term future won't [necessarily] be things that are tractable or reasonable from a decision-maker’s point of view. I acknowledge that these two things are in tension. It's hard to figure out how to compromise between them sometimes.

That being said, I think AI strategy and governance research can aim to have good things to say about a given decision-maker’s (1) priorities, (2) strategies, and (3) tactics. Those are three broad buckets to dig into a bit more.

Priorities: I think priorities are basically people’s goals. What benefits are they incentivized to pursue, and what costs are they willing to bear in the process of pursuing those goals? For example, if you manage to convince a lab that safety leads to product excellence, that can make safety a goal for the lab. If you manage to convince a government that cooperation is necessary for technology leadership in an international world, that can make cooperation a goal.

Strategies: You may aim to have useful things to say about certain strategies that [decision-makers adopt]. For example, resource allocation is a pretty common strategy that one could aim to influence. How are they distributing their budgets? How are they investing in research and development efforts across various streams? You also may have things to say about what a given actor chooses to promote or advocate for versus [ignore]. For example, in the case of influencing a government, you might want them to pursue certain pieces of legislation that can help you achieve certain goals. In the case of labs, you might want them to invest in certain types of new programs or different workstreams.

Tactics: The third area is tactics. These include public relations tactics. What do they signal to the external world, and how does that affect their ability to achieve their goals? And what about relationship tactics — with whom do they coordinate and cooperate? Whom do they trust (and distrust)? Whom do they decide to invest in?

To make this a little bit more concrete, I'm going to pick on an actor who needs a lot of good [advice]: the U.S. government.

One of the biggest risks is that nation states will slide into techno-nationalist economic blocs. The framing of strategic competition that we have around AI now could exacerbate a number of AI risks. I won't go into detail now, but we've written a fair amount about it at The Governance of AI. We want to prevent nations from sliding into various economic blocs and the nationalization of bits of AI research and development.

What would a caricature of the U.S. government's position look like? (I say “caricature” because it's not at all clear that they actually have a coherent strategy.) It looks something like sliding into these economic blocs. And that's a bad thing. Their [overarching] priorities are technology leadership, in both an economic and military sense, with a corollary of preserving and maintaining national security. Costs that they may be willing to bear in extreme circumstances include anything that is required to gain control of an R&D pipeline and secure it within national borders.

Now they are making moves in the strategy and tactics space — for example, announcements of export controls that the U.S. government made in November 2018 indicate that they want to preserve domestic capacity for R&D at the cost of investing in international efforts and transnational communities. They also indicate an explicit intention of shutting out foreign competitors and adversaries. [Overall], their AI strategies and tactics point in the direction of “America first.” And the footnotes there suggest that when America is first, over the long term the world suffers. That's too bad. So, those are the kinds of stances that the U.S. posture points toward.

If one has [the chance to try persuading] the U.S. government, one could aim to convince them that their priorities, strategies, and tactics should move in a different direction. For example, a desirable priority could be technology leadership, but leadership could mean leading with a global, cosmopolitan viewpoint. You [could focus on influencing them to] bear the cost of investing in things like safety research in order to pursue this priority in a responsible way. The strategies and tactics you could inform them of when they conduct this research could [involve international outreach]. With whom should they ally themselves and cooperate? What kinds of signals should they send externally to ensure that others with a similar view of technology leadership will [take steps in] the same direction?

This is the type of decision set that you want to influence when conducting upstream AI strategy and governance research. [Once you] have a broad sense of what you think is good, you have the mega-task of trying to make those good things happen in the real world.

I have a few suggestions for how to approach that.

The first is to [focus on]] a few tractable good things. I say “tractable” here to mean things that will make sense to, or sit well with, decision-makers, such that they are likely to do something about it.

One way to do that is to find hooks between things that you care about and things that a decision-maker cares about. Find that intersection or middle part of the Venn diagram. One canonical bifurcation — which I don't actually like all that much — is the bifurcation between near-term and long-term concerns. Near-term concerns are things that are politically salient. [They allow you to] have a discussion in Congress and not look nuts. Long-term concerns are often things that make you look a little bit wacky. But there are some things at the intersection that could lead you to talk about near-term concerns in a way that lays the foundation for long-term concerns that you actually care about and want to seed discussions around.

For example, the automation of manufacturing jobs is a huge discussion in the U.S. at the moment. It’s a microcosm of a much larger-scale problem [involving] massive labor displacement, economic disruptions, and the distribution of economic power in ways that could be undesirable. That’s a set of long-term concerns. But talking about it in the context of truck drivers in the U.S. could be an inroad into making those long-term concerns relevant.

A similar thing can be said about the U.S. and China. People in Washington, D.C. care about the U.S.’s posture toward China, and what the U.S. does and signals now will be relevant to how this particular bilateral relationship pans out in the future. And that's incredibly relevant for how certain race dynamics pan out.

Once you've filtered for these things that are tractable, then you need to do the work of translating them in a digestible way for decision-makers.

The assumption here is that decision-makers are often very time-constrained and attention-constrained. They will [be more likely to respond to messages that are] easy to remember and [relayed] in the form of memes. And unfortunately, long, well-argued, epistemically robust pieces end up [having less impact] than we would hope.

Superintelligence is perhaps one of the best examples. This is an incredibly epistemically robust [topic]. But ultimately, the meme it was boiled down to for the vast majority of people was: “Smart Oxford academics think AI is going to kill us.” So don't try to beat them with nuance. Try to just play this meme game and come up with better memes.

Here are three examples of memes that are currently in danger of taking off:

The U.S. and China are in an arms race.

Whoever wins will have a decisive strategic advantage.
AI safety is always orthogonal to performance.

It's not clear to me that all of these things are true. And for some of them I'm quite sure that I don't want them to be true. But they are being propagated in ways that are informing decisions that are currently being made. I think that's a bad thing.

One thing to focus on, in terms of trying to have good things to say and making those good things heard, it to translate them into messages that are similarly digestible.

Candidates for memes we might propagate are things like: “the equivalent of leading in the AI space is to care about safety and governance”; “the windfall distributions from a transformative AI should be distributed according to some common principles of good”; and “governance doesn't equal government regulation, so multiple actors carry the responsibility to govern well.” Unless we propagate our messages in easy ways, it's going to be very hard to compete with the bad narratives out there.

The last step is to ensure that [our messages] reach some circles of influence. To do that, model your actor well. For example, if you want to target a specific lab, try to figure out who the decision-makers are, what they care about, and whom they listen to. Then, target your specific messages and work with those particular circles of influence in order to get heard. That's my hot take on how [research] can be made slightly more relevant in a real-world sense.

Some final points that I want you to take away:

Ultimately, the impact of this work is contingent on how good our tactics are. The claim that I've made today is that we need to put far more work into this. I’m uncertain how well we can do that — and how much effort we should put into it. But broadly speaking, as soon as we have relevant insights, we should be intentional about investing in propagating them.

Second, exercising this influence is going to be a messy political game. The world’s [approach to] decision-making is muddled, parochial, and suboptimal. We can have a bit of a cry about how suboptimal it is. But ultimately, we need to work within that system. [Using] effective navigation strategies is going to require us to work within a set of politics that we may disagree with to some extent in terms of values. But we need to be tactical and do it.

Finally, governance is a very hard navigation challenge. We have no track record of doing it well, so we should be humble about our ability to do it. At the moment we don't know that we can succeed, but we can try our best.

Moderator: Thank you for that talk. I’d like to start with something that you ended with. You said that we're dealing with systems that are difficult to operate in. To what extent do you even think it's possible to get people to think more clearly? Should we instead just be focusing on institutional change?

Jade: I think there are things that we need to try out. Institutional change is valuable. Attempting to communicate through existing decision-makers in existing institutions is valuable. But I don't think we know enough about what's necessary and how tractable certain things are in order to put all of our eggs in one basket.

So maybe one meta-point is that as a field, we need to diversify our strategies. For example, I think some people should be focusing on modeling existing decision-makers — particularly decision-makers that we think [have enough credibility] to be relevant. And I think others could take the view that existing institutions are insufficient, and that institutional change is ultimately what is required. And then that becomes a particular strategy that is pursued.

The field is shrouded in enough uncertainty about what's going to be relevant and tractable that I would encourage folks to diversify.

Moderator: You focused on the U.S. government as one of the actors that people might pay particular attention to. Are there others that you would recommend people pay attention to?

Jade: Yeah. I generally advocate for focusing on modeling governments [based in places that are likely to be relevant] more than particular private actors. For example, the Chinese government would be worth focusing on. I think we have a better shot at modeling them based on history. There are more variants and anomalies in private spaces.

Second, focus on organizations that are important developers of this technology [AI]. The canonical ones are DeepMind and OpenAI. There are others worth focusing on too.

Moderator: Someone could construe your advice as trying to understand what's happening currently in the policy landscape and in a variety of academic disciplines that people spend their lives in, and then melding all of those together into a recommendation for policymakers. That can feel a little overwhelming as a piece of advice. If someone has to start somewhere and hasn't worked in this field before, what would you say is the minimum that they should be paying attention to?

Jade: Good question. If you're not going to try to do everything (which is good advice), I think one can narrow down the space of things to focus on based on competitive advantage. So think through which arenas of policy decisions you're likely to be able to influence the most. Then, focus specifically on the subset of actors in that space.

Moderator: And assuming a person doesn't have expertise in one area and is just trying to fill a vacuum of understanding somewhere in this AI strategy realm, what would you [recommend] somebody get some expertise in?

Jade: That’s a hard question. There are a lot of resources that are out there that can help orient you to the research space. Good places to start would be our website. There's a research agenda, which has a lot of footnotes and references that are very useful. And then there's also a blog post by the safety research team at DeepMind — they’ve compiled a set of resources to help folks get started in this space.

If you're particularly interested in going deeper, you’re always welcome to email me.

effectivealtruism.org is a project of The Centre for Effective Altruism, a registered charity in England and Wales (Charity Number 1149828). All content © Centre For Effective Altruism or its respective authors, unless otherwise noted. For enquiries and feedback contact us.

Jade Leung: How Can We See the Impact of AI Strategy Research? | Effective Altruism

The Talk