Belief Daemons

Overview §

Our survival depends our collective decision making ability, which depends on the quality of our collective understanding, and our ability is being outpaced by our need. Can we use the exponential technology curve threatening us to advance our resilience towards it?

This is a sketch of a fractal system that can operate at all levels: humankind, nations, cities, organizations, and individuals.

I still don’t know if this is really a good idea, but I’ve written this from the perspective of an advocate so that I can try it on for size.

Individual Maps §

The foundation of this proposal is that every individual has access to a personal artificial intelligence called a belief daemon. This intelligence helps them maintain a ledger of beliefs they endorse, called a belief map. The individual has an ongoing informal dialog with their daemon to grow their map. Some individuals would drive this dialog, using their daemon as a thinking tool, scribe, and journal. Some would not, yet still their daemon would drive forward this process based on the goals and demands of the concentric levels of organization encapsulating the individual.

Belief maps need to be as formal as possible while still being easy enough for someone of average literacy to understand them. They should not be opaque databases that only machines can read, for reasons I believe strongly but which would be tangential (see structuring).

Thus, one way to build a map is as a collection of argument trees, which are directed acyclic graphs of statements and reasoning.

One simple and informal example of such an argument:

• I should be vegan

  • Because all of
    • Being vegan minimizes suffering
    • I should minimize suffering when easy
    • Being vegan would be easy
  • Because all of
    • Being vegan results in a smaller CO2 footprint
    • I should minimize my CO2 footprint

• We should build new power plants

  • Because all of
    • Power plants lead to net human flourishing
    • We should seek human flourishing
  • Building power plants maintain national sovereignty
    • Because all of
      • If we stop building power plants we risk losing the capacity to build more later.
      • Sovereign nations can build their own power plants.

I have spent years building such trees for myself, and have found that even simple informal ones are labor-intensive. Formal ones (see formalisms) are worse. Thus I believe they must be created with the aid of an external intelligence. If everyone is to have them, that intelligence must be artificial. To put it another way, a Belief Daemon has to be an AI running on your phone.

When a belief daemon is helping grow your map, it could feel like a casual conversation. I do not imagine everyone will begin a habit of bookish, philosophic pursuit. Instead I imagine that as someone goes about their day, driving or going for a walk, they have an opportunity to have a dialog with their daemon. They may be aware of books, articles and media you consume, and may then ask you what you think about them as a tactic to grow the map.

Of course, a map won’t be a full or accurate ledger of what is in an individual’s mind, that is neither possible nor desirable, and yet that (impossible) goal is still what the daemon aims for.

Organizational Maps §

From this foundation, we can build upwards. Consider a small group such as a startup, activist organization, research outfit, or think tank. For those choosing to evolve past top-down command and control structures, the goal must be faster convergence of beliefs between members on topics relevant to the organization.

Today, when members of such a group disagree on a policy, they must express their entire opinion, hear others, debate, and repeat, until converged, or more commonly, until enough get exhausted that holdouts “win”. Instead, imagine that questions are entered into the organizational map, and members’ daemons are asked to collaboratively fill the argument map by answering it. Two processes run in parallel, one of finding the beliefs of individuals (growing their map to include current beliefs on this question) and a second of feeding back group opinions to individuals to see if convergence can be reached by changing minds or reframing the question. Just as one person’s mind and belief map are engaged in a reciprocal process of convergence, the group’s shared map and the member’s personal maps are in a process of convergence. All three layers are in a process of simultaneous convergence.

Complete convergence will not happen, but it will become evident which parts are agreed on, and which parts are a frontier for future conversation. While a startup will likely keep their maps secret, simply using the shared map to act on the world, groups interesting in guiding their wider society may export their best formed arguments to adjacent groups or layers above.

Societal Maps §

I hope the fractal arrangement of multiple layers is becoming clear, even if the exact process is not, such that you can imagine the maps that represent organizations of organizations: cities, political parties, and even nations.

A society that has cultivated this sort of collective mapping is going to have much better governance, for two main reasons. The first is by increased agreement by experts and technocrats about what we think is true. What is our model of climate change? How effective will a given act of humanitarian aid be? What is the landscape of risk from advancements in AI? The second is by increased agreement among citizens on what we ought to do. What is our time preference for spending money on climate mitigation? What is our obligation to offer humanitarian aid? How do we value the economic advancements promised by AI, and how do we weigh various chances of catastrophic risk attached to it?

For a practical and fleshed-out version of this we could start building today, see my post citizens-lobby

Towards Global Coordination §

My ultimate hope is that this fractal arrangement of maps improves global coordination. Consider that transnational multipolar traps come in two flavors, externalized costs (CO2 emissions, overfishing) and arms races (biological, nuclear, and space). Some, like AI development, are a swirl cone of both. The presence of better collective maps won’t present a direct solution to such traps, but it makes them more tractable.

First, a global exchange of belief maps can make it easier to debate and converge on what is true (relevantly, the nature of existing threats). The political class engaging in negotiations is more likely to find cooperation if all participants have legible and overlapping concerns. We have seen that when threats are legible and shared, such as the threat of biological weapons, the coordination on mitigation has been effective. The map-making I propose is an attempt to bring complex paths to ruin, such as ecological harm or AI development, into such a category of shared legibility. When a threat is made clear, the interventions and agreements needed to address it will also be made clear.

Second, a global exchange of belief maps reduces the sheer imposed by language barriers. Presently we have one collective conversation about the possibilities and threats of AI in English while having another in Chinese. I do not observe any attempt at comparison, differentiation, or general synthesis of these two conversations. Imagine if these two conversations were reciprocally tightened into converging belief maps. Such a state of affairs would help the expert class align on potential perils and the political class align fruitful lines of negotiation.

Building this may seem impossible, but laying out steps may help. The first is to push the state of belief maps forward by grounding them in a useful application. One option is a citizens-lobby but a more modest one might be in organizational deliberation tools. Next, we need AI intelligent enough, and cheap enough, to serve as daemons. My tests so far indicate LLMs are just not quite good enough, as the process of architecting maps is actually rather difficult, and thus I suspect another breakthrough of the same scale as that of the transformer architecture that powers modern LLMs is still necessary. If it never comes, I’ll be relieved, as it is the very fear of this breakthrough that makes me think we need a reciprocal breakthrough in global coordination. Finally, we’ll need to build the three-level convergence I wrote about in the organizational maps section. At this point, we’ll have an organization using the concept, and if the concept is any good, the organization should be able to promote it and seed a fractal deployment around and above it. While I haven’t committed myself to doing the above, I am eyeing the opportunity to do so, and thus I welcome feedback, criticism, or collaboration.