The Leap of Faith

Notes: emphasize derived objectives (and make the implied objective for landscape clearer), make less convoluted, handwave less and justify more, compare with typical contrarianism (says nothing regarding objective and assumes flaws are derivable given data/priors), emphasize size of the action/outcome space, mention Taleb/Thiel regarding existing ideas on betting (black swan, contrarianism); the priors are priced, the self is not. Will finish this once I resolve my other shit.

Alexandr Wang has an essay about betting on unknown unknowns. The argument is that realistic predictions underperform optimistic ones at the frontier, because the path forward runs through breakthroughs that cannot be charted from where you are standing. mRNA vaccines, Moore’s Law, the Transformer — these were not predictable from inside the paradigms that preceded them. The optimistic bets won because they were positioned where breakthroughs concentrate.

Wang’s advice is to find generating conditions like tenacity and creativity, and bet on environments that have them. You cannot predict the specific breakthroughs, but you can predict that they will happen at frontiers populated by certain kinds of people.

I want to abstract this one level further. The same structure shows up in how people form priors, build their lives, and decide what to pursue. In any domain where the possibility space is much wider than the model of it, the unseen exceeds the seen, which must be priced in your bets. Calculation handles the unknowns you can already name; when you account for things you don’t even know exist yet, the calculation becomes much harder. At some point reliable generating conditions run out, and what remains is the choice of whether to commit to a direction you cannot fully justify from where you stand.

That is the leap of faith.

The basin

A hiker has been walking downhill through fog for most of the day, with visibility maybe ten feet in any direction. They want to find out how low the ground can go. The basin they are in is large enough that the bottom is not something they would reach in their lifetime; however, they still want to try. The slope has kept tilting down long enough that they have stopped noticing, and though this slope has been nice for the most part, the path has grown more circuitous with every hour and each mile now requires more navigation than the last. To get to any other valley they would have to climb, but every step uphill reads as a step in the wrong direction. There is just no apparent way to tell from inside the fog which upward step would lead somewhere else and which would just lose them what they have already made. The cost of turning back grows with every hour they keep walking. The slope is the only signal available, and the slope says they are already going the right way.

Priors are what allow models of the world to be made. They are intuitions about how the world behaves, built from experience and used to reason about new situations without starting from scratch. However, when priors compound invisibly, overfitting to the known environment becomes more compelling. This is in large part due to cognitive biases. Overconfidence, recency, and confirmation bias each contaminate the same loop: the priors you already have shape what you attend to, which shapes the evidence you gather, which shapes the next priors you form. Information-gathering is itself conditioned on what you have already gathered, reinforcing the same model. Over time, this compounding distorts your sense of how big the space is. You don’t just have wrong priors about what’s out there — you have wrong priors about how much “out there” there is. Exploitation feels rational because the map says the territory is small. The map is wrong, but the map is also what you’d use to evaluate whether the map is wrong. The only evidence that would show you the map is wrong is evidence you’d need to already be looking for.

Wang has a case that makes this visible. In early 2020 a reporter at the New York Times built a forecasting model for how long it would take to produce a COVID vaccine. The model drew on historical vaccine timelines, expert interviews, and the best information available. Its fastest projection was 2023. The reasoning was rigorous, and the conclusion was wrong, because mRNA shifted the landscape beneath the data the model was trained on and nothing inside the model’s distribution could have seen it coming.

The reporter was not being careless. He was updating on the information his priors admitted as relevant, and the updates inside that set were correct. What was missing was a way to see that the distribution had changed. Priors set what counts as signal, and evidence that would revise the priors sits outside what the priors recognize as evidence. The map confirms itself, not because it is right, but because it controls the input channel.

Two further features of the world complicate this. The first is that experience is lossy. Nobody observes the full state of anything; perception is compressed through attention, context, and whatever is being pursued in the moment. Two researchers read the same paper and see different things, because their models prepare them to see different things. The second is that the world does not hold still. New technologies arrive, feedback often has high variance, and the surface on which priors were trained moves under them. Priors that were well-calibrated at formation decay, because the thing they were calibrated to has changed.

Even if you could see the world clearly and track its changes, society actively shapes what you do with what you see. People often start with objectives larger than their environment and ambitions that don’t yet fit nicely within the culture’s reward model. Because of societal incentive structures, initially-idealistic objectives slowly become diluted until society stops being something to shape and becomes a home for the domesticated self. The things you want become, gradually, the things that are available.

Inherited objectives also absorb responsibility. If you’re given a direction, you lose full accountability for outcome. Derived objectives offer no such cover. If you chose it and it fails, that’s on you. Settling into what’s given is partly a way of escaping the responsibility of defining meaning for yourself.

The leap

Escaping a basin is not a gradient step. The basin holds even rational people precisely because the basin is rational by every available criterion: the approach is falsifiable, the signal is positive, the updates are correct. The alternative can’t be evaluated on the same timescale. Falsifiability has a temporal scope, and some commitments exceed it.

The inherited self is another basin artifact. Education, job titles, track records — these are all basin-derived conceptions, and are also what many reach for to get a sense of the self. However, the self that can leap is not necessarily the self you currently think you are.

One’s self is unique in this landscape because it is the one the basin cannot lock from outside. Priors are locked by the world that shapes them, and the sense of what is realistic becomes defined by the basin. Information in the basin is often downstream of objects in the basin, and can be systematically biased. The self is not downstream in the same way because its objectives and very definition requires its own cooperation.

The capacity for self-definition is always there. The basin can suppress it through social pressure, through compounding priors, or through the relief of inherited responsibility, but it can’t eliminate it because eliminating it would require the self to cooperate in removing the very capacity that makes it a self. Being basin-locked, for the self specifically, is something you participate in. Which means the possibility of withdrawing that participation is always available, even when the basin makes it feel impossible.

Fast feedback is useful. It is how you get better quickly at whatever you are already doing, and that is its job. Inside a basin, fast feedback accelerates one’s progress and understanding of it. The question then becomes “how long am I willing to be in a worse basin for one that might be better?”. The leap sometimes operates on a different timescale. The commitments that might matter for the leap might have horizons longer than the scope of the tools you trust, and the tools you trust were built to operate inside the frame the leap is trying to leave.

However, urgency can cause a temptation to lean on priors, and accelerating validated approaches is still basin-preserving. The willingness to commit to your bets before positive signal, even despite going against priors, is different. One approach is staying in the basin with a higher velocity. The other is how you leave.

Just as learning and overfitting have the same signature from inside the basin, self-definition and self-delusion have the same signature from inside the self. You cannot tell with certainty whether what you have derived is genuinely yours or a more sophisticated form of inheritance.

I reject any notion of responsibility being offloaded. One must leap anyway.