David R. MacIver's Blog: The Manual that solves all your problems

The Manual that solves all your problems

2 June 2024

This post was originally published at https://drmaciver.substack.com/p/the-manual-that-solves-all-your-problems.

Paul Erdős had a concept of The Book, in which God had written down the shortest most elegant proof of every conceivable mathematical problem. It is, of course, impossible to ever write the Book - it is infinitely large and its contents are intrinsically unknowable. Yet, sometimes you can look at a proof, and go “Surely, this must be from the book”. At least, Erdős could. Erdős was unique, but in much the same way that he could sometimes glimpse the Book, we can sometimes glimpse the mind of Erdős. Our access to the book did not die with him.

This is, in a technical sense, not true. If you were to sufficiently quantify “most elegant”, then there is a mechanical procedure that will take any mathematical problem for which you can demonstrate a proof and return The Book’s version of it. There are a bunch of caveats to this claim, the most important of which is that it’s basically technically impossible even once you’ve resolved all of the fiddly details. This still leaves you with the problem of enumerating all provable mathematical results. Fortunately, you can do that mechanically too!

Unfortunately, thanks to Kurt Godel’s various results, what you can’t do is mechanically determine whether any given claim is provable. That is, the set of provable mathematical results is recursively enumerable (you can write a program that will eventually list all of them) but not recursive (you can’t write a program that checks if a given claim is provable).

What this means is that, theoretically, you can write the Book, but you can never write its index.

None of this is relevant to this article but if you don’t want to follow my random grasshopper jumps, don’t read my footnotes.

But anyway, I haven’t really been a mathematician in some 20 years. I’m still interested in mathematics, but not to the same degree, and I no longer quest after the Book. Instead, I’m looking for the Manual.

The Manual is where you go when you have a problem, or want your life to be better in some way. Whenever you want your life to be better in some way, you consult the Manual, and it walks you through a series of steps and choices for you to make.

I’m imagining something like the diagnostic manual for a complicated machine - not the thin leaflet that you get when you buy it, but the really comprehensive one that when an engineer turns up and they’re a bit baffled, they pull out a giant tome and look up “Frobnicator” and then follow a flow chart that says “If when you pull out the doodad gauge it makes a whistling noise, turn to page 524 where we will walk you through purging the thingummy channels…” Elsewhere there is a detailed discussion on the various tradeoffs in price, efficiency, and maintenance burden of various different types of oil you can put in the frobnicator.

Engineers reliably use a mix of their own working knowledge and the manual, with a general distrust of the manual as a source of truth because of how necessarily they oversimplify the world and try to replace skilled understanding with mechanical rote. I’m imagining a platonic version of this where the procedures really are good enough to be reliable, but it’s still true that it’s trying to replace skilled understanding with mechanical rote. You don’t need the manual when you already deeply understand the problem, you need it when you start to feel out of your depth.

You could imagine a manual like this for just about anything. Indeed, a sufficiently comprehensive manual like this for anything has to be a sufficiently comprehensive manual for everything, because everything is connected to everything else.

Consider a manual such as this for your car. Suppose, for example, some utter degenerate of a human being had for some incomprehensible reason decided that your car should just randomly play music whenever you turn it on, or unplug your phone, or interact with it in any way. You’re trying to disable this behaviour because you are a normal human being with normal human preferences who would never even consider this is a reasonable thing for a car to do because what the hell man?

Anyway, you consult the manual for this. Ideally the manual tells you the simple way to disable this feature. Now, suppose that no such simple way exists because the perpetrator of this feature needs to be tried for war crimes. Well, now, please turn to the page where the manual explains what to do to calm down from a screaming rage…

Again, hypothetically, and not based on any specific cars such as the 2017 Ford Focus that I drive. Though should you find yourself in such a hypothetical scenario, it might interest you to know that two separate groups of allegedly human beings have decided this is a good idea. One of them is the Android Auto team, and you can turn off the thing where your phone autoplays when you plug it in by going to the Android Auto settings and turning off “play media automatically when driving”. The other people who did this are the Ford Focus people themselves, who think that turning the radio on when you start the car is a good idea, presumably because they are in league with Satan. You cannot fix this, as far as I can tell, but what you can do is physically remove your radio antenna, at which point the car will start with static rather than loud music, which is at least an improvement.

Similarly, the car manual tells you how to do maintenance. And how often to do maintenance. But what if you’re not very good at remembering to do things…? Well, turn to the part of the Manual where it gives you helps you learn to deal with poor organisation…

The book that has all these procedures in it is The Manual.

The Manual is an infinitely large book which, whenever you have a problem, you consult the Manual and it will tell you the best path for you to take to try to solve it.

Importantly, the Manual is not a genie. It doesn’t know anything about your situation, it doesn’t do anything for you, it doesn’t even promise success if you follow its procedure. It merely provides clear procedures for you to follow that are more likely to work than anything else you might try.

This, necessarily, includes diagnosis - the Manual doesn’t know what your problem is, you have to follow its procedures for figuring that out. Probably diagnosis will be mixed in with problem solving - try this thing, maybe it’ll solve your problem. If not, depending on what happened, turn to this page or that page and…

“That seems like a problem. Have you tried solving it?” is one of my catch phrases, and I think one helpful way to start on something like this is “If I looked in the Manual, what would it tell me to do?”

When is the Manual useful?

I think one important thing to understand about the Manual is that it should rarely need to be consulted. It’s a book that tells you how to solve any problem, but generally speaking most of the problems you encounter should be ones you’ve got a pretty good idea how to solve.

This is as I understand it people’s relationship with actual shop and service manuals. You don’t pull it out until you get stuck, because if you know what you’re doing it will always be faster to just do it than it will be to pull out the service manual and tediously work through its procedures.

The purpose of having the shop manual is that, even when you don’t use it, it gives you a reliable slow fallback plan that means that many more problems are solvable than your current expertise allows for.

It doesn’t preclude the need for general competence in the area - if you give me and an experienced mechanic the shop manual for my car and tell us to fix some problem with the car neither of us have ever seen, the mechanic has a lot more general skill to draw on, and will do much better. Similarly, the Manual will draw on your existing expertise.

When you lack relevant expertise it can take you through the longer, slower, methods, some of which will result in you developing expertise, some of which will help you work around your lack.

I think a good way of thinking about this is that the manual gives you a way of operating competently inside your zone of proximal development. It helps you evolve competencies by giving you slow methodical ways of doing the things that, if you do it often enough, you will want to be able to do fast.

On the impossibility of the Manual

One of the big differences between the Manual and the Book is that the Book expresses timeless truths, and the Manual… still expresses timeless truths if it’s to serve its function of being universal, but the first section of its diagnostic procedure is something like “OK, in order to know which of these timeless truths to present to you… what time are you in?”. It is timelessly true that if you are in the UK in the early 21st century while in possession of a 2017 Ford Focus…

Even within a given time and place and physical context, the Manual has to find out all sorts of things about you before it makes sense to start problem solving. e.g. the solution it will give you if you’ve got some skillset or characteristic that makes a problem easier. If you say to the Manual “I’ve got this large box I need carried up three floors”, if you’re a burly young man in his twenties it’ll say “Lift with your legs, not with your back”, if you’re a small old lady in her eighties it’ll say “OK here’s how to find some local helpers in a hurry…”.

In some sense it makes more sense to think of the Manual as ever-changing and personalised. There is one Manual per person, and that Manual is constantly being updated as that person changes and changes context.

I don’t like this view.

The reason I don’t like this view is that it makes it very hard to think about shared access to the Manual. Yes, the answer I get when I consult the Manual is not going to be the same as that when you do”, even with very similar problems, but to the degree we are similar people we will get similar answers.

When imagining using the Manual though it’s probably best to assume that a lot of questions are filled in, decision trees implicitly taken, etc.

Both parts of this are important: Understanding that the things in your “version” of the Manual are also in other people’s, but also that the more like you someone else is the more likely they are to need to read from the same pages as you.

Catching glimpses of the Manual

Despite the impossibility of reading from the Book, Erdős frequently declared that a proof was or wasn’t from it. Erdős could get away with this because he had almost unparalleled mathematical taste, to the point where one imagines that if there really is a God writing a Book, when Erdős appeared in the afterlife God may well have asked him to consult on the project.

Unfortunately, I’m no Erdős, so I don’t have anything close to his access to the Manual. Fortunately, bits of the Manual are easier to access than others.

I imagine the Manual as consisting of essentially two types of material:

A collection of very well explained procedures - things that you can do to achieve some effect.
An organising diagnostic tool that tells you which procedures to perform in what order by asking you a series of questions, telling you which procedure to perform, and what to do with the results of them.

The hard part of accessing the Manual is the diagnostic tool. The procedures are easy - pretty much anything you can physically do is in the Manual as a procedure, because it solves the problem “I want to do the thing”.

Most reasonable procedures solve more than that though.

For example, a very general purpose procedure that works in a lot of different scenarios is “Turn it off and on again” - take the device that’s causing problems, turn it fully off, turn it back on again, and see if it’s still showing the problem.

Another very general purpose procedure is “Write down all the steps you need to perform to do the thing in enough detail that you have no uncertainty left.”

These are pretty broadly useful tools, and as described are very simple, though they do hide some complexity. For example, to turn something off and on again, you first have to answer “How do I turn it off safely?” and “How do I turn it back on safely?”.

You also might have to figure out what the actually relevant “off and on” states are! Once I’d borrowed my parents’ old land rover. I’d driven it somewhere, got back in, and the engine wouldn’t start. I tried multiple times, turning the ignition off all the way, taking the keys out, etc. Nothing worked.

Eventually I called dad up and he said “OK, so I know this is ridiculous, but have you tried locking and unlocking the doors?”

Turns out the problem was that the car alarm had not properly disengaged, and so was blocking the engine from turning on. The thing that needed to be reset to a known good state was not the engine but the alarm.

In retrospect there was a flashing red light on the dashboard that indicated this, but I had no idea what it meant at the time. Dad knew how to solve this because he had encountered this very specific problem before.

In this case I knew the procedure, and I knew to apply it to the problem, but there was a secondary issue of figuring out the right property of the problem to apply it to.

Sometimes the answer will be “You can’t”. e.g. this is not a good procedure to perform on a human . It’s also not an easy procedure to perform on things that you would sort of logically expect you could - e.g. yes you can in theory turn the national grid off and on again, but the “on again” part is pretty under-tested and maybe don’t?

The point though is that “try turning it off and on again” is a pretty generalisable and usually easy procedure, that you can apply over and over again, and will achieve useful results and provide useful information in a wide variety of circumstances.

The result is that it’s likely to be a very commonly referenced procedure in the manual, and I think we can often glimpse what these look like.

Their two key characteristics are:

They are relatively simple.
They can be usefully applied in a reasonable variety of situations.
12
That’s not to say that they can’t be quite specific. e.g. you probably want to have a reliable procedure for giving a pill to a cat. You do not need this procedure outside of the very narrow task of giving a pill to a cat. But you probably don’t want a “Give this one pill to my cat Elvis when it’s a Tuesday and it’s raining outside” procedure.
Applying them is unlikely to be a bad idea, or the procedure comes with built ins to make sure you check that it’s safe first.

It’s possible the platonic manual doesn’t do this, but instead walks you through a variety of hyper-specific procedures for your problem, but I don’t think so, because one of the factors that you need to take into account when deciding on a procedure is how well you’ll do it, and how familiar you are with the procedure being performed is a big indicator of that. These simple, general, procedures end up forming the foundation of the Manual because you know how to do them and thus they make good starting points.

How do you figure out these procedures?

Well, trial, error, and refinement. When you do something that works, notice that, figure out what about it made it work, what you could do better. Are there any extraneous bits? Can you fix those? Can you add bits in to make the hard parts easy?

Unfortunately, the procedures section of the manual is the relatively easy part to access. The hard part is the decision tree.

How to pick a procedure

The thing about super formalised decision procedures is that they’re mostly not how experts make decisions. They’re useful when you don’t know what you’re doing, because they walk you through all the steps of decision making, but real expert decision making isn’t this sort of formalised decision tree of if A then do B else do C, it’s pattern matching based. You see a situation, it reminds you of something, you consider doing the thing that your experience suggests is a good idea from that. Then you critique the suggestion and discard it if it doesn’t work. This is what expertise looks like.

This is a much more intuition based process than the Manual’s detailed procedures, which is unfortunate if you’re in a novel situation where you don’t have much intuition for it yet, but it can serve a similar function, and I think there are ways to work around its limitations.

Specifically, you try to become an expert in the procedures rather than the problems, particularly procedures that come from the main section of the Manual that I described above that are quite general and usually safe to apply. This means that when you encounter a novel problem you can try to go “OK, what procedure do I apply to this?”

This may or may not work, but you win either way: Either you achieve something, or you learn something more about the procedure and where it’s useful. This might result in you refining it further, or it might result in you realising something about its limitations or the nature of the problem that mean it can’t work, which will let you avoid using it where you shouldn’t in future. It might also just mean you need to practice it more.

When this happens it’s often worth taking a little time to pause and reflect about it to really increase the learning - do a mini retrospective of “Why did this work / not work? What can I learn from that?”. It doesn’t have to take a long time - often like a minute or two of reflection is enough - but it’s worth trying to really cement the learning.

The big advantage in terms of thinking this way is that these procedures are refinable, and learnable, and they turn your problem solving strategies into concrete operations that you can work on and improve.

Thinking of them in terms of access to the Manual helps, in much the same way that Erdős’s Book does. When you solve a problem, you can look at how you solved it, and ask “Was that what the Manual would have told me to do?”

When the answer is no, learn from that - note the problems, see how they could be better.

When the answer is yes, write it down, because you’ve glimpsed a fragment of the divine, and that should be remembered and shared so that future you and others can benefit from it.

Some books you might like

Here are some books that feel vaguely related to me that I think you’ll enjoy:

Talking about machines by Julian Orr - this is a great book about photocopier repair technicians working at Xerox, and includes some interesting bits about manuals.
Working knowledge by Douglas Harper - this isn’t much about manuals at all, but it’s a great ethnography of some guy’s workshop in a small community in rural America, and if you liked Talking about machines you’ll like this.
Sources of Power by Gary Klein - about how people actually make decisions. It’s informed a lot of my understanding of expertise, as has his subsequent work.