On bottlenecks, magic bullets and optimization

Bottlenecks are ubiquitous in all areas in life. In computing they are often obvious and examine; you can often tell if your system can handle only so many requests, because it lacks one of the following:

  • disk I/O
  • network I/O
  • storage
  • RAM
  • processing power
  • racks for new servers

If your system is limited by any of these factors, performance will not improve whatever you do, unless you can change this, either by throwing hardware at the problem or doing some smart algorithmic sorcery. And these bottlenecks are at least easy to identify, as you only have to look at your monitoring tools. In other areas the nomenclature may differ, but the principle stays the same. For example, in agriculture it runs under the name of Liebig's minimum principle, stating that if things like

  • light
  • water
  • nutritional factors
  • soil salinity
  • soil pH

limit plant growth, it does not matter how much one improves the other factors; further growth will only occur if and only if this so called minimum factor will be improved.

Clearly not every process has a bottleneck. Seldom have to deal with some sort of optimization hydra, where, once you cut one bottleneck, the next one appears. Or there is some single limiting factor, but you are at the moment in no position to change this, e.g. you have only so much land, so much rackspace or so much time. If there is no single one limiting factor which might be removed, you are in the beautiful realm of optimization. Improving any factor might yield some improvement, but often improving a specific combination of factors has a bigger yield. This is beautiful, because it is fun, tweaking the parameters, having some trial and error going on, A-B-testing, nice models and what not. But in terms of effectiveness or cost-efficiency, it is often not as satisfying as removing a bottleneck. As you see, optimization often yields only miniscule improvements in terms of output, say only a few percent, whereas you might just scale out the problem by dumping money and resources at it.

However, optimization might not only improve your current output, but also your return on investment. Taking a step back and playing the metagame, one sees that this is simply another optimization process; do you invest directly into resources or into the optimization how you allocate these resources. Also it might make you wonder, why you have really smart people at one point in the organization, engineering your production processes, but leave the task of organizing or, one might say, engineering your whole operation, to people with some sort of business degree. While I do not hold prejudices against people with a business degree, I know something about people who are afraid of uppercase sigmas and formulae involving more than two letters. But back to business. In fact, identifying bottlenecks is an optimization problem in the classical sense, justifying the nomenclature in many cases, where such a thing is called SomeGuy's minimum principle and in some way it is optimization all the way up.

And then there is always the point of diminishing returns, where progress stalls out and neither optimization nor dumping resources on the problem yields significant improvement. And almost always this is because of some other bottleneck appearing, although at some point it might actually be an insurmountable one. Otherwise, just GOTO 10 and the whole story begins anew.

Now onto the so called magic bullets. You know one when you see one: They are often praised to be the one and only solution to the problem and the testimonials speak for themselves, as there are droves of people who just had this problem, tried everything, but nothing else worked to solve it. Of course most of these things are scam, which is the exact reason why we call them magic. But then again, there seem to be many legitimate magic bullets. The nice observation here is just the following: Most (legitimate) magic bullets are just removing bottlenecks. Pair this with the general inability of the public to reflect things and actually understand what they are doing, and you see how people think that some particular thing is just the shit.

A few examples come to mind:

  • any diet fad
  • any exercise you absolutely should be doing right now
  • deep work
  • flow
  • grit
  • any hip organizational scheme
  • frugality

These things are all fine and dandy and probably solve a big problem for large parts of the general population. But, as anyone who did not suffer from exactly this problem will confirm, they are not the be-all and end-all. They are just part from a larger picture. Indeed, sometimes one magic bullet seem to be somewhat diametrically opposed to another, as for example deep work and flow.

The problems occur once people become dogmatic about their magic bullets, which is, of course, because they do not understand the mechanism how their process works sufficiently. Which is in so far acceptable, as truly understanding how something works, is fucking hard and long work. And sometimes you just do not care about it. And sometimes you really think that you have figured out, explain it to people, people use your stuff and most of them actually have some success. Some don't, but in general there will be some twofold survivorship bias. First, people who finally solve some problem they had for a long time, are happy and excited and tell everyone about it, while people who do not find a solution, are busy with searching for another solution. Second, most problems are somewhat ego-involved. If you earn more money, get leaner, are more productive, whatever, you most of the time think higher of yourself. Now, we are mean creatures by default and if by method X you achieved goal Y, you are prone to deride anyone who does not get Y by doing X. He did not work enough, did not apply it properly, whatever. Maybe you do not do this, but some person will do this and others will chime in. And probably that assumption is right, however, the false positives are harmful for understanding the problem.

If however, you understand the problem beforehand and see a limiting factor, you know what kind of solution you have to look for. And then it is often either obvious or seems impossible.

So, summing up the conclusions:

  • if some growth is stalled out, there is often a single limiting factor
  • if there is no single limiting factor, you have to engineer further growth
  • optimization often only becomes cost effective by increasing the ROI on further investments
  • when magic bullets work, then almost always because they remove some sort of bottleneck
  • being dogmatic about magic bullets is quite useless and hinders understanding
  • if someone tells you about how he finally solved something, ask questions instead of blindly applying that magic bullet, maybe you can figure out why exactly it did work