Optimize Limiting Factors

Critical Fallibilism says it’s important to differentiate ideas in terms of success and failure at goals rather than differentiating them by degree of goodness. Why? It’s technically correct and I’ve explained various reasons. Here I’ll focus on one major point: most degree gains are useless. Consequently, it’s a bad idea to try to increase positive factors and decrease negative factors in order to maximize an overall score. In other words, most changes to most factors should be worth zero points or have zero weight.

Most optimizations of a factor (increasing a good factor or decreasing a bad factor) don’t really matter. Why? Because we already have enough of most good factors (and a low enough amount of most negative factors). Getting more of something when you already have enough is not useful. A large change could matter, but an incremental change is ignorable.

If most changes are worth zero points, then the weighted factor approach to decision making is wrong and misleading, because its conclusions are sensitive to small changes to any factors. Any small change to any factor results in a (small) change to the overall conclusion. Many people think it’s good to be sensitive to any changes – it’s similar to someone saying that they will update their conclusion based on any new evidence (although it may be a small update). They think it’d be irrational to get some new evidence and make no adjustment to your conclusion. But it’s actually better to have robust conclusions that are insensitive to most details. If most details are irrelevant, then you can focus your attention on key issues.

Another way to view the issue is that most factors are non-linear, and treating them as linear is not a reasonable approximation.

Another explanation it is that there’s already a well known theory about systems that are sensitive to tiny changes. It’s called “chaos theory” because those systems are very chaotic and unpredictable. Chaos is bad and we should try to avoid it. So we should reach conclusions using thinking processes that aren’t sensitive to tiny changes.

Another perspective on this is that errors are common (especially small errors), so we want to think in ways that are resilient to error. People make mistakes and there are random fluctuations (variance) in the world. So we want to have some margin of error, which means most small differences will fall within the margin of error and won’t change the overall result. Being sensitive to any difference means not having a margin of error, not having a buffer, not having excess capacity.

Elevator Example

Consider an elevator that can lift 5,000 pounds. A family weighing a total of 750 pounds rides the elevator up to their hotel room. Lift capacity of an elevator is a positive factor (more is better; less is worse). If the elevator could lift 5,001 pounds, would the family have a better experience? No, it’d be the same. The extra pound is worth zero points of benefit. Even if it could lift 10,000 pounds – double – it’d make no difference to the family, and be worth zero additional points. It already lifts more than enough. If it could lift 4,900 pounds, that also wouldn’t matter and should subtract zero points, because there’s still plenty of excess capacity left. If it could only lift 200 pounds, that would matter. Some extreme differences do matter. A 200 pound lift capacity would result in failure at the goal of lifting the family, rather than success. Differences matter when they make the difference between success and failure. Most differences don’t do that.

You shouldn’t try to multiply the number of pounds an elevator can lift by how valuable a pound of elevator lift is, nor do any non-numerical, intuition-based version of that. Instead, you should look at whether the elevator can lift enough, or not, for some goal (such as to lift the heaviest passengers who you expect to use it, plus 25% more for a margin of error). Including a margin of error in your goal makes the elevator insensitive to most changes. You don’t have to worry about most small issues because the margin of error will handle them.

Elevator lifting capacities vary all the time due to e.g. the temperature of the steel cables changing. Does that mean how good the elevator is keeps changing? No, it’s still the same elevator. As long as nothing significant changes, we should have a stable opinion of it.

Optimization Away from the Bottleneck Is Wasted

As Eli Goldratt taught us with his Theory of Constraints, we must learn to think in terms of limiting factors – bottlenecks, constraints, key factors. Optimization of a non-limiting factor is wasted, because you’re still limited by the limiting factor(s). We already have extra of the non-limiting factors. And we should design systems that way on purpose: we need extra for most factors. We shouldn’t try to trim the fat and have exactly enough of everything, because then the system will fail when there are random fluctuations.

Goldratt even argues that, in some sense, there is only one bottleneck or limiting factor per system – as a chain has only one weakest link.

Foot Race Example

If you’ve already eaten enough, then you can’t run faster in a race by eating more, because available calories are not your limiting factor. It would be incorrect to say that energy from food is a positive factor, assign it a weight like 3 points per calorie, and estimate how well you’ll do in the race by summing positive factors (and subtracting negative factors). It would appear absurd if someone tried eating ten main courses worth of food before their race in order to increase a positive factor and therefore get a better time. You can’t just keep adding goodness with the easiest-to-get factor, because you get enough of it and become limited by something else like muscle fitness. Overeating would actually do nothing or, if it was too extreme, lead to a worse race time.

Woodworking Example

Consider two people with a woodworking hobby. One has plenty of money, wood and tools, but he’s really busy. His limiting factor is time. If he had more time, he’d get more woodworking done. If you give him a pile of wood, it won’t help. But score systems, factor weighting systems and their non-numeric equivalents would count both wood and time as positives, and assign more points for more wood. Wood is a positive factor but he already has enough. That’s how most factors are in life in general. Only a few factors are key, limiting factors that we should focus our attention on, optimize, and try to get more of (or less of for negative factors).

The second woodworker has plenty of free time, and has tools, but he doesn’t have enough money or wood. Giving him a pile of wood would help him get more projects done. Giving him tools would not help (what’s he going to do with two hammers?). Teaching him how to work faster, to save time, would not help increase his woodworking output either since time isn’t his limiting factor.

Libraries Example

Imagine a university that wants to do better in college rankings. One of the factors considered is libraries, so they build 500 extra libraries in order to rank higher. Does that make any sense? Or were the first few libraries good enough?

In general, factors have specific ranges: too little (0 to 1 libraries), enough to work but more would be better (2 to 3 libraries), enough and more doesn’t really matter (4 to 8 libraries), and too much (9 or more libraries). Too much means definitely not useful, and actually cluttering things up, wasting resources, and getting in the way. The numbers I gave are just examples that would vary by university size and situation.

Most of the time when we deal with something, most factors are already in the good range (not too much nor too little) and don’t need to be adjusted. They have a margin of error in both directions (less or more), so differences in the factor don’t really matter and shouldn’t change our conclusion or overall evaluation. In other words, most large universities already have a reasonable number of libraries, and an increase or decrease of one book worth of library capacity should not affect your evaluation of the university. A change of a whole bookshelf of library capacity shouldn’t affect your evaluation of the university. For larger universities with multiple libraries, one whole library more or less could easily be within the margin of error.

Some people are proud to update their opinions based on all new evidence (such as finding out more precisely how many books worth of library capacity a university has, or finding out that they built or removed some library capacity). They say this works because if the factor has only minor importance, then their update will be small.

I say we should instead be proud if our opinions are resilient enough that they usually don’t need to be updated. We should be looking to reach stable conclusions that have substantial margins of error on most if not all factors. If there are a lot of factors where we don’t have adequate margin of error, then we’re in a bad situation – we need to gain margin of error (e.g. by improving the factor, getting more accurate data, or understanding things better) or else reach a conclusion like “this system is pretty unstable”. Inadequate margin of error can be due to our understanding or due to the underlying reality itself that we’re trying to understand.

Three Categories

There’s a standard pattern for how to look at many factors: there’s too little, a good amount, or too much. You can put the factor in one of those three categories.

This is partly due to our use of numbers (reals or integers). If we can represent a factor with a number, then it can go up or down. That’s how numbers work: they can change in exactly two directions. We choose what we think of as “one factor” partly by what we can represent with one number, so we end up with factors that deal with a single dimension (like a number line). If a factor had more dimensions and needed to be represented by multiple numbers, then we’d see it as multiple factors that haven’t yet been isolated. We do make some special exceptions like position in 3-dimensional space, which we often view as a single thing because we live in 3-dimensional space.

Some factors only have two categories because either too much or too little doesn’t apply. For example, how could you have too little pollution? So pollution would be either a good amount (small enough to be harmless) or too much. (Actually it’s possible that our body and immune system needs some pollutants, irritants and germs to practice on, and an overly sterile environment could be harmful, especially for children. Dealing with minor problems could help it be more robust. Real world examples are often complicated or context-dependent.)

Sometimes there are acceptable amounts that are a little high or low, so we’d like to adjust them but they can work. In other words, they’re near the borderline between categories.

To be clearly good, a factor needs to not only be within the good range, but be an appropriate margin of error away from the border with too much or too little. In a way, we can expand the number of categories to include borderline-too-little and borderline-too-much. Those extra categories mean it’s in the margin of error between two original categories, rather than being clearly in a single category.

Most factors should not be within the margin of error between categories. If many factors are borderline, then the system you’re looking at is unstable or you don’t know enough about it. In order to work with something effectively, you need most factors to be good enough, so that you don’t have to worry much about those factors (since you know what category they’re in and they have enough stability plus margin of error to stay in that category, and also they’re in the good category). Most factors need to be good and stable – they just work, they’re robust, and they don’t need optimization. You can only focus attention and optimization effort on a few factors, not split your attention to dealing with everything at once.

This model is dramatically different than trying to be infinitely precise and being responsive or sensitive to every difference in every factor. Instead, it tries to find ways to ignore most things so that we can focus our attention on key issues. We want most factors to be in a good, stable state so that if they have small changes it doesn’t matter. A reason this is important is because the world is full of small, random fluctuations (variance).

Limiting Factors and Excess Capacity

Good judgment involves looking at the specific factors that are limiting reaching specific goals, rather than looking at whether a factor is good or bad in general. In other words, adjust your thinking to your specific context. In other words, don’t multiply the amount of each factor (like skill-adjusted time available or quality-adjusted amount of wood available) by how good it is in general. That will not give you a reasonable guide to what decisions, actions or interventions will be effective for what goals. You can’t just say that time and wood are both good, so more of either will help. You can’t just assign a value to an hour, a value to a log, and add up the total value to find out how good a woodworker’s situation is. You can’t combine different dimensions into a single conclusion like that.

A standard objection people would have is that adding up time and wood omits other factors like having a roofed area to keep your wood dry or having strong enough arms. But my criticisms weren’t about incompleteness. Even if you have a complete list of factors, the approach of adding up weighted factors doesn’t work. Most factors have excess capacity.

If you actually have a situation without most factors having excess capacity, then that situation is unstable. It’s chaos. Or it’s a new thing that hasn’t worked yet. When you start a new hobby, you’re missing lots of required things. But if you’ve been doing the hobby for a while, and things have been going OK, then most factors have excess capacity (otherwise things wouldn’t have gone OK).

If a factory has run for a while, and built stuff, then most factors involved in production have excess capacity. But if you’re building a new factory and haven’t started yet, then you may have many limiting factors. You need walls and all the tools and people and everything. Without adding all those things, the factory won’t work. This can be seen in Goldratt’s chain analogy. A chain has one weakest link. But that assumes the chain is actually built and functional. If you haven’t made the chain yet, then every link is a limiting factor – they all need to be made before the chain will work. So it’s important to differentiate between functional systems (most factors have excess capacity) and systems that haven’t functioned yet (they may have many limiting factors before they will work).

Whenever you see something reasonably orderly and successful, that means that most factors have enough excess capacity to handle random fluctuations and minor mistakes and still work. That makes it possible to focus attention on a small number of key factors without your attention getting chaotically dragged around to other issues all the time. Having a bunch of factors with too little, or barely enough (inadequate margin of error against random variations) leads to crisis mode and “firefighting” (rushing around to put out fires in different places, with “fires” being a metaphor for crises).

It’s important to stop thinking “more of a good thing is approximately always better” (which people already know is wrong in some contexts – e.g. water is good but too much will drown you). Instead, think “only a minority of good things are currently important to get more of; for most good things we already have enough (in reasonable, regular, non-crisis situations)”. Focus your optimization and analysis on a few key factors. Key factors are factors that will actually lead to a significantly better outcome if they’re improved. A better outcome is success instead of failure at some goal you care about, such as completing an additional woodworking project instead of not completing it. This approach works better than trying to add up the total goodness of all the positive factors.

Some Factors Are Negligible

Another common, related reason that incrementally more or less of most factors doesn’t matter is that some factors are orders of magnitude more important than others. Programmers trying to speed up software know to look for bottlenecks and make those faster. Speeding up non-bottlenecks doesn’t matter even though it does in fact make the software faster.

Why? Because a bottleneck might take 30 seconds, which is why the software is slow, while some other function might take 40 milliseconds (0.04 seconds). That kind of time difference is realistic and common – some functions in software are much, much faster than others. Speeding up from 40 milliseconds to 35 milliseconds won’t matter and isn’t worth a programmer’s time and attention (plus making any change risks creating a bug, so even if the change took no time, it still might be better to leave the code alone).

When over 95% of the time for software to run is spent on one to three slow functions (the bottlenecks), then it’s crucial to focus optimization efforts on the slow functions, not waste your effort speeding up anything else. Most of the functions are already fast enough – they already have excess capacity regarding speed – even though the software as a whole is not fast enough and you still want more speed.

Many real-world systems are like that. The Pareto Principle says 20% of the causes are responsible for 80% of the effects. But that assumes independence. As Goldratt explained, it’s more like 1% causing 99% when you deal with systems with a lot of dependencies (which are common). So we should focus our improvement efforts on the few factors that cause most of the effects.

Non-Linear Factors

Another useful viewpoint is that performance of most factors is highly non-linear. There’s too little (or too much) and there’s enough (including a margin of error), and that’s it. A linear factor means each unit or increment of that factor adds about as much value as the previous one. A non-linear factor means that some units are worth much more than others. For example, water has high value when you’re thirsty, but low value once you’ve drunk enough, so it’s non-linear. For negative factors, there’s too much and low enough, which is also non-linear. Linear is like a straight line, but most factors instead have breakpoints – cliffs, jumps or discontinuities. We should evaluate factors by what side of a breakpoint they’re on – or in other words what conceptual category they’re in – not by degree or amount.

A breakpoint is a point on a spectrum where something important changes. It’s a point where there’s an explanation that differentiates things on either side and puts them into different categories (such as enough vs. not enough, or enough vs. too much). Breakpoints are sparse – in other words, there are many, many more numbers than breakpoints, so most numbers aren’t near a breakpoint. Most factors are not near breakpoints. You should only optimize the few factors that are near breakpoints. Those are the limiting factors such that changing them actually matters.

If no factors are near breakpoints, then it doesn’t need optimization. Your effort would be better spent elsewhere. If you really want to improve it, you can come up with a more ambitious goal, which can move the breakpoints.

Bottlenecks are factors that are near breakpoints, and non-bottlenecks are not near breakpoints (they are more than an appropriate margin of error away from any breakpoint).

You can learn more about bottlenecks, constraints and limiting factors (and excess capacity and focus) by reading Eli Goldratt’s books like The Goal: A Process of Ongoing Improvement. Note: The application of these concepts to epistemology, and discussion of breakpoints, isn’t in Goldratt.

Conclusion

Our society widely uses weighted factor models or views them as an ideal that’s hard to perfectly reach but should be strived for. This is done both explicitly with numbers and analysis, as well as intuitively in how people think about issues. A lot of people’s thinking is roughly equivalent to pro/con lists (considering good and bad factors, and how strong they are), which is equivalent to the weighted factor model, which I also refer to as using score systems.

That approach is fundamentally flawed. Instead, we need to find ways to focus on key issues. Changes to a few factors are much more important than changes to other factors. We should look for big wins and focus on the important stuff, not worry about every detail. This isn’t a practical compromise; it’s actually the theoretical ideal.