How the World Really Works

Institutional stupidity is the bane of modern human existence. As a customer, a citizen, a soldier, or an employee we all deal with organizations that behave in such a mindbogglingly idiotic fashion that it’s a wonder they survive. And yet they do. Why is that? Why is it that people in groups seem to be so much dumber than individuals? What happens to the wisdom of the crowd when you give them all cubicles?

As it turns out, there’s an answer. It’s not that people are dumb. It’s that they are doing work that they don’t really care about. 

If you’re Robinson Crusoe and you decide you want something, you’re only options are to try and make it yourself or decide that it’s not worth the effort. If you make the decision to make the effort, then you’re going to work as efficiently and as effectively as possible because you don’t want to waste your own time and effort. There’s a direct connection between your input and your output, and it makes incentives clear and direct.

Once you start working in an organization, however, you run into the principal-agent problem (also known as the agency dilemma). In the simplest case of the principal agent problem, an owner hires employees to help run his business. The owner gets money when the business does well, so his incentives are aligned. But the employees get money when the business owner thinks they are contributing to the company, and the owner can’t observe that perfectly. A principal’s incentives are based on success or failure, but an agent’s incentives are based on credit or blame.

In a tiny example where a business owner and their employees work side-by-side, this isn’t a big problem, but imagine what happens when you have thousands of employees working in layers on top of layers of management and no clear owner at all.

Insanity Reigns

Let’s imagine how a company might set security policy. It starts with with an executive (say, the Chief Risk Officer) who probably knows that protecting company data is very hard but protecting his job is very easy. The CRO could try to design a custom security solution, but it won’t guarantee the data gets protected and if something goes wrong they will be judged by non-experts who won’t be able to appreciate the work they put into designing a sensible policy. So instead, the CRO opts to rolls out something that conforms to industry standards. It’s a fraction of the cost, but practically ironclad in terms of blame-avoidance.

So the executive goes to his underlings and says “Design and implement a security policy that meets industry standards”. The middle managers aren’t evaluated based on the policy, however. They are evaluated based on their progress reports, presentations, and evaluations. They have their own incentives to take credit or avoid blame, and they know perfectly well–based on the CRo’s request–that authentically creating a security procedure is not really on the agenda.

So they create a policy that crosses t’s and dots i’s and hand it off to their underlings to implement. Now you’ve got a bunch of folks who’d rather be playing World of Warcraft being told to implement a policy that anyone can plainly see is full of holes. It’s quite obvious at this point that no one with a brain designed this policy (everyone involved was using half a brain, at most, because it was deemed not worth the effort), but if they don’t execute on this order, their boss won’t have his ass covered, and if his ass isn’t covered he’ll have their’s.So, they do what their told.

The result is, depending on your perspective, either a monument to human ingenuity or to human incompetence. As a security policy it’s an inconsistent, hard-to-maintain, and ineffective quagmire that does more harm than good by getting in the way of honest employees trying to do their job. But as a system for blame dispersal and avoidance, it’s top-rate.

The Good Guys

So what happens when someone comes to work for this company with a whole suitcase full of optimism and principles? Well, they run face-first into a brick wall. Starting at or near the bottom, they have very little influence. Constructive criticism is largely lose-lose. If you’re wrong: you look incompetent. If you’re right: you’re actually attacking the blame-avoidance, credit-accumulating structure someone who has been there longer than you has put into place. That is unlikely to make you friends.

As a result, no matter how noble or idealistic you might be when you enter a system like this, it immediately start to wear away your integrity like acid attacking your stomach lining, leaving you with nothing but ulcers and regrets.

There are basically two alternatives at this point. Most people disengage. The easiest way out of an ethical dilemma is to ignore it and hope it goes away, and so they do. Meanwhile, because they prefer having a job, they pick a strategy of minimizing effort while keeping a basic level of job security. They become blame-avoiders, and the cubicles are where you will find them.

A small minority initially refuse to quit playing the game. Instead, they try to game the system. They are the credit-seekers, and the bottom rungs of management are where you will find them. Superficially these guys look like the movers and the shakers in the company. You might even think that they could save the institution from apathetic negligence. Unfortunately, while they are definitely expending much more effort than the blame-avoiders, they aren’t necessarily contributing meaningfully to the institution. Their motivation is to pick highly visible projects that will easily win the praise of their superiors. But if you only do what your superiors can easily and universally be convinced is a great idea, then by definition you’re not really an agent of change. You’re largely just a hyperactive version of the problem.

The one saving grace is that since the high-credit opportunities are pretty slim pickings down at the bottom, the lowest-level credit-seekers are forced to try out ideas that aren’t universally accepted. So this is where they good ideas come from, but only a tiny trickle of really incremental advancements.

There’s a second problem for the credit-seekers, and that is simply that ladder-climbing is exhausting. It’s literally exhausting because they have to be the first-in and last-out every day. They are the ones clamoring to impress higher-level management and, along the way, spending huge amounts of energy practicing office politics against the rival credit seekers. After all, there are fewer and fewer slots the higher up the food chain you want to go. But this external exhaustion pales in comparison to the internal fatigue that comes from living a lie every day.

A lot of the credit seekers are dynamic and intelligent, and every day they have to live with the painfully obvious gap between what would be good for the company and what they actually choose to do. The common rationalization is that if they compromise on what’s true now, they’ll acquire more power to have influence to do even better things in the future, but sooner or later the credit-seekers run out of appetite for the meat-grinder of the soul and plateau somewhere in the upper-tiers of the organization, transitioning into high-level blame-avoiders with much better skills and more powerful networks to forestall any kind of genuine change that would threaten to pull them back into the maelstrom of work. These are basically the Sarlaccs of the office, and they will eat you if you threaten their hard-won status and its resulting equilibrium.

It’s a Wonder They Survive

So, if companies get dumber and dumber as they get larger and larger, how do they survive? The first part of the answer is that there are benefits from size that outweigh a lot of the detriments: returns to scale and higher barriers to entry are two examples. For example, no matter how smart an automotive company with 12 employees might be, they are simply unable to compete with Ford.

The second part of the answer is simply this: sometimes they don’t survive. Institutional stupidity is the chief cross-industry advantage of smaller companies. The smaller the company, the fewer layers of management there are to pervert incentives and the closer everyone is to the unforgiving forces of the market. It’s an atmosphere that fosters better-aligned incentives, and that in turn keeps the companies slightly more nimble and innovative, providing a constant threat to the big players.

One way to think of this is as a predator-prey model, where the established institutions are the prey and the startups are the predators. The predators therefore add some pressure to the evolutionary forces of who wins and who loses in the marketplace. The trouble is that natural selection isn’t an optimization routine. Darwin observed this when he noticed that honeybees die after stinging something. Obviously it would be better if honeybees evolved to be able to sting many times before dying, but Darwin observed that this won’t happen until they have to. In natural selection you don’t have to be faster than the bear, you only have to be faster than the other guy. Once you get that fast, there’s no incentive to go even faster.

And that’s why smarter companies are a little less stupid then big companies, but both of them are pretty stupid. Not just companies either, mind you, but governments, schools, the military, or even a simple committee. Wherever you find a group of people trying to do work where it’s possible to avoid blame for things that are your fault and take credit for things that you didn’t do, you’re going to see this fundamental break down in ethics and effective intelligence.

Next week, I’ll have a slightly more serious look at the individual implications for this sad reality.