Authors: Warren Ragoonanan and Katherine Lorriman
During World War II, the U.S Air Force had a problem. They were losing too many planes in the theatre of war. Sometimes they were shot down by Nazis, other times it was friendly fire. They knew that to curb their losses, they needed to add armour to the planes, but how much? Too much, and they would be too heavy to fly. Too little, and the armour would be useless.
For help, the Air Force brass turned to the Statistical Research Group (SRG), a classified program that brought together top U.S mathematicians and statisticians. Chief among the SRG team was Columbia University statistics professor Abraham Wald. Like any good statistician, to work his mathematical magic Professor Wald needed data. Th
e military provided him with damage reports on the planes that returned from Europe. They broke the figures down by the number of bullet holes per square foot over the: engine, fuselage, fuel system, and the rest of the plane. The data showed the highest number of bullet holes in the fuselage and the lowest in the engine.
The Air Force looked at the data and thought the answer was simple – put the armour over the fuselage. Professor Wald looked at the data and came to the opposite conclusion – put the armour over the engine. Professor Wald saw something that the Air Force officers did not. The data provided only covered planes that survived the attacks; what was missing was data on the planes that the enemy shot down. There were no reports on those, meaning that the data set from the military was skewed towards the planes that survived. In a representative sample, you would find bullet holes evenly distributed all over the plane, including the engine. If planes were coming back with fewer bullet holes in the engine, then, Professor Wald reasoned, the engine must be a major vulnerability. The fuselage, fuel system and other parts could take damage without taking the whole plane down. The engine could not (for the full story on Professor Wald check out the book How Not To Be Wrong by Jordan Ellenber).
The Air Force leadership assumed that they were working with a full set of data, and that nothing was missing. They were exhibiting what we know today as survivorship bias. This is a cognitive bias where a human being focuses on information that has survived a selection process and overlooks what was weeded out. Indeed, survivorship bias is one of many types of cognitive biases that act on our brains. These biases leave us with blind spots. The legal profession is no exception. As lawyers, we are sensitive to bias and any perception of bias. Hence the need for conflict-of-interest rules and confidentiality built into the system of legal ethics. However, lawyers rarely talk about cognitive bias even though it impacts what we do every day.
Look at the task of contract drafting. In the profession, we often hear the old maxim that, when it comes to drafting, we should not “mindlessly follow precedents”. Except that, thanks to our cognitive biases, it is easy to end up doing just that without even realizing it. When drafting a contract, lawyers rarely start from nothing – working from a precedent is the norm. And when they do, survivorship bias can creep into how lawyers choose that precedent. Contract drafting is a broad area. It can encompass consulting service agreements, employment contracts, real estate agreements of purchase and sale, share/asset purchase agreements, and other types. Moreover, there are different contracts tailored to different industries, and rarely is there a standard form (if there is a standard form, clients often draft the contract themselves and bypass lawyers completely). So where do lawyers find their precedents? They can reach out to colleagues within or outside their firms. They can go to a precedent encyclopedia like the O’Brien’s Encyclopedia of Forms. They can also do a random Google search and hope that yields some results. These are common precedent-hunting approaches, but not one of them will guarantee a representative sample connected to the client’s needs. It is very likely that the lawyer’s “data set” of precedents will be skewed in one form or another. Lawyers will be looking at the catalogue of precedents that they happen to have and selecting one as the starting point. This is survivorship bias in action.
You may be thinking to yourself, ‘Surely lawyers read the precedent before they start drafting right? How much of an impact can survivorship bias really have?’ It is obviously necessary to read the precedent and if it turns out to be completely irrelevant to the mandate, you can set it aside and use something different. But what happens where the precedent is 50% relevant, or maybe 60% to 70% relevant, and you cannot find another one quite as good? The usual practice is to work with the document you have and try to revise it as best you can to suit the client’s needs; after all, that is usually a better option than burning more time trying to find the “perfect” precedent. It is at this point that another cognitive bias can show up, known as anchoring bias. Anchoring bias is a tendency for us to let a reference point subconsciously influence our decisions. The reference point is usually the first thing you see. If you are negotiating a business deal, that anchor may be the first price you see, which in turn influences how you negotiate and what price you actually pay. When it comes to you as a lawyer drafting a contract, the reference point is the precedent you are using. The same one in which only 50% to 70% of the clauses are relevant to your client’s deal. As the lawyer, utilizing professional judgement, you must make decisions on how to rework or remove those irrelevant clauses. Those decisions are going to be anchored to the language that is already in the document.
Thanks to both survivorship bias and anchoring bias, it can be difficult for us to spot issues that make it necessary to draft additional clauses that the precedent in front of us excludes. These biases also make it difficult to see which contract terms are only tangentially relevant and should be deleted. Hence you have the phenomenon where the drafter keeps in clauses that they do not completely understand just in case taking them out could hurt the client in an unknown future scenario. It is easy to end up with a contract that is both overly complicated, and only passingly relevant to the business transaction the client is trying to complete.
Here is our challenge:
How do business lawyers overcome cognitive biases when drafting and negotiating contracts?
This is a big question. In the productive thinking approach to problem solving (outlined in the book Think Better: An Innovator’s Guide to Productive Thinking), we call this a Catalytic Question – a question specifically crafted to help us think. As we draft, let’s keep this Catalytic Question in mind. There may be one answer, or there may be several. Regardless, keeping our Catalytic Question in mind as we draft is the first step to stopping contract precedents from thinking for us.