Noise: A Flaw in Human Judgment

Probably not for everyone — a bit too wordy even for me — but important reading regardless for those interested in justice and fairness. Noise hurts us disproportionately, and thus hurts us all. We all need to develop awareness, although not necessarily four hundred pages' worth. This is more a book for policy makers and shapers than for us in the trenches.

I liked the breakdown of noise into categories: level noise (two different immigration judges, one approves 88% of cases, one approves 5%); pattern noise (close to, but not quite, bias); and occasion noise (judges issuing sterner sentences before lunch than after, or if the weather is crappy). I liked the attempts to differentiate noise from bias. I liked the suggestions for evaluating noise and for minimizing it. (I did not care for their using a smooth Gaussian in some examples, but can grudgingly understand why they did so).

“Organizations all over the world see bias as a villain. They are right. They do not see Noise that way. They should. In many areas, the current level of Noise is far too high. It is imposing high costs and producing terrible unfairness. What we have catalogued here is the tip of the iceberg. Laws should do much more to reduce those costs and combat that unfairness.”

Cognitive psychology is a major field of interest to me. If you don't know Daniel Kahneman, he is an absolute titan in the psychology field, after having written Thinking Fast and Slow, a book that has been referenced across and influenced many different fields. This was his follow up book, along with two co-authors: Oliver Sibony and Cass Sunstein.

This book examines the phenomenon of Noise and how it affects our judgment. Sources of noise are essentially anything that can impact a person's judgment- lack of sleep, stomach pain, a breakup, or miniscule things you may not even be aware of like being nutrient deficiency. Noise can even be the simple fact that we may treat people who subtly remind us of a sibling in a manner that's different than we would otherwise. Noise impacts the way we make decisions, but it also causes us to be inconsistent in the way we make decisions. The book focuses a fair amount on judges, as they are commonly studied in cases like this, and judges will frequently hand out different sentences across time that are inconsistent with themselves and each other.

Noise also delves into the wisdom of crowds effect, where the aggregate judgment of a group is most likely to be nearly correct, but that individual judgments are usually more wrong. There's lots of little tidbits like this, and the science is far from settled.

The goal we should strive for in society is a reduction of noise. Police, lawyers, politicians, judges, doctors, and many more are all making judgment calls that are rife with noise and ideally, we should be finding ways to limit this.

This was hard to rate as a lot of the info was not new to me, but was presented in new ways. The book is not dense and does a really good job for people new to the subject!

8/10

Wherever there is judgement, there must be noise.

Judgments are human measurements. We predict if a tumor is benign or not, we decide how long the appropriate prison sentence for a certain crime should be, we choose who the best hire for an open position is. And we want these processes to be consistent and fair. We want the sense of security of receiving the same prognosis from different doctors, we want a crime to receive a similar sentence even if judged by different judges, we want the confirmation that the best candidate always receives the job.

But, obviously, this is not the case. Human judgements are clouded by biases and noise. Biases are easier to pick out, easier to attack, easier to take as the scapegoat. But Kahneman & Co show that noise is equally to blame for large variabilities in human judgement.


System noise ... noise observed in organizations that employ interchangeable professionals to make decisions
Level noise ... when some judges are harsher and others are milder, due to the ambiguity of the judgement scale.
Pattern noise ... variability due to each judge's individual opinions and experiences.
Occasion noise ... if the same person judges different based on time of day, mood, or external influence.

Important to note that this book deals with noise in systems where there should be no noise. There are many other domains of judgements where diversity in opinions is welcome.

Interesting topic, going too much into detail for me though. I would have preferred for it to be a bit more concise. But it's probably eye-opening for someone who deals with reproducible human judgement every day.

This book is excellent. Much like Kahneman's* definitive book on bias, Thinking Fast and Slow, Noise provides an excellent, fairly comprehensive treatment of another source of error in human judgement, which the authors define as noise. Noise is, as a term in this book, used to describe inconsistency in human judgment, as opposed to bias, which is a systematic departure from “correct” results. There is some overlap in terms here, as, for example, hungry judges systematically make harsher decisions, which is referred to as bias in Thinking Fast and Slow, but because we're looking at error across the entire range of outcomes in a different way here, is called occasion noise. I do not believe this detracts from what the book brings to the table, but it's worth noting that in this book, bias is used to refer to the difference between the average outcome and the “correct” outcome, or other errors across the range of outcomes such as minorities being treated differently in cases where there isn't a “correct” outcome to measure.

What this book does not do is claim that all noise should be completely eliminated. Eliminating noise has costs. However, a wide disparity of outcomes in similar cases can be extremely unfair. Should two people with similar histories and mitigating/aggravating factors have several years of difference in sentencing for the same crime? Should the luck of who evaluates your insurance policy or what mood they're in when they do make hundreds of dollars in difference to your premiums or policy payouts? Certain types of judgements are judgements where inconsistency is inherently unfair.

Noise looks at these judgements. It looks at hiring decisions where projection is inherently difficult and outcomes are hard to evaluate. It looks at expert judgement in fields like forensics where experts are asked to make evaluations of objective facts and whether there is noise in those outcomes as well.


I feel like I should be writing many more paragraphs about this book, but for now I'll leave it here. This book is held to a high standard of rigor and is evidence backed throughout, again in line with Kahneman's Thinking, Fast and Slow. The two books combine to provide an extreme amount of information on how to improve your judgement as an individual or an organization. I highly recommend this book and it will be very close to the top of my list of “must read” books on intelligence and the human brain.



*There are three authors here and I don't wish to downplay any role of Cass Sunstein or Oliver Sibony, which I am obviously not in position to evaluate. This book has led me to investigate their other work and likely will result in me reading at least one from each. However, Kahneman is the most well known partly because he's one of the most influential figures in the field of human judgement, widely cited by psychologists and behavioral economists, and Thinking Fast and Slow is in my opinion is probably the best book on the brain everyone should read.