Professor James Liebman and colleagues at Columbia University recently released A Broken System II: Why There Is So Much Error in Capital Cases, the follow-up to their groundbreaking, A Broken System: Error Rates in Capital Cases 1973 * 1995.
Answers to frequently asked questions about the study are provided below.
Q: How is this study different from the previous study?
A: The previous study, A Broken System: Error Rates in Capital Cases 1973 * 1995 determined the numbers and types of errors made in death penalty trials while this study, A Broken System Part II: Why There Is
So Much Error in Capital Cases, and What Can Be Done About It identifies those factors that lead to the high error rates.
The first study, A Broken System I, simply counted the number of death penalty appeals and the number of reversals, and divided to determine
the error rate. The study also looked at the reason the court gave for those reversals, and added them up.
This second study, A Broken System II has more to say about what kind of mistakes get made and explores why those mistakes get made * what
conditions exist to create these errors? The study uses a number of statistical tests to identify those factors that lead to high error
rates in capital cases. It also explores some potential solutions to those problems.
Q: What was the goal of this study?
A: The goal of the study was to identify factors that lead to the high error rate in death penalty trials and appeals.
The study found 76% of the state and federal reversals at the two appeal stages where data are available were because of egregiously incompetent
defense lawyers, police or prosecutor misconduct, or misinformed and biased judges or juries. 82% of those retried after their death verdicts were reversed at the second of three appeals (or, during the state post-conviction appeal) were given a sentence less than death, and 9% of those retried * nearly 1 in 10 * were found innocent.
This study tried to find out why. What is it about some states or counties that lead them to have higher error rates than others? Why do
some areas have better lawyers, judges and juries than others? This study attempts to answer those questions, and to identify what * if
anything * can be done to solve those problems.
Q: What was the main finding of this study?
A: The more often states and counties sentence people to death, the
more often they get it wrong.
This study uncovered a number of conditions related to error in capital
cases, including politics, race, crime control and the courts. But
running through all the data was a simple finding * the more a state or
county sentences people to death, the more often they make mistakes.
This isn’t just a matter of numbers, this is about percentages.
Everything else being equal, when death sentencing increases from the
lowest to the highest rate in the study, the reversal rate increases
six-fold, to about 80%. The more often states and counties use the
death penalty for every, say 10 or 100 homicides, the more likely it is
that any death verdict they impose will later be found to be seriously
flawed, and the more likely it is that the defendant who was found
guilty and sentenced to die will turn out to be not guilty.
Q: What are the factors that lead to these errors?
A: In addition to aggressive death sentencing, there are four key
factors which lead to errors: homicide risk to whites and blacks; the
size of the black population; the rate at which police catch and punish
criminals; and politically motivated judges. There are three additional
important findings: heavy use of the death penalty not only leads to
errors but drives up costs and slows down all courts; shoddy work at the
initial trial leads to mistakes; and the problem isn’t getting better
over time.
Everything else being equal, when the risk of a white person getting
murdered is high relative to the risk of an African-American getting
murdered, twice as many appeals are reversed than where that risk is
low. The study’s best explanation of why this occurs is that when
whites and other influential citizens feel threatened by homicide, they
put pressure on officials to punish to punish as many criminals as
severely as possible * with the result that mistakes are made, and a lot
of people are initially sentenced to death who are later found to have
committed a lesser crime, or no crime at all.
The more African-Americans there are in a state, the more likely it is
that serious mistakes will be made in death penalty trials. This could
be because of fears of crime driven by racial stereotypes and economic
factors.
It is disturbing that race plays a role in the outcome of death penalty
cases, whatever the reasons.
The lower the rate at which states catch, convict and punish serious
criminals, the higher the error rates. Everything else being equal,
states that put one offender in prison for every 100 serious crimes (100
FBI Index Crimes) committed in the state have about 75% of their death
sentences reversed. But states that put four offenders in prison for
every 100 serious crimes only have an error rate of about 36%, and
those states that catch and punish the most criminals only make mistakes
in death penalty trials about 13% of the time. States that do a bad job
of catching and convicting criminals also make a lot of mistakes at
death penalty trials. This may be because if a state doesn’t catch many
criminals, officials are under pressure to prove they are tough on crime
by trying to punish those they do catch as severely as possible. When
this leads to additional death sentences, many of which are flawed, the
overall result is that no more criminals are caught, crime remains high,
and the person they attempted to sentence to death almost always winds
up with a lesser more appropriate sentence, or is later found to be
innocent.
The more often, and more directly, state trial judges are subject to
popular election, and the more partisan those elections are, the higher
the rate of serious error. Judges who face elections keep one eye on
justice and one on the polls * and when push comes to shove, the polls
often win.
There are several other key findings.
Heavy use of the death penalty prevents the entire criminal justice
system from doing its job. Death penalty trials are long and complex,
and as already indicated, having many death penalty trials leads to many
errors and more trials. These trials upon trials drive up costs, divert
resources and prevent the courts from doing their job as effectively and
efficiently. In states with more than 20 verdicts under review, the
appellate process often comes almost entirely to a halt.
Bad trials lead to mistakes. This seems obvious * without the resources
to do it right the first time, courts must try and get it right the
2nd or 3rd time. In the meantime, all trials are slowed, victims
are left in limbo, and tax dollars keep getting spent.
The problem isn’t getting better over time. After considering the
influence of all the other factors discussed above, direct appeal
reversal rates increased 9% a year during the study period. This means
that reforms based on the factors known to lead to capital error may not
work because other factors that are not as easy to pinpoint and fix may
continue to push capital error rates substantially higher.
Q: How did the researchers select the appeals they examined?
A: The researchers did not select appeals * they examined the outcome of
every death penalty appeal from 1973 * 1995, more than 5,400 state and
federal appeals.
Rather than attempt to sample death penalty appeals, which would
invariably lead to bias or error, the authors examined every single
death penalty appeal over a 23-year period and reported the decision
reached by the state or federal court that reviewed the death penalty
for serious error.
Q: Why did the study only look at completed trials and appeals?
A: By looking at completed trials and appeals, the study eliminated
guess-work and bias.
By looking only at those trials and appeals that have been completed,
the study avoids speculation, guess work and bias. Only by examining a
job after it’s done can one determine how well or badly the job was done
* which is why batting titles are only given after the baseball season
is over. A lot of players hit .500 on opening day, only to end the
season with a far lower average. By looking only at those cases that
are closed, the study looked only at batters who had completed the
season. If the study had guessed what the appeals under review will
look like, they would be declaring a lucky player a star, and miss the
truly great ball player. Court decisions are all subject to review and
reconsideration and only by looking at completed appeals can the outcome
of the painstaking process of court review be known.
Q: How did the researchers select the variables to test in their analysis?
A: A careful survey of legal, criminological and sociological research
identified conditions that might conceivably be associated with capital
error rates. The authors went through a careful process of testing each
condition separately and in conjunction with others to see if they were
in fact related to capital error rates.
Those who study crime, punishment and public safety have explored a host
of factors that may lead to violence and violence prevention. Rather
than reinvent the wheel, or try to independently determine what may or
may not impact trials, the authors used the variables experts in the
field use. The authors then tested each condition separately, and also
in conjunction with other factors, through a statistical technique
called regression analysis, to determine which variables had an impact
on error rates, and how strong or weak that impact was.
Q: What is a serious and reversible error?
A: A mistake that was prejudicial, was brought up in a timely manner and
was discovered.
The legal reasons that permit death verdicts to be overturned set a high
burden of proof for prisoners to meet, and assure that reversals occur
because the death sentence is demonstrably unreliable. First, it is not
enough that even a blatant legal error has occurred. In addition, the
error must be prejudicial, which in most cases means the defendant must
show that the error likely affected the outcome of the trial, or that
similar errors often affect outcomes of death penalty trials. Second,
the error must be brought up in a timely manner and in accordance with
rules of the court * an error, regardless of the severity of the
mistake, was not counted if it wasn’t brought up under the rules of the
court or if it wasn’t actually overturned by the court. Finally, of
course, the error must be discovered, not rumored or alluded to. By
sticking to such a strict definition of serious and reversible the
authors excluded trivial and technical mistakes.
In addition, the authors included four case studies in the report of
people who were innocent but were approved for execution at all 3
appeals. In each of these cases, the courts found errors that in fact
had led innocent people to be convicted, but the courts nonetheless let
the conviction and death sentence stand because the defendants were
unable to prove themselves innocent, and as a result the courts were not
permitted to overturn the verdict. It was left to reporters, students,
film makers and others to prove the innocence of those four men on death
row.
Q: How have recent changes to rules about appeals effected the quality
of trials? In other words, is the problem getting better or worse?
A: The problem is not getting better as a result of changes to the law.
The more people we try to execute, and the faster we try to execute
them, the more mistakes we make. These mistakes lead to longer delays,
and more innocent people on death row * all at taxpayer expense.
In 1996, Congress passed laws designed to decrease the number of death
penalty appeals and speed the process from conviction to execution.
Texas and other states passed similar laws at about the same time,
further cutting back on review in the state courts. In addition, states
passed laws both adding crimes for which someone could be sentenced to
death and elements of crimes that could lead to a death sentence. The
result was not fewer appeals, more efficient trials, or an increase in
the percent of people being executed (executions have decreased over the
past two years but the numbers of people on death row continues to grow).
The result of these changes was actually to make the system worse.
Courts are now trying to do more with less, with predictable results.
Rather than reserve the death penalty for the worst of the worst, and
spend time making sure trials are conducted fairly and completely,
courts rush through trials, handing down death sentences when they are
not clearly called for, as a result of overlong lists of overbroad
aggravating circumstances, hoping mistakes get caught further down the
line. The appeals courts are therefore faced with a barrage of
questionable death penalty convictions, through which they must sort.
The greater the backlog, the less careful the review * faced with the
same problem as the trial courts, the appeals courts rush the job and
get it wrong. This, of course, leads to more appeals, which puts even
more cases before the courts, which they rush through and more often
than not get wrong. Recent changes in the rules that control appeals
make matters worse by requiring courts to work even faster than before
and by adding obstacles that keep them from overturning verdicts that
are flawed and unreliable.
Q: Can you explain the methodology?
A: The authors used 19 different statistical tools, called regression
analysis, to figure out why so many mistakes are made in death penalty
trials.
Regression analysis is a statistical method for explaining the
relationship between something you want to explain and things that might
explain it. It allows you to look at the effect of individual factors
while holding other factors constant. For example, a lot happens in a
baseball game * running, hitting, pitching, defense, home field
advantage, and so on. Which of those things has the most impact? All
other factors held constant, how much difference does good pitching make?
The best way to answer these kinds of questions is to take all of the
variables, all of the things that happen, and put them into a computer
program that performs a series of calculations to determine which things
can be said with confidence to matter, and how much they matter. There
are a number of these models or programs, each with its own strengths
and weaknesses. In this study, the authors use what they thought was
the best model and then used 18 additional different models to test the
data (for a total of 19 different tests) looking at the question of why
there are so many mistakes from a number of different angles. This way
the authors get the best and most complete view of the problem.
Q: What types of finding and conclusions does the study reach?
A: The study identifies a number of facts about how the death penalty
system operates * for example, what conditions usually are present in
states, counties and years in which above average numbers of errors are
committed in capital cases. The study also offers some explanations for
why these conditions might lead to high rates of error in capital cases.
The study’s findings reveal the number of reversals of capital verdicts
in different times and the conditions that tend to be present in places
where, and at times when, rates of capital error are high. Those are
the facts of the matter. Explanations provide reasons why reversals may
occur, and why particular conditions may lead errors and reversals to
occur. For example, the study find states where judges are elected tend
to have more of their death penalties reversed because of serious
errors. That’s a fact. Other facts are that when the proportion of
African-Americans in a state goes up, or when the number of times a
state imposes a death sentence per homicide goes up, error rates also go
up.
The best explanation for this is that elected judges face pressures to
impose death sentences to calm crime fears, even if that punishment is
not legally appropriate or if corners have to be cut during trial in
order to obtain a death sentence. Judges, that is, feel the need to
keep one eye on justice and one eye on politics, and may find it to be
better politically to sentence a lot of people to death in the short run
and have a lot of those sentences reversed many years down the road than
it is to use the punishment more sparingly and have fewer mistakes
discovered later on. Whether this explanation or another one applies,
the main point is that politics interferes with the reliability of death
verdicts.
In listing policy options, the authors were careful to rely on the facts
of the matter as reasons for needing reform and for identifying the most
beneficial reforms.
Q: Don’t courts that are biased against the death penalty make it look
like there are more errors than there really are?
A: The opposite very often is true * judges are under political pressure
to approve death penalties that are flawed, and do so. And the
standards they apply are so strict that they sometimes are forced to
approve unreliable verdicts, including where the defendant is innocent
of the crime.
One of the findings of the study is that judges who feel political
pressure, primarily through elections, uphold questionable death
sentences that higher courts have to overturn because they are unreliable.
While some accuse some of courts having an anti-death penalty bias
others point to courts that have a pro-death penalty bias. An important
reason for using statistical analysis is that it looks for patterns that
are bigger and more powerful than the actions of particular judges. In
this study, the authors reviewed over 5,000 decisions made by several
thousand state and federal judges in three sets of appeals in 34 states
and across 23 years. When high error rates are found by all these
judges in all these appeals in all these states and years, it is
impossible to blame the problem on particular individual judges. It is
the system as a whole that is generating the problem, and it is the
public, taxpayers and victims across the country and across decades who
are suffering the consequences.
The authors checked to see if the judges who issue most of the reversals
are likely to have views about the death penalty that are out of line
with the views of the public and are biased against capital punishment.
They found the opposite was the case: 90% of the reversals were by
judges elected by the public. In more than half of the remaining cases
that were reversed, a majority of the non-elected judges who found
serious error and overturned the verdicts were appointed by Republican
Presidents, further arguing against anti-death penalty bias.
As the four cases in the report reveal, judges must find that strict
tests are satisfied before they may reverse. In close cases, they
usually are not permitted to reverse, and reversal usually requires
defendants to prove not only that errors occurred in their cases but
also that those errors probably caused the jury to reach the wrong
outcome. Because that is so hard to prove, courts often end up
approving unreliable death verdicts * including verdicts that sent
innocent people to death row. Only the intervention of those not
associated with the court system saved the lives of these innocent people.
Q: Aren’t most of the people who have their cases overturned guilty?
A: Not of a capital crime.
About 1 in 10 of those who have their cases overturned at the 2nd
stage of review (which was the focus of this part of the study) and are
sent back for retrial are found innocent. Imagine if 1 in 10
medications killed the patient, or one of every ten school buses
exploded * we simply would not accept that level of avoidable risk.
More than 8 of 10 reversals following the 2nd stage of review
result in a sentence less than death * which means that courts initially
sentenced someone to die where that penalty was not legally
appropriate. A premise of our system is that the punishment should fit
the crime. Our most severe punishment must be reserved for the worst of
the worst. When that is the case, there are few errors and few
reversals. The problem comes when courts try to violate the basic
premise that crime and punishment must be aligned.
Q: Based on this study, should we abolish the death penalty?
A: That is a question voters and their elected officials have to
decide. If they decide to keep the death penalty, they should take
steps to fix the mistakes.
If a mechanic tells us our car is broken, we choose to either fix the
car or get rid of it. One of our public policies, the death penalty, is
broken. Voters and their elected officials must decide to either fix it
or abandon it. If the decision is to fix the system, the study
identifies ten steps that can be taken to decrease the amount of errors,
increase the fairness of the punishment, and decrease the chances we
will execute an innocent person.
Q: Aren’t a lot of these reversals just because the rules of the game
change?
A: No. Most reversals are for violating rules that date back hundreds
of years.
The study analyzed the reasons for about 500 of the reversals * all
those occurring at the second and third stages of review. Far and away
the most common reason a court reversed a death penalty at those 2
stages was because of incompetent counsel. The right to adequate
counsel has been in the US Constitution since 1792, and the obligation
to provide a qualified and competent attorney has been recognized at
least since the 1930s. The second most common reason for reversal is
prosecutors suppressing evidence of innocence or mitigation. The rules
violated in these cases also date back several decades. The right to a
properly informed jury * the right violated third most often * is as old
as the nation.
Q: The study says one solution is reserving the death penalty for the
worst of the worst. What is the worst of the worst?
A: The worst of the worst are those crimes with a high concentration of
aggravating factors.
There is no such thing as a good murder. But the laws of states with
the death penalty list factors that make murders more aggravated
because they make the crime more serious or the offender more deserving
of punishment. The study finds that death verdicts imposed where not
just one, but several, of these factors are present are substantially
less likely to be overturned due to serious error. Everything else
equal, the addition of each additional aggravating circumstance made
reversal 15% less likely at the third stage of review.
Q: Doesn’t the high reversal rate really prove the system is working
because it catches mistakes?
A: No. Evidence that a lot of mistakes are made is evidence that the
system is broken.
It may be tempting to think that a high error rate demonstrates that the
system is catching all of its mistakes, but that itself would be a
mistake. It’s like saying, the fact that we get it wrong most of the
time proves we get it right most of the time * clearly a nonsensical
proposition. That we catch so many errors is evidence that we make a
lot of mistakes. It also implies that there are probably a lot of
errors we don’t catch.
Suggesting that the number if serious errors the courts find is proof
that the system works is like suggesting that the more time your car
spends at the garage the better it works * that the higher the bills you
have to pay to fix your car, the better it runs. Clearly such claims
are absurd.
Q: What’s the bottom line?
A If a state is going to have the death penalty, it should be saved for
the worst of the worst, time and effort should be put into the first
trial, and those making life and death decisions shouldn’t be subject to
the whims of politics. In short, if you’re going to use the death
penalty, do it sparingly and do it right.
(source: Justice Project)
