Professor James Liebman and colleagues at Columbia University recently released A Broken System II: Why There Is So Much Error in Capital Cases, the follow-up to their groundbreaking, A Broken System: Error Rates in Capital Cases 1973 * 1995.
Answers to frequently asked questions about the study are provided below.
Q: How is this study different from the previous study?
A: The previous study, A Broken System: Error Rates in Capital Cases 1973 * 1995 determined the numbers and types of errors made in death penalty trials while this study, A Broken System Part II: Why There Is So Much Error in Capital Cases, and What Can Be Done About It identifies those factors that lead to the high error rates.
The first study, A Broken System I, simply counted the number of death penalty appeals and the number of reversals, and divided to determine the error rate. The study also looked at the reason the court gave for those reversals, and added them up.
This second study, A Broken System II has more to say about what kind of mistakes get made and explores why those mistakes get made * what conditions exist to create these errors? The study uses a number of statistical tests to identify those factors that lead to high error rates in capital cases. It also explores some potential solutions to those problems.
Q: What was the goal of this study?
A: The goal of the study was to identify factors that lead to the high error rate in death penalty trials and appeals.
The study found 76% of the state and federal reversals at the two appeal stages where data are available were because of egregiously incompetent defense lawyers, police or prosecutor misconduct, or misinformed and biased judges or juries. 82% of those retried after their death verdicts were reversed at the second of three appeals (or, during the state post-conviction appeal) were given a sentence less than death, and 9% of those retried * nearly 1 in 10 * were found innocent.
This study tried to find out why. What is it about some states or counties that lead them to have higher error rates than others? Why do some areas have better lawyers, judges and juries than others? This study attempts to answer those questions, and to identify what * if anything * can be done to solve those problems.
Q: What was the main finding of this study?
A: The more often states and counties sentence people to death, the more often they get it wrong.
This study uncovered a number of conditions related to error in capital cases, including politics, race, crime control and the courts. But running through all the data was a simple finding * the more a state or county sentences people to death, the more often they make mistakes. This isn't just a matter of numbers, this is about percentages. Everything else being equal, when death sentencing increases from the lowest to the highest rate in the study, the reversal rate increases six-fold, to about 80%. The more often states and counties use the death penalty for every, say 10 or 100 homicides, the more likely it is that any death verdict they impose will later be found to be seriously flawed, and the more likely it is that the defendant who was found guilty and sentenced to die will turn out to be not guilty.
Q: What are the factors that lead to these errors?
A: In addition to aggressive death sentencing, there are four key factors which lead to errors: homicide risk to whites and blacks; the size of the black population; the rate at which police catch and punish criminals; and politically motivated judges. There are three additional important findings: heavy use of the death penalty not only leads to errors but drives up costs and slows down all courts; shoddy work at the initial trial leads to mistakes; and the problem isn't getting better over time.
Everything else being equal, when the risk of a white person getting murdered is high relative to the risk of an African-American getting murdered, twice as many appeals are reversed than where that risk is low. The study's best explanation of why this occurs is that when whites and other influential citizens feel threatened by homicide, they put pressure on officials to punish to punish as many criminals as severely as possible * with the result that mistakes are made, and a lot of people are initially sentenced to death who are later found to have committed a lesser crime, or no crime at all.
The more African-Americans there are in a state, the more likely it is that serious mistakes will be made in death penalty trials. This could be because of fears of crime driven by racial stereotypes and economic factors.
It is disturbing that race plays a role in the outcome of death penalty cases, whatever the reasons.
The lower the rate at which states catch, convict and punish serious criminals, the higher the error rates. Everything else being equal, states that put one offender in prison for every 100 serious crimes (100 FBI Index Crimes) committed in the state have about 75% of their death sentences reversed. But states that put four offenders in prison for every 100 serious crimes only have an error rate of about 36%, and those states that catch and punish the most criminals only make mistakes in death penalty trials about 13% of the time. States that do a bad job of catching and convicting criminals also make a lot of mistakes at death penalty trials. This may be because if a state doesn't catch many criminals, officials are under pressure to prove they are tough on crime by trying to punish those they do catch as severely as possible. When this leads to additional death sentences, many of which are flawed, the overall result is that no more criminals are caught, crime remains high, and the person they attempted to sentence to death almost always winds up with a lesser more appropriate sentence, or is later found to be innocent.
The more often, and more directly, state trial judges are subject to popular election, and the more partisan those elections are, the higher the rate of serious error. Judges who face elections keep one eye on justice and one on the polls * and when push comes to shove, the polls often win.
There are several other key findings.
Heavy use of the death penalty prevents the entire criminal justice system from doing its job. Death penalty trials are long and complex, and as already indicated, having many death penalty trials leads to many errors and more trials. These trials upon trials drive up costs, divert resources and prevent the courts from doing their job as effectively and efficiently. In states with more than 20 verdicts under review, the appellate process often comes almost entirely to a halt.
Bad trials lead to mistakes. This seems obvious * without the resources to do it right the first time, courts must try and get it right the 2nd or 3rd time. In the meantime, all trials are slowed, victims are left in limbo, and tax dollars keep getting spent.
The problem isn't getting better over time. After considering the influence of all the other factors discussed above, direct appeal reversal rates increased 9% a year during the study period. This means that reforms based on the factors known to lead to capital error may not work because other factors that are not as easy to pinpoint and fix may continue to push capital error rates substantially higher.
Q: How did the researchers select the appeals they examined?
A: The researchers did not select appeals * they examined the outcome of every death penalty appeal from 1973 * 1995, more than 5,400 state and federal appeals.
Rather than attempt to sample death penalty appeals, which would invariably lead to bias or error, the authors examined every single death penalty appeal over a 23-year period and reported the decision reached by the state or federal court that reviewed the death penalty for serious error.
Q: Why did the study only look at completed trials and appeals?
A: By looking at completed trials and appeals, the study eliminated guess-work and bias.
By looking only at those trials and appeals that have been completed, the study avoids speculation, guess work and bias. Only by examining a job after it's done can one determine how well or badly the job was done * which is why batting titles are only given after the baseball season is over. A lot of players hit .500 on opening day, only to end the season with a far lower average. By looking only at those cases that are closed, the study looked only at batters who had completed the season. If the study had guessed what the appeals under review will look like, they would be declaring a lucky player a star, and miss the truly great ball player. Court decisions are all subject to review and reconsideration and only by looking at completed appeals can the outcome of the painstaking process of court review be known.
Q: How did the researchers select the variables to test in their analysis?
A: A careful survey of legal, criminological and sociological research identified conditions that might conceivably be associated with capital error rates. The authors went through a careful process of testing each condition separately and in conjunction with others to see if they were in fact related to capital error rates.
Those who study crime, punishment and public safety have explored a host of factors that may lead to violence and violence prevention. Rather than reinvent the wheel, or try to independently determine what may or may not impact trials, the authors used the variables experts in the field use. The authors then tested each condition separately, and also in conjunction with other factors, through a statistical technique called regression analysis, to determine which variables had an impact on error rates, and how strong or weak that impact was.
Q: What is a serious and reversible error?
A: A mistake that was prejudicial, was brought up in a timely manner and was discovered.
The legal reasons that permit death verdicts to be overturned set a high burden of proof for prisoners to meet, and assure that reversals occur because the death sentence is demonstrably unreliable. First, it is not enough that even a blatant legal error has occurred. In addition, the error must be prejudicial, which in most cases means the defendant must show that the error likely affected the outcome of the trial, or that similar errors often affect outcomes of death penalty trials. Second, the error must be brought up in a timely manner and in accordance with rules of the court * an error, regardless of the severity of the mistake, was not counted if it wasn't brought up under the rules of the court or if it wasn't actually overturned by the court. Finally, of course, the error must be discovered, not rumored or alluded to. By sticking to such a strict definition of serious and reversible the authors excluded trivial and technical mistakes.
In addition, the authors included four case studies in the report of people who were innocent but were approved for execution at all 3 appeals. In each of these cases, the courts found errors that in fact had led innocent people to be convicted, but the courts nonetheless let the conviction and death sentence stand because the defendants were unable to prove themselves innocent, and as a result the courts were not permitted to overturn the verdict. It was left to reporters, students, film makers and others to prove the innocence of those four men on death row.
Q: How have recent changes to rules about appeals effected the quality of trials? In other words, is the problem getting better or worse?
A: The problem is not getting better as a result of changes to the law.
The more people we try to execute, and the faster we try to execute them, the more mistakes we make. These mistakes lead to longer delays, and more innocent people on death row * all at taxpayer expense.
In 1996, Congress passed laws designed to decrease the number of death penalty appeals and speed the process from conviction to execution. Texas and other states passed similar laws at about the same time, further cutting back on review in the state courts. In addition, states passed laws both adding crimes for which someone could be sentenced to death and elements of crimes that could lead to a death sentence. The result was not fewer appeals, more efficient trials, or an increase in the percent of people being executed (executions have decreased over the past two years but the numbers of people on death row continues to grow).
The result of these changes was actually to make the system worse. Courts are now trying to do more with less, with predictable results. Rather than reserve the death penalty for the worst of the worst, and spend time making sure trials are conducted fairly and completely, courts rush through trials, handing down death sentences when they are not clearly called for, as a result of overlong lists of overbroad aggravating circumstances, hoping mistakes get caught further down the line. The appeals courts are therefore faced with a barrage of questionable death penalty convictions, through which they must sort. The greater the backlog, the less careful the review * faced with the same problem as the trial courts, the appeals courts rush the job and get it wrong. This, of course, leads to more appeals, which puts even more cases before the courts, which they rush through and more often than not get wrong. Recent changes in the rules that control appeals make matters worse by requiring courts to work even faster than before and by adding obstacles that keep them from overturning verdicts that are flawed and unreliable.
Q: Can you explain the methodology?
A: The authors used 19 different statistical tools, called regression analysis, to figure out why so many mistakes are made in death penalty trials.
Regression analysis is a statistical method for explaining the relationship between something you want to explain and things that might explain it. It allows you to look at the effect of individual factors while holding other factors constant. For example, a lot happens in a baseball game * running, hitting, pitching, defense, home field advantage, and so on. Which of those things has the most impact? All other factors held constant, how much difference does good pitching make?
The best way to answer these kinds of questions is to take all of the variables, all of the things that happen, and put them into a computer program that performs a series of calculations to determine which things can be said with confidence to matter, and how much they matter. There are a number of these models or programs, each with its own strengths and weaknesses. In this study, the authors use what they thought was the best model and then used 18 additional different models to test the data (for a total of 19 different tests) looking at the question of why there are so many mistakes from a number of different angles. This way the authors get the best and most complete view of the problem.
Q: What types of finding and conclusions does the study reach?
A: The study identifies a number of facts about how the death penalty system operates * for example, what conditions usually are present in states, counties and years in which above average numbers of errors are committed in capital cases. The study also offers some explanations for why these conditions might lead to high rates of error in capital cases.
The study's findings reveal the number of reversals of capital verdicts in different times and the conditions that tend to be present in places where, and at times when, rates of capital error are high. Those are the facts of the matter. Explanations provide reasons why reversals may occur, and why particular conditions may lead errors and reversals to occur. For example, the study find states where judges are elected tend to have more of their death penalties reversed because of serious errors. That's a fact. Other facts are that when the proportion of African-Americans in a state goes up, or when the number of times a state imposes a death sentence per homicide goes up, error rates also go up.
The best explanation for this is that elected judges face pressures to impose death sentences to calm crime fears, even if that punishment is not legally appropriate or if corners have to be cut during trial in order to obtain a death sentence. Judges, that is, feel the need to keep one eye on justice and one eye on politics, and may find it to be better politically to sentence a lot of people to death in the short run and have a lot of those sentences reversed many years down the road than it is to use the punishment more sparingly and have fewer mistakes discovered later on. Whether this explanation or another one applies, the main point is that politics interferes with the reliability of death verdicts.
In listing policy options, the authors were careful to rely on the facts of the matter as reasons for needing reform and for identifying the most beneficial reforms.
Q: Don't courts that are biased against the death penalty make it look like there are more errors than there really are?
A: The opposite very often is true * judges are under political pressure to approve death penalties that are flawed, and do so. And the standards they apply are so strict that they sometimes are forced to approve unreliable verdicts, including where the defendant is innocent of the crime.
One of the findings of the study is that judges who feel political pressure, primarily through elections, uphold questionable death sentences that higher courts have to overturn because they are unreliable.
While some accuse some of courts having an anti-death penalty bias others point to courts that have a pro-death penalty bias. An important reason for using statistical analysis is that it looks for patterns that are bigger and more powerful than the actions of particular judges. In this study, the authors reviewed over 5,000 decisions made by several thousand state and federal judges in three sets of appeals in 34 states and across 23 years. When high error rates are found by all these judges in all these appeals in all these states and years, it is impossible to blame the problem on particular individual judges. It is the system as a whole that is generating the problem, and it is the public, taxpayers and victims across the country and across decades who are suffering the consequences.
The authors checked to see if the judges who issue most of the reversals are likely to have views about the death penalty that are out of line with the views of the public and are biased against capital punishment. They found the opposite was the case: 90% of the reversals were by judges elected by the public. In more than half of the remaining cases that were reversed, a majority of the non-elected judges who found serious error and overturned the verdicts were appointed by Republican Presidents, further arguing against anti-death penalty bias.
As the four cases in the report reveal, judges must find that strict tests are satisfied before they may reverse. In close cases, they usually are not permitted to reverse, and reversal usually requires defendants to prove not only that errors occurred in their cases but also that those errors probably caused the jury to reach the wrong outcome. Because that is so hard to prove, courts often end up approving unreliable death verdicts * including verdicts that sent innocent people to death row. Only the intervention of those not associated with the court system saved the lives of these innocent people.
Q: Aren't most of the people who have their cases overturned guilty?
A: Not of a capital crime.
About 1 in 10 of those who have their cases overturned at the 2nd stage of review (which was the focus of this part of the study) and are sent back for retrial are found innocent. Imagine if 1 in 10 medications killed the patient, or one of every ten school buses exploded * we simply would not accept that level of avoidable risk.
More than 8 of 10 reversals following the 2nd stage of review result in a sentence less than death * which means that courts initially sentenced someone to die where that penalty was not legally appropriate. A premise of our system is that the punishment should fit the crime. Our most severe punishment must be reserved for the worst of the worst. When that is the case, there are few errors and few reversals. The problem comes when courts try to violate the basic premise that crime and punishment must be aligned.
Q: Based on this study, should we abolish the death penalty?
A: That is a question voters and their elected officials have to decide. If they decide to keep the death penalty, they should take steps to fix the mistakes.
If a mechanic tells us our car is broken, we choose to either fix the car or get rid of it. One of our public policies, the death penalty, is broken. Voters and their elected officials must decide to either fix it or abandon it. If the decision is to fix the system, the study identifies ten steps that can be taken to decrease the amount of errors, increase the fairness of the punishment, and decrease the chances we will execute an innocent person.
Q: Aren't a lot of these reversals just because the rules of the game change?
A: No. Most reversals are for violating rules that date back hundreds of years.
The study analyzed the reasons for about 500 of the reversals * all those occurring at the second and third stages of review. Far and away the most common reason a court reversed a death penalty at those 2 stages was because of incompetent counsel. The right to adequate counsel has been in the US Constitution since 1792, and the obligation to provide a qualified and competent attorney has been recognized at least since the 1930s. The second most common reason for reversal is prosecutors suppressing evidence of innocence or mitigation. The rules violated in these cases also date back several decades. The right to a properly informed jury * the right violated third most often * is as old as the nation.
Q: The study says one solution is reserving the death penalty for the worst of the worst. What is the worst of the worst?
A: The worst of the worst are those crimes with a high concentration of aggravating factors.
There is no such thing as a good murder. But the laws of states with the death penalty list factors that make murders more aggravated because they make the crime more serious or the offender more deserving of punishment. The study finds that death verdicts imposed where not just one, but several, of these factors are present are substantially less likely to be overturned due to serious error. Everything else equal, the addition of each additional aggravating circumstance made reversal 15% less likely at the third stage of review.
Q: Doesn't the high reversal rate really prove the system is working because it catches mistakes?
A: No. Evidence that a lot of mistakes are made is evidence that the system is broken.
It may be tempting to think that a high error rate demonstrates that the system is catching all of its mistakes, but that itself would be a mistake. It's like saying, the fact that we get it wrong most of the time proves we get it right most of the time * clearly a nonsensical proposition. That we catch so many errors is evidence that we make a lot of mistakes. It also implies that there are probably a lot of errors we don't catch.
Suggesting that the number if serious errors the courts find is proof that the system works is like suggesting that the more time your car spends at the garage the better it works * that the higher the bills you have to pay to fix your car, the better it runs. Clearly such claims are absurd.
Q: What's the bottom line?
A If a state is going to have the death penalty, it should be saved for the worst of the worst, time and effort should be put into the first trial, and those making life and death decisions shouldn't be subject to the whims of politics. In short, if you're going to use the death penalty, do it sparingly and do it right.(source: Justice Project)