Research Challenges Claims of COPS Effectiveness

For the past eight years, the most prominent of all federal crime-prevention initiatives has been the Community Oriented Policing Services (COPS) program. The COPS program was instituted to give grants to state and local law enforcement agencies to help them reduce crime by increasing community policing services. Its stated goal was to put 100,000 additional officers on America's streets.

Evaluating Effectiveness

Since the inception of the COPS program, local law enforcement agencies have used billions of its grant dollars for officer salaries, computer technology, and clerical support. However, in spite of its intentions, COPS has not proven successful when its performance has been measured by standards of social science research. Research by The Heritage Foundation, the U.S. Department of Justice, and the U.S. General Accounting Office has found consistently that COPS has failed to come close to the goal of assigning 100,000 additional officers for community policing.1

The purpose of this paper is to review recent research regarding the COPS program.

Research by The Heritage Foundation

Some observers claim that the COPS program is a proven success because crime has declined every year since the program's creation.2 In May 2001, The Heritage Foundation's Center for Data Analysis (CDA) published an evaluation of the COPS program that examined the relationship between COPS funding and changes in crime from 1995 to 1998.3

The CDA report found that grants used to hire additional officers and purchase technology were ineffective in reducing violent crime. In contrast, grants that were narrowly focused and used to target specific problems--such as domestic violence, youth firearm violence, and gangs--were somewhat effective in reducing violent crime.4 The Heritage Foundation analysis builds on research that demonstrates that how the police are deployed is more important in reducing crime than how many officers are funded.

Research by the University of Nebraska

Approximately six months after the publication of the Heritage Foundation's COPS evaluation, researchers at the University of Nebraska at Omaha and Southwest Texas State University published a federally funded evaluation of COPS.5 This study (hereinafter referred to as the Nebraska study) was financed through two COPS office grants totaling over $156,000.6

The Nebraska study found that two types of COPS grants--hiring grants and narrowly focused grants--reduced crime rates in cities with populations over 10,0007 but that redeployment grants failed to reduce crime. With regard to smaller cities with populations between 1,000 and 10,000, the Nebraska study shows that COPS grants were correlated with higher crime rates. In these cities, hiring grants were associated with an increase in violent and property crime while redeployment grants were associated with an increase in property crime. The results of the COPS-funded research have been used to support claims about the program's effectiveness.8

COMPARING THE HERITAGE AND NEBRASKA STUDIES

The Nebraska study was highly critical of prior research that did not "control for extraneous factors that may be correlated with both increases in the number of police officers and increases in crime rates, such as local politics, or fluctuation in the local economy of cities."9 Regrettably, data limitations did not permit the Nebraska study researchers to improve on the existing research. The Nebraska study failed to use data that accounted for important socioeconomic and demographic changes on a yearly basis. It also did not control for the efforts of local law enforcement.

Ignoring Important Socioeconomic and Demographic Changes

Data for all localities for six out of seven socioeconomic variables in the Nebraska study were not available on a yearly basis.10 Therefore, rather than using data for each year between 1994 and 1999, this study held the following control variables constant at 1990 levels: minority population percent, single-parent household percent, young people percent, homeownership percent, and percent of people in the same house since 1985.11 In addition, the 1994 crime rate was used as a control variable.

In a study covering the period 1994 to 1999, the use of data exclusively from 1990 for most of the control variables is inappropriate and is likely to reduce the validity of the findings. By holding controlvariables constant at 1990 levels, the study starts with outdated information and does not take into account significant demographic changes that occurred on a yearly basis between 1994 to 1999. For example, from 1990 to 1999, the portion of the population accounted for by minorities increased by almost 16 percent.12

Holding most of the control variables constant at 1990 levels fails to account for the geographic mobility of Americans. From 1991 to 1998, the percentage of Americans moving to new residences ranged from 16 percent to 17.3 percent each year.13 The Nebraska study's use of 1990 data failed to take into account many important changes during the past decade that may have influenced crime rates, such as changes in the minority and youth populations.

Disregarding the Impact of State and Local Law Enforcement

Perhaps the most surprising aspect of the Nebraska study is its assumption that state and local law enforcement efforts do not influence crime rates. The statistical model used by the researchers considers only the effect of federal funds. This approach ignores the impact of state and local expenditures on policing that dwarf the funds provided through the COPS program. During the 1994-1999 period, the COPS program had a nationwide budget of $6.9 billion, but state and local governments allocated more than $280 billion for police agencies.14 Put another way, for every $1 spent on COPS initiatives, over $40 was spent by state and local governments for police protection.

The Heritage Foundation used an alternative approach in which the statistical model took into account state and local investments in policing. This model used county-level data, which include more complete information on local spending as well as information on important socioeconomic factors that is available on a yearly basis. The Heritage Foundation study found that state and local police expenditures had a significant impact on the reduction of crime.

The Nebraska study's approach tends to bias the results toward a finding that COPS is more effective than is really the case. Although the Nebraska study asserted that the Heritage Foundation's use of county-level data is flawed because "some counties have only a small number of COPS funded agencies,"15 Heritage analysts focused on localities that received substantial COPS funding. The median amount of total COPS funding to the counties in the Heritage Foundation data set between 1995 and 1998 was $498,664, with 95 percent of the counties receiving between $424,337 and $553,953.16 If COPS grants were as effective as the Nebraska study researchers believe, this amount of COPS funding within these counties should have had a measurable impact on rates of violent crime.

Comparison with Other Studies

To help reconcile the different approaches used by the Heritage Foundation and Nebraska studies, analysts in the Center for Data Analysis reviewed 33 studies of the effect of the police on crime across multiple jurisdictions. (For a list of these studies, see the Appendix.) All of the studies have been published in academic journals.17

Variable Selection

One important variable regarding the effect of local law enforcement is best accounted for by a variable that captures the deterrent effect of police presence. Deterrence theory holds that increased police activity deters crime by making criminals believe that the probability of their arrest and
punishment is higher.18 This increased risk of detection decreases the benefits of illegal activities, and criminals who fear arrest and punishment may have second thoughts before committing crime.

A related theory suggests that increased police activity not only increases deterrence, but also increases the incapacitation of criminals.19 Increases in incapacitation can be achieved by increasing the percentage of offenses in which offenders are arrested and temporarily removed from society (crimes that are "cleared by arrest").

Of the 33 studies reviewed, the independent variables estimating the effect of the police on crime can be placed into three categories: clearance rates, number of police employees, and police expenditures. The studies and their respective control variables for the deterrent effect of the police are presented in Table 1.

Clearance rates are defined as the percentage of known offenses that result in an arrest of an offender; they are usually measured on an annual basis. Increased clearance rates are thought not only to have a deterrent effect, as some offenders will perceive criminal activity as more risky, but also to reduce the opportunity for the arrested offenders to commit additional crimes. After an arrest for a violent crime, offenders are frequently detained in jail while awaiting trial and, possibly, incarceration.

Though clearance rates do not specifically measure the incapacitation effect, they can be used as a proxy in measuring the effects of incapacitation on crime. Six studies captured the deterrent effect of the police by using clearance rates.20 In 1978, the National Research Council Panel on Research on Deterrent and Incapacitative Effects questioned the validity of using risk-of-apprehension ratios such as clearance rates because of possible measurement error.21 Measurement error occurs in clearance rates when police departments underreport known offenses in relation to arrests. By underreporting offenses, police departments can artificially inflate their clearance rates and thus appear to be more efficient in solving crimes.

As a substitute for clearance rates, police employment levels and expenditures have been used to estimate the risk of apprehension. Increasing the number of officers on the beat, measured either through actual employment levels or through expenditures, is thought to be a reasonable variable for detecting the deterrence effect of the police. A total of 19 studies conceptualized the police variable through the number of police employed,22 while nine used police expenditures in estimating the effect of the police on crime.23 To control for the deterrent effect of local law enforcement, the Heritage Foundation obtained annual state and local law enforcement expenditures on the county level from the U.S. Census Bureau.

In response to Heritage Foundation testimony presented before the Subcommittee on Crime of the U.S. House Committee on the Judiciary,24 the Nebraska study's principal researcher, Dr. Jihong Zhao, attempted to refute the Heritage criticisms in a letter to Subcommittee Chairman Lamar Smith (R-TX). Responding to the Heritage criticism that the Nebraska study failed to control for the efforts of state and local law enforcement, Dr. Zhao asserts that his study used two methods to control indirectly for local law enforcement.25 The first method is the inclusion of the 1994 crime rate as a control variable.26

Dr. Zhao asserts that "It is reasonable to postulate that the level of crime rates in individual cities reflect law enforcement efforts in controlling crime incidents in these cities."27 This reasoning, however, misunderstands the relationship between law enforcement agencies and crime. For example, under this type of reasoning, any city with a crime rate lower than New York City's would have a better police department than the New York City Police Department. But while New York City may have had a higher total crime rate of 4,031 per 100,000 residents in 1999 than Yonkers, New York, with a total crime rate of 3,169, it would be incorrect to assume that Yonkers had a better police department.28 This sort of reasoning does not account for the dramatic drop in crime in New York City as a result of innovative policing.29

It should be noted that, with good reason, none of the 33 studies cited in the Appendix uses crime as a control for the effects of local law enforcement. Using crime rates to control for the effect of local law enforcement on crime rates is not a sound technique because it does not measure the deterrent effect of local law enforcement. Variables that reflect changes in the risk of apprehension, such as police expenditures, are in standard use in the academic literature. The Nebraska study's approach represents a significant and unjustifiable departure from the current literature.

The second method used to control for local law enforcement according to Dr. Zhao will be discussed in the next section.

Modeling Technique and Data Type

Both The Heritage Foundation and Nebraska studies examine the effect of COPS grants on multiple jurisdictions and years. The Heritage Foundation study used a panel data set. Panel data sets contain information on multiple units of analysis (for example, counties and cities) over multiple years. The Heritage panel data set consists of data on 752 counties, which comprise a majority of the country's population, over four years (1995 to 1998). Its variables contain values that are unique to each county and year.

The Nebraska study contains data on 6,100 cities from 1994 to 1999, but it is not a true panel data set, because most of the control variables are held constant at 1990 levels. In his letter, Dr. Zhao asserts that dummy control variables for each city are used to measure the effects of local law enforcement on crime rates.30 In the academic literature, this type of method is called a "fixed-effects" analysis.

Fixed-effects analyses can be used to control for systematic cross-sectional and time-specific differences between the units of analysis. Specifically, fixed-effects models assist researchers in controlling for unobserved factors that are not accounted for by the control variables. The Heritage Foundation and Nebraska studies employed the fixed-effects technique by using cross-sectional dummy variables for each unit of analysis to control for unobserved differences between the units.31

Of the 33 studies identified, six used panel data sets.32 As shown in Section A of Table 2, all of the academic panel data studies, with one exception, used control variables that varied on a yearly basis. (The 1989 study by Joseph Friedman, Simon Hakim, and Uriel Spiegel did not use any control variables.33 ) Section B of Table 2 describes the Heritage Foundation and Nebraska studies. The Nebraska study's use of control variables that do not vary year by year appears to be unsupported in the academic literature that examines the effects of the police on crime. After controlling for appropriate variables, such as local law enforcement, five out of the six academic studies (Table 2, Section A) used the fixed-effects model to help control for unobserved factors.34

It is important to note that each of these fixed-effects studies included a variable that directly controls for the deterrent effect of the police. Dr. Zhao's assertion that the use of the fixed-effects model can be used as a control for the deterrent effect of local law enforcement is not supported by the academic literature, since the fixed-effects model is typically used as a technique to enhance a study, but not as a substitute for appropriate control variables.

Despite being supported by more than $156,000 in COPS funds, the Nebraska study adds little to current knowledge about the COPS program's effectiveness. The study would be substantially improved if it controlled for the deterrent effect of local law enforcement and used control variables that were updated annually.

SUMMARY

The COPS program, when tested by social science methods, was not shown to be an effective crime-fighting program; nor has it fulfilled its measurable goal of putting 100,000 additional officers on America's streets. The Heritage Foundation's research findings are based on the best available data for evaluating the effectiveness of the COPS program. The COPS-funded Nebraska study, however, is critically flawed; specifically, it failed to account for factors that may significantly influence crime rates, and its use of outdated control variables and exclusion of a control for the efforts of local law enforcement are not supported by the academic literature.

David B. Muhlhausen is a senior policy analyst in the Center for Data Analysis at The Heritage Foundation.

1. Gareth Davis, David B. Muhlhausen, Dexter Ingram, and Ralph Rector, "The Facts About COPS: A Performance Overview of the Community Oriented Policing Services Program," Heritage Foundation Center for Data Analysis Report No. CDA00-10, September 25, 2000; U.S. Department of Justice, Office of Justice Programs, National Evaluation of the COPS Program, 2000; Michael R. Bromwich, Management and Administration of the Community Oriented Policing Services Grant Program, U.S. Department of Justice, Office of Inspector General, Audit Division, Report No. 99-21, July 1999; and U.S. Department of Justice, Office of Inspector General, Special Report: Police Hiring and Redeployment Grants, Summary of Audit Findings and Recommendations, Report No. 99-14, April 1999.

2. Senator Joseph R. Biden, Jr., "Bush: Don't Cut COPS," The Baltimore Sun, April 16, 2001, p. A7.

3. David B. Muhlhausen, "Do Community Oriented Policing Services Grants Affect Violent Crime Rates?" Heritage Foundation Center for Data Analysis Report No. CDA01-05, May 25, 2002.

4. Ibid .

5. Jihong "Solomon" Zhao and Quint Thurman, A National Evaluation of the Effect of COPS Grants on Crime from 1994 to 1999, University of Nebraska at Omaha, December 2001.

6. COPS grant awarded Jihong Zhao and Quint Thurman with two grants: 2001-CK-WX-K002 ($116,735) and 2001-CK-WX-K053 ($39,902).

7. Zhao and Thurman, A National Evaluation of the Effect of COPS Grants on Crime from 1994 to 1999. The narrowly focused grants, which Zhao and Thurman call innovative grants, fund specific activities that address such problems as gang violence, domestic violence, and illegal youth firearms possession.

8. Senator Joseph R. Biden, press release, December 5, 2001, at http://biden.senate.gov/~biden/press/release/01/12/2001C05740.html (February 19, 2002).

9. Zhao and Thurman, A National Evaluation of the Effect of COPS Grants on Crime from 1994 to 1999, p. 6.

10. Ibid., p. 11, Table 1.

11. Ibid .

12. U.S. Bureau of the Census, Current Population Reports, P25-1095. The minority population grew from 24.3 percent to 28.1 percent of the total population between 1990 and 1999.

13. U.S. Bureau of the Census, Geographical Mobility: Population Characteristics, January 2000, Table A, p. 2.

14. Calculations based on Table 2 in Sidra Lea Gifford, "Justice Expenditure and Employment in the United States, 1999," Bureau of Justice Statistics Bulletin, February 2002.

15. Zhao and Thurman, A National Evaluation of the Effect of COPS Grants on Crime from 1994 to 1999, p. 19.

16. Calculations by the Center for Data Analysis. These figures represent the total amount of COPS funding lagged one year. Between 1995 and 1998, the mean amount of total COPS funding lagged one year to the counties in the Heritage Foundation data set was $1,833,413, with 95 percent of the counties receiving between $1,298,875 and $2,367,956. Total funding calculations change when the data are not lagged. The median amount of total non-lagged COPS funding to the counties in the Heritage Foundation data set between 1994 and 1998 was $665,894, with 95 percent of the counties receiving between $578,839 and $760,444. Between 1994 and 1998, the mean amount of total non-lagged COPS funding to the counties in the Heritage Foundation data set was $2,596,154 million, with 95 percent of the counties receiving between $1,786,572 and $3,405,737.

17. These studies were identified after comprehensive searches at the Library of Congress and the library at the University of Maryland Baltimore County.

18. Colin Loftin and David McDowell, "The Police, Crime, and Economic Theory: An Assessment," in What Works in Policing, ed. David H. Bayley (New York: Oxford University Press, 1988), pp. 10-25; originally published in the American Sociological Review, Vol. 47 (June 1982), pp. 393-401.

19. Isaac Ehrlich, "Participation in Illegitimate Activities: A Theoretical and Empirical Investigation," Journal of Political Economy, Vol. 81, No. 3 (1973), pp. 521-565.

20. R. A. Carr-Hill and N. H. Stern, "An Econometric Model of the Supply and Control of Recorded Offenses in England and Wales," Journal of Public Economics, Vol. 2 (1973), pp. 289-318; Jeffery I. Chapman, "An Economic Model of Crime and the Police: Some Empirical Results," Journal of Research in Crime and Delinquency, Vol. 48 (1976), pp. 48-63; Steven G. Craig and Eric J. Heikkila, "Urban Safety in Vancouver: Allocation and Production of a Contestable Good," Canadian Journal of Economics, Vol. 22, No. 4 (1989), pp. 867-884; Simon Hakim, Uriel Spiegel, and J. Weinblatt, "Substitution, Size Effects, and the Composition of Property Crime," Social Science Quarterly, Vol. 65 (1984), pp. 719-734; Thomas F. Pogue, "Effect of Police Expenditures on Crime Rates," Public Finance Quarterly, Vol. 3, No. 1 (1975), pp. 14-45; and David Sjoquist, "Property Crime and Economic Behavior: Some Empirical Results," American Economic Review, Vol. 63, No. 3 (1973), pp. 439-446.

21. Alfred Blumstein, Jacqueline Cohen, and Daniel Nagin, eds., Deterrence and Incapacitation: Estimating the Effects of Criminal Sanctions on Crime Rates (Washington, D.C.: National Research Council, Panel on Research on Deterrence and Incapacitation, 1978).

22. Roy W. Bahl, Richard D. Gustely, and Michael J. Wasylenko, "The Determinants of Local Government Police Expenditure: A Public Employment Approach," National Tax Journal, Vol. 31, No. 1 (1978), pp. 67-79; Carr-Hill and Stern, "An Econometric Model of the Supply and Control of Recorded Offenses in England and Wales"; Hope Corman and Theodore Joyce, "Urban Crime Control: Violent Crimes in New York City," Social Science Quarterly, Vol. 71, No. 3 (1990), pp. 567-584; Hope Corman, Theodore Joyce, and Norman Lovitch, "Crime, Deterrence and the Business Cycle in New York City: A VAR Approach," Review of Economics and Statistics, Vol. 69 (1987), pp. 695-700; Christopher Cornwell and William N. Trumbull, "Estimating the Economic Model of Crime with Panel Data," Review of Economics and Statistics, Vol. 76 (1994), pp. 360-366; Edwin T. Fujii and James Mak, "Tourism and Crime: Implications for Regional Development Policy," Regional Studies, Vol. 14 (1980), pp. 27-36; David F. Greenberg, Ronald C. Kessler, and Colin Loftin, "The Effect of Police Employment on Crime," Criminology, Vol. 21, No. 3 (1983), pp. 375-394; Michael Greenwood and Walter Wadycki, "Crime Rates and Public Expenditure for Police Protection: Their Interaction," Review of Social Economy, Vol. 31 (1973), pp. 138-152; Roy Howsen and Stephen Jarrell, "Some Determinants of Property Crime: Economic Factors Influence Criminal Behavior But Cannot Completely Explain the Syndrome," American Journal of Economics and Sociology, Vol. 46, No. 4 (1987), pp. 445-457; C. Ronald Huff and John M. Strahura, "Police Employment and Suburban Crime," Criminology, Vol. 17, No. 4 (1980), pp. 461-470; Stephen Jarrell and Roy M. Howsen, "Transient Crowding and Crime: The More `Strangers' in an Area, the More Crime Except for Murder, Assault, and Rape," American Journal of Economics and Sociology, Vol. 49, No. 4 (1990), pp. 483-493; Tomislav Kovandzic and John J. Sloan, "Police Levels and Crime Rates Revisited: A County-Level Analysis from Florida (1980-1998)," Journal of Criminal Justice, Vol. 30 (2002), pp. 65-76; James P. Levine, "The Ineffectiveness of Adding Police to Prevent Crime," Public Policy, Vol. 23, No. 4 (1975), pp. 523-545; Steven D. Levitt, "Using Electoral Cycles in Police Hiring to Estimate the Effect of Police on Crime," American Economic Review, Vol. 87, No. 3 (1997), pp. 270-290; Loftin and McDowell, "The Police, Crime, and Economic Theory: An Assessment"; Thomas B. Marvell and Carlisle E. Moody, "Specification Problems, Police Levels, and Crime Rates," Criminology, Vol. 34, No. 4 (1996), pp. 600-645; John Mikesell and Maureen A. Phrog-Good, "State Lotteries and Crime: The Regressive Revenue Producer Is Linked with a Crime Rate Higher by 3 Percent," American Journal of Economics and Sociology, Vol. 49, No. 1 (1990), pp. 7-19; Douglas Morris and Luther Tweeden, "The Cost of Controlling Crime: A Study in Economics of City Life," The Annals of Regional Sciences, Vol. 5, No. 1 (1976), pp. 33-49; and William Trumbull, "Estimations of the Economic Model of Crime Using Aggregate and Individual Data," Southern Economic Journal, Vol. 56 (1989), pp. 423-439.

23. Ehrlich, "Participation in Illegitimate Activities: A Theoretical and Empirical Investigation"; Joseph Friedman, Simon Hakim, and Uriel Spiegel, "The Difference Between Short and Long Run Effects of Police Outlays on Crime: Policing Deters Criminals Initially, But Later They May `Learn by Doing'," American Journal of Economics and Sociology, Vol. 48, No. 2 (1989), pp. 177-191; Simon Hakim, "The Attraction of Property Crimes to Suburban Localities: A Revised Economic Model," Urban Studies, Vol. 17 (1980), pp. 265-276; Simon Hakim, Arie Ovadia, and J. Weinblatt, "Crime Attraction and Deterrence in Small Communities: Theory and Results," International Regional Science Review, Vol. 3, No. 2 (1978), pp. 153-163; Kenneth C. Land and Marcus Felson, "A General Framework for Building Dynamic Macro Social Indicator Models: Including an Analysis of Changes in Crime Rates and Police Expenditures," American Journal of Sociology, Vol. 82, No. 3 (1976), pp. 565-604; Yih-Wu Liu and Richard H. Bee, "Modeling Criminal Activity in an Area in Economic Decline: Local Economic Conditions Are a Major Factor in Local Property Crimes," American Journal of Economics and Sociology, Vol. 42, No. 2 (1983), pp. 385-392; Lee R. McPheeters and William B. Stonge, "Law Enforcement Expenditures and Urban Crime," National Tax Journal, Vol. 27 (1974), pp. 633-644; Eugene Swimmer, "Measurement of the Effectiveness of Urban Law Enforcement--A Simultaneous Approach," Southern Economic Review, Vol. 40 (1974), pp. 618-630; and Eugene Swimmer, "The Relationship of Police and Crime," Criminology, Vol. 12, No. 3 (1974), pp. 293-314.

24. David B. Muhlhausen, "Evaluation of Effectiveness Within the Office of Justice Programs," testimony before the Subcommittee on Crime, Committee on the Judiciary, U.S. House of Representatives, March 7, 2002, at http://www.heritage.org/Research/Crime/Test030702b.cfm.

25. Letter from Jihong Zhao to Representative Lamar Smith, Chairman, Subcommittee on Crime, March 12, 2002; copies available upon request.

26. Ibid.

27. Ibid.

28. Calculations based on data from the U.S. Department of Justice, Federal Bureau of Investigation, Crime in the United States--1999.

29. George L. Kelling and William H. Sousa, Jr., "Do Police Matter? An Analysis of the Impact of New York City's Police Reforms," Manhattan Institute Civic Report No. 22, December 2001.

30. Letter from Jihong Zhao to Representative Lamar Smith, March 12, 2002.

31. For more information on fixed-effects analyses, see Cheng Hsiao, Analysis of Panel Data (Cambridge: Cambridge University Press, 1999).

32. Cornwell and Trumbull, "Estimating the Economic Model of Crime with Panel Data"; Friedman, Hakim, and Spiegel, "The Difference Between Short and Long Run Effects of Police Outlays on Crime"; Kovandzic and Sloan, "Police Levels and Crime Rates Revisited"; Levitt, "Using Electoral Cycles in Police Hiring to Estimate the Effect of Police on Crime"; Marvell and Moody, "Specification Problems, Police Levels, and Crime Rates"; and Mikesell and Phrog-Good, "State Lotteries and Crime."

33. Friedman, Hakim, and Spiegel, "The Difference Between Short and Long Run Effects of Police Outlays on Crime."

34. Cornwell and Trumbull, "Estimating the Economic Model of Crime with Panel Data"; Kovandzic and Sloan, "Police Levels and Crime Rates Revisited"; Levitt, "Using Electoral Cycles in Police Hiring to Estimate the Effect of Police on Crime"; and Marvell and Moody, "Specification Problems, Police Levels, and Crime Rates."

Sign up for Morning Bell Email