More debunking of the "political profiling" study

I've previously shown numerous errors and biases in the study by Professors emeritus Donald C. Shields and John F. Cragan. Based on conversations here and at Obsidian Wings, I've developed a few more thoughts on their study, other sources of bias and flaws in their data.

Updates on yesterday's post

Thanks to commenter John Doe at Obsidian Wings, I've found someone that did ask Professor Shields about their methodology, and his answer confirms my suspicions. After a local Democrat tried to use the study to deflect attention from his own indictment, Michael Smerconish of the Philadelphia Inquirer looked into it. He reports that Shields and Cragan did a simple search (probably on Lexis/Nexis based on the search terminology reported) of news media for articles which matched the search: federal grand jury or public corruption AND elected. That's about what I expected based on their data source table. Smerconish also reports that they did a "census" of press releases from some U.S. Attorneys offices.

Also in the comments at Obsidian Wings, commenter Glasnost has been vigorously defending the story. His main assertion is that the table of investigations is a random sampling of investigations, and thus my finding of 6 unreported investigations does not undermine it at all. If this were true, that would only make disclosure of the selection criteria all the more important and highlight the relevance of all the other selection and other biases I described. I don't think it's true, but follow through the exchange between me and Glasnost to see what you make of it.

Continue reading for more

A better measure, and a good source of data

As I pointed out in my initial post "under investigation" is a term with no real legal significance or definition. Measuring "investigations" is really measuring reporters' conjecture rather than facts. A much better, more objective measure of prosecutors' actions is indictments. How many charges were actually filed by the U.S. Attorneys?

Thanks again to commenter John Doe for pointing me to a data source which I should have cited before, the U.S. DoJ's Public Integrity Section. That office supervises charges against politicians and political figures precisely to guard against partisanship in such cases. Each year the office publishes a report detailing all charges filed and convictions obtained involving public corruption, broken down by federal, state, and local officials.

According to their 2005 report, 309 local and 96 state officials were formally charged by the Department of Justice that year. That's all officials, not just elected ones. It includes cabinet secretaries, chiefs of staff, accountants, chief administrative officers, appointed chiefs of police, deputy sheriffs, assistant prosecutors, you name it. Thus, the number is going to be noticeably larger than the number of elected officials.

Data on charges (i.e., indictments) filed is publicly available. Indictments are a very objective criteria that have legal significance and are not subject to interpretation. Either someone has been indicted or they haven't. It would be a far more reliable indicator of any political bias by U.S. Attorneys. Shields and Cragan could have easily studied this data instead, as I've yet to see a federal indictment of an elected official that wasn't reported in some paper somewhere, and usually lots of them. From there, they could easily find the party membership of the person indicted.

Elected officials versus other political figures

The study looks only at elected officials. As noted above, this excludes party officials, cabinet secretaries, contract review officers, chiefs of staff, and anybody else who works for elected officials. This choice of which category of "investigated" individuals to study could also introduce a bias in the results. Skimming through left-wing sites blasting the Bush Administration and Republicans for being corrupt, I've found many instances of indictments of local Republican party officials, as well as Republican chiefs of staff, etc. As they are only party officials or government employees, not elected officials, they are not included in the study. The study doesn't look to see if perhaps more Republican party officials are being investigated than Democratic ones. It's entirely possible that Republican corruption is showing up first at the party level rather than the elected official level. Or that investigations into Republicans have focused on staff (I seem to recall that the GOP held most of the governorships during most of the past 6 years).

Local actions, not national

U.S. Attorneys, while generally subject to the direction of the Attorney General, act independently of one another, and only within the district to which they are appointed. A bias in favor of targeting Democrats, and avoiding Republicans, would have to be carried out at the local level, for the most part. Shields and Cragan's fundamental theory is that the bias shows up in local prosecutions because it flies under the radar, showing up only where the press isn't looking very closely to spot trends. It is thus instructive, even necessary, to look at the state-by-state breakdowns of the (flawed) data they provided. Local politics are local politics. I'm sure there are some conspiracy theorists out there who can imagine a directive going out from Washington to investigate only local elected officials who are Democrats, but in reality, the federal government and federal politicians have much bigger fish to fry, and could care less about the local aldermen. Any local bias is going to come from the individual actions of particular U.S. Attorneys. Broken down by state, even Shields' and Cragan's flawed data don't show a nation-wide problem.

First, they found federal investigations of state or local candidates and elected officials in only 42 states and Puerto Rico. One wonders whether the other 8 states have no corruption, lazy prosecutors, or both. I'll do a follow-up soon, and I suspect that I'll quickly find more investigations of Republicans which were omitted by Shields and Cragan. These and the following figures are based solely on the data provided by Shields and Cragan. Unless noted, I have not included any of the investigations which I discovered to exist but which were not included by Shields and Cragan.

Of the 43 states (including Puerto Rico), only 29 of them had a ratio of Democrats to Republican individuals greater than 1 (to avoid division-by-zero difficulties, I treated states where only 1 Democrat was investigated and no Republicans as a ratio of 1). Four states had more (or only) Republicans under investigation. Treat related "investigations" together as I recommended in my earlier post, and the data shows that only 26 states, just half the total, have a Democrat-to-Republican investigation greater than 1.

In the comments at Obsidian Wings and here, Tully has pointed out that one would expect to find a greater concentration of Democratic officials in large urban communities, and that these larger cities are more likely to produce corruption at a level large enough to interest a federal prosecutor. Rarely is a rural aldermen investigated by the feds for giving his brother-in-law a truckload of free gravel. The relevant population of elected officials, then, is the population in those 29 states.

We can then narrow the field further. I haven't found data on local government partisan makeup, but the National Conference of State Legislatures provides year by year partisan breakdowns of each legislature in the country. Picking a year at random, I've used the 2003 data to determine the ratio of Democrats to Republicans in the state legislature for each of the 29 states. Three states drop off, because the ratio of Democratic to Republican "investigations" is equal to or less than the ratio of Democratic to Republican legislators. For example, Massachusetts had 5.86 times more Democratic legislators in 2003, but only had 5 times as many Democratic investigations. If anything, Democrats in Massachusetts were under-investigated. If you again look at this by grouped investigations, rather than counting each individual subpoenaed as an investigation (according to the methodology I posted yesterday), 2 more states drop out, and we're left with only 24 states having a disproportionate number of Democratic investigations, based on the ratio of Democrat to Republican legislators in each state.

So, in reality, Shields' and Cragan's OWN DATA (ignoring for the moment all the data they have omitted and the selection and definitional and other biases in their study) show that fewer than half the states show a propensity toward "profiling" of Democrats for political prosecution by the U.S. Department of Justice. Even this number is probably inflated, as local governments in the south (many of the 29 states are in the south) tend to be dominated by Democrats even when the state votes Republican in statewide or federal elections, and thus the legislative partisan breakdown might not reflect the partisan breakdown of local elected officials.

If I have time in the next few days, I'm going to use this narrower list of states to look for more overlooked "investigations." I've already found several more individual elected officials and candidates who were not included in the study, but I'll save them until I've done a more comprehensive search.

UPDATE: I've done more analysis of the issue, available here and at other posts linked in that one.