Beyond the word’s strict definition, corruption is difficult to measure.  Einstein once observed that everything we can measure does not necessarily count, whereas everything that counts we cannot necessarily measure. The same is true of corruption. Why?

Corruption is generally thought of as illicit activity, but gathering data based on such a definition can be difficult. It leaves us with mere proxies for any empirical understanding of how much corruption in a given area takes place. 

Proxies for measuring the extent of corruption include reported allegations, cases, or convictions and survey data.  Each can produce false negatives (a system appears clean but it is not) and false positives (perceptions of systemic corruption that are exaggerated, if not entirely incorrect). This is the major structural flaw of the index of Transparency International on corruption, which is based on the citizens’ perception. The other criteria are no less elusive.

Reports of actual cases are difficult to evaluate.  They tend to focus on big cases and rarely touch on the more common, smaller cases that do not rise to public attention or disclosure. The existence of such cases alone may indicate either the tip of the iceberg (reflecting deep-rooted, systemic corrupt behaviors) or in contrast that the system is working well by identifying and punishing exceptional, corrupt practices.  Without more information, it is difficult to determine which of these inferences to draw.

In particular, observers may attempt to draw inferences from the number of disciplinary actions that are reported.  This is troublesome because the lack of disciplinary action may be indicative of a more serious problem, whereas a greater number of reported actions may reflect a strong and healthy commitment to eliminate corrupt practices.

For example, a survey of 21 countries in Asia revealed that out of 238 actions against judges for corruption, approximately half of the complaints generated no action. Most of the actions were for incompetence. Only a half dozen involved bribery or corruption. 

Additionally, two-thirds of the countries involved the chief justice in the process, and most of those surveyed were satisfied with current procedures.  The author concluded from this data that these systems were effective, observing positively that there is “not only one way to fight corruption effectively.”  Alternatively, his survey may lead to a more negative conclusion, and a more correct one, that there are many ways to fight corruption ineffectively.  

Second, where evidence is unavailable or difficult to interpret, another strategy is simply to ask questions directly of officials or citizens through surveys. Surveys provide information that would otherwise be difficult, if not impossible, to obtain. Such data indicates how widespread the perception of corruption has become. 

However, the surveys also carry a number of methodological weaknesses. First, those interviewed, if asked to compare domestic to foreign experience, have little basis for comparison to other national contexts. Second, they are increasingly aware of results of previous or other surveys and have a tendency to internalize those findings. 

Third, the questions they are asked are either bluntly stated (in order to make them accessible) and either too imprecise or too detailed and technical. Fourth, surveyors may not be persuaded of the protections of confidentiality and may worry about retribution. And questions may not be objective or based on direct experience; instead, they lead to bias in the respondents, who in turn dismiss the extent of corruption or exaggerate its reach.

Beyond actual cases and surveys, scholars have begun to develop macro-quantitative and economic frameworks for identifying corruption, but these studies are in early stages of development for assessing the extent of many different kinds of corruption.

Notwithstanding all of these concerns, it is vital to note that perception itself has an impact on corruption. Just as recent research has shown that there is a network effect on social conditions and behaviors, from happiness to obesity, the same is true of tax compliance and corruption.

People who believe their neighbors do not pay taxes are less likely to do so. Likewise, persons who perceive corruption to be widespread or common are more likely to participate in corrupt practices.

This underscores the need for developing better empirical techniques: more accurate and qualitative reporting, better survey methodologies, improvements of other macro-quantitative approaches – all tied together to triangulate competing truths on corruption by taking a more humble and rigorous approach to weaknesses of current methods.

The better we can measure corruption, the more accurate will become our understanding of its reach and depth, as well as impacts. Without improving our methodologies, effective reforms will continue to rest on shaky empirical foundations.

Professor Hiram E. Chodosh is dean of the S.J. Quinney College of Law at the University of Utah. He is a world expert on comparative law and author of “Global Justice Reform: A Comparative Methodology,” published in New York in 2005.