By comparing the ratio of the Serb mean of Indicted versus enemy civilian casualties to the other group (Bosnians, Albanians, Tutsis, Germans and Croats) we see that we have 5,13 more Serbs convicted per enemy civilian casualty than the other groups.

No you don't. To find the ratio of indicted Serbs to other people's civvies, you need to do a weighted average, but you're doing a vanilla average.

When you do a weighted average, you get 2.16 indicted Serb for every 1000 enemy civilians, and 2.16 indicted Croat or Bosniac for every Serb civilian. And you get 1.02 convicted Serb for every 1000 enemy civilians, while you get 0.89 Bosniac and Croat convictions for every 1000 Serb civilians.

Also do note that the ratio of convictions will likely change as time progresses, because there is a number of indictees whose trials are not yet finished (that's a story in and of itself, but let's save that for another day).

The figures for Rwanda and Nürnberg are of somewhat dubious value to this discussion: The way you are using it assumes a linear relationship between the number of civilian deaths and the number of war criminals - that's a pretty reasonable assumption when we're talking about wars in the same ballpark (in this case between a couple of thousand civilian deaths and a couple of tens of thousands). But it may or may not be reasonable to extend it to wars involving hundreds or millions of civilian casualties.

(And of course, Re: Nürnberg, one should not forget that the Russians shot a couple of thousand Nazis in the DDR for war crimes and other stuff...)

- Jake

Friends come and go. Enemies accumulate.

These figure were then compared to the number of civilian casualties during each of these 3 wars. I limited the casualties to civilians because this is a priori what constitutes a war crime.

Er No, that would be a war crime under the 4th Geneva convention, dealing with the mistreatment of civilians, you can be indicted under the first three for war crimes too, so any mistreatment of prisoners of war also would count as war crimes dealt with under ICTY rules. So the base numbers you are working from are not necessarily correct.

Any idiot can face a crisis - it's day to day living that wears you out.

A decent diary, vlad. I don't know enough about statistics other than to say that weighting is practiced surveys aimed at predicting outcomes. I have no idea whether it applies to analysis of this kind. The statistical universe you present is small enough that were it a survey attempting to predict, weighting would definitely be called for.

That being said, your overarching point is well taken by me, that there may indeed be (and I am not as certain of it as you are evidently - these are after all, UN proceedings) a political bias involved. Right now, I'm halfway through a book about humanitarian intervention in international law. The author's main point is that historically, there has been virtually no intervention without political motives being involved, be they unspoken publically.

So far it looks impossible to divorce geopolitical competition from humanitarian interventions or human rights prosecutions (I think a reasonable extrapolation here). Press coverage is a highly determining factor here, and Russia will never catch up with the West in cases like this because few believe the Russian press. How can we tell when RIA Novosti, Moscow Times, Russia Today, Voice of Russia, Interfax, etc when there's little evidence of journalistic independence? (Yes, I read all those -  just not lately.)

The bias you propose probably exists to some extent, but is it a result of conspiracy or what I'll call the "quality of information" that good people called upon to make judgements possess? This is a problem that has been raised in the humanitarian intervention debate - the uneveness of enforcement, mostly due to uneven attention given in the press. I wouldn't be surprised if the same dynamic wasn't working here as well. These are highly, highly charged issues.

And with that, I'm off to bed. I must be mad. The alarm clock goes off in 3 1/2 hours.

"It Can't Be Just About Us"
--Frank Schnittger, ETian Extraordinaire

What you've done is shown that Serbs are convicted more often than you'd expect in random experiments.

But these aren't random experiments, they're not meant to be: there are a lot of possible explanations, other than bias, for the outcome.

Possibly Serbs did commit more war crimes than the others. Maybe Serbs are less likely to testify or bring evidence to the tribunal against others, making it harder to convict them.

Can I  suggest that your conclusion of bias is based on validating your own biases: you might be right, but this doesn't demonstrate it at all.

In fact, I'd be pleasantly surprised, given the narrative and political environment it's operating in, if there wasn't a bias of some sort against Serbs in the tribunal's operations.

The ground situation in Yugoslavia also mitigates against your argument. The Serbs weren't the victims here, they were in political control and had a commanding force on the ground. These facts alone would explain the greater number of Serbs brought to trial.

Does it produce a political dynamic which correlates to the legal process? Absolutely.

"It Can't Be Just About Us"
--Frank Schnittger, ETian Extraordinaire

Of course it's biased. Look at the judges - 90% of them are from NATO countries.

Of course it's biased. Courts are ALWAYS biased to reflect political orientations. From the Inquisition to modern day Italy (where Berlusconi has completely fucked up the legal system for his personal benefit).

Of course it's biased. The ITCY will never bring to trial those who ordered and executed NATO bombings against civilian targets in Serbia - against the Geneva convention and without UN approval.

Of course it's biased. Whereas a large portion of the Serb political and military LEADERSHIP has been indicted and convicted (or died in the process of), the same treatment was NEVER applied to Croats, Albanians or Muslims.

Of course it's biased. The ratio of indicted and convicted to civilian casualties from the 4 ethnic and/or religious groups STRONGLY SUGGEST that Serbs are getting the most severe treatment...

I completed a statistical analysis to test whether the difference between the means of the Serb data set and the non Serb data set was significant. The results indicated that, indeed, the means were significantly different with a 95% confidence interval.

Jake and Migeru have criticized this that and the other assumption, indicator, test used, etc. to demonstrate that in fact... what? What is it that you want to demonstrate? That the ICTY isn't biased? Is that what you believe? If yes, can you offer evidence to back your claim?

This is (imho) a partly interesting discussion.

The bickering is not interesting. The statistics are.

Props to vladimir for trying to show a statistical bias. Props to Jake and Migeru for critizising the flaws in the analysis.

I must admit that I did not find the statistics courses the most interesting math, maybe they would have been more itneresting if it was analysis of political functions. Anyway, is anyone going to assume a Poisson distribution and test that?

Come on, you are almost there, no need to resort to bickering when there are Poisson distributions to be had. Jake? Migeru?

Sweden's finest (and perhaps only) collaborative, leftist e-newspaper Synapze.se

is that I don't see a consensus on a sensible statement of what "an unbiased ICTY scorecard" would look like. [By the way, that's what a null hypothesis would be, vladimir seems to be calling anti-Serb bias "his null hypothesis" which is precisely what he's trying to prove, so calling it a null hypothesis is begging the question]

The observed scorecard is

To my mind, it is a differential rate of conviction to acquittal that would indicate bias - and then it would be a question of whether more/fewer people are indicted from a given group because of bias in the workings of the trials, or because of bias in the issuing of indictments. For instance, if more Serbs are convicted than Albanians, proportionally, is it because more Albanians were indicted than should have been, or because fewer Albanian indictees were convicted than should have been? And that is a question that cannot be answered from the court's scorecard.

Which I suppose is why we ended up comparing the number of indictments to the number of civilian casualties of the various factions. But again without a sensible consensus on what the "null hypothesis" would be. So you start fishing for correlations. And when you fish for correlations you cannot then use a test to show that the correlation you found is significant - that's a well-known source of confirmation bias.

So, to sum up - the situation is that there is a reference population from which indictees can come, and then we have two "error" probabilitites:

• probability of wrongly indicting an innocent person
  
• probability of wrongly not indicting a war criminal
  

These are two parameters that would have to be estimated. Then you can estimate the
* number of actual war criminals
And then you look at the trial process, and you have

• probability of wrongly convicting an innocent indictee
  
• probability of wrongly releasing a guilty indictee

This is the minimum number of parameters one would have to estimate per faction or per war or altogether.

And there's the vexed question of what the "reference population" and the "likely number of war criminals" would be.

Again, since [after over 200 comments in two threads] I don't see a sensible consensus on what an "unbiased court" would look like, I can't proceed with a statistical analysis.

Most economists teach a theoretical framework that has been shown to be fundamentally useless. -- James K. Galbraith

What you are doing above is taking three linear fits of data sets with one data point each, and averaging their slope. If you plot that operation, it looks something like the graph on the left:

When you actually plot it like this, it's pretty obvious that this isn't a sound way to analyse data. If anybody is confused about why such a plot is nonsense, I'll be happy to make an instalment of How To Lie With Numbers about fitting to something that isn't your entire data set. But for the moment, I'll assume that the graph speaks for itself.

The way you fit three independent points properly is all together. Then it looks like the graph on the right. And the slope you get from this fit might actually say something meaningful about the data.

If you do this for Serbian indictees and non-Serbian indictees, you get slopes of 0.0019 indictees/civilian and 0.0019 indictees/civilian, respectively. Uncertainties are impossible to determine, as long as we don't have even the vaguest guesstimate for the uncertainty of the underlying data.

Now, instead of fitting to a linear function, you can also fit to a linear function with offset. This may or may not be meaningful, however: On the one hand, there might be a systematic positive offset - if, say, it takes a certain minimum number of people to plan and carry out war crimes. On the other hand, one might argue that if there is no war, then there are no war criminals (by definition), which would be an argument for forcing the fits through the origin.

(As an aside, fitting techniques exist that incorporate both of these considerations, but they are a lot more work than is justified given the crudity of the data.)

Here's what I get from my fits:

Assuming proportionality (i.e. f(x) = a*x):

Serbian indictees: 0.0019 indictees/civilian
Non-Serbian indictees: 0.0019 indictees/civilian

Serbian convicts: 0.00093 convicts/civilian
Non-Serbian convicts: 0.0009 convicts/civilian

The fits where I permit an offset actually gives slopes that would naïvely indicate a bias in favour of Serbian suspects (0.0017 vs. 0.0021 and 0.00085 vs. 0.0012, for indictments and convicts, resp.). But I don't think they should be given much weight because a) we don't have any good model for what an offset might mean vis-a-vis bias, b) the relative uncertainties on the points in the lower left corner of the plot are almost certainly greater than on the upper right corner (and the lower-left points play a great part in determining the offset) and c) the number of fitted parameters is already perilously close to the total number of data points.

[All fits carried out in GNUPLOT]

- Jake

Addendum: I actually have graphs for all the fits mentioned in this comment, and if anybody is interested, I'll be happy to post them. But for reasons of space, I decided to post just the two most illustrative examples.

Addendum 2: There is a certain sense of irony to the fact that after we have jumped through all these hoops to accommodate an assumption that I found specious from the outset (namely that these three wars constitute independent data points), we can note that doing so gives a result that is even less favourable to the hypothesis advanced in this diary than the result of simply aggregating the Serb and non-Serb data into two ratios, like I did in the very first comment I made...

Friends come and go. Enemies accumulate.

I'm not qualified to comment on the statistical methods most appropriately employed in this analysis, so I will restrict myself to some non-statistical points.

1. Are we not assuming that the average number of civilians killed by a war criminal will be the same for all ethnic/national groups.  Clearly the war criminal/civilian death ratio would be different if (say) the average Serbian war criminal killed more than the the average Croat war criminal - perhaps because of access to more powerful weaponry.  In that case you would expect a lower Serb prosecution ratio.

2. Are we not conflating a huge number of different processes required to bring a war criminal to justice, any one of which could have a differential impact on the likelihood of successful prosecution?  E.g. Press coverage of atrocity, availability of survivors/witnesses, "Visibility of crime", degree of outrage created, ability to identify and detain suspects, quality of forensic and other evidence available, need to be seen to "do something" about an atrocity, quality/fairness of the judicial process itself.

3. The realities of power.  Just as someone who scams Billions can get away with a relatively minor sentence whilst a common thief gets years, the person who personally tortures/kills a few civilians is more like to be prosecuted than a Cheney responsible for the deaths of hundreds of thousands.  Justice has always been an elite/winner determined process, and is more an attempt to restrain the worst excesses of the victors than and even handed administration of justice on the same basis for all.

notes from no w here

To put the issue of conviction rates to rest, consider the following table

	    Total Serb Croat Bosniac Albanian Unknown Macedonian
Indicted      157  101	  32	   9	    9	    4	       2
Convicted      69   48	  13	   4	    3	    0	       1
Dismissed      18   11	   2	   0	    1	    4	       0
Ongoing        16    9	   6	   1	    0	    0	       0
Acquitted      14    2	   4	   3	    4	    0	       1
Died	       14   12	   1	   1	    0	    0	       0
Unknown        13    9	   4	   0	    0	    0	       0
Transferred    11    8	   2	   0	    1	    0	       0
At Large	2    2	   0	   0	    0	    0	       0

under the hypothesis that the court is not biased, this is the expected distribution (obtained by taking the "total" column and the "indicted" row and multiplying them)

	    Total   Serb  Croat Bosniac Albanian Unknown Macedonian
Indicted      157 101.00 32.000   9.000    9.000   4.000     2.0000
Convicted      69  44.40 14.100   3.960    3.960   1.760     0.8790
Dismissed      18  11.60  3.670   1.030    1.030   0.459     0.2290
Ongoing        16  10.30  3.260   0.917    0.917   0.408     0.2040
Acquitted      14   9.01  2.850   0.803    0.803   0.357     0.1780
Died	       14   9.01  2.850   0.803    0.803   0.357     0.1780
Unknown        13   8.36  2.650   0.745    0.745   0.331     0.1660
Transferred    11   7.08  2.240   0.631    0.631   0.280     0.1400
At Large	2   1.29  0.408   0.115    0.115   0.051     0.0255

At this point one would have to aggregate some of the rows and columns on order to ensure that the "expected" number in each cell is at least about 5. But we won't do this yet. First, for each cell in the table (except for the "Total" column and the "indicted" row) we compute (observer - expected)^2/(expected) as a measure of the deviation from the null hypothesis. We get

	    Total Serb Croat Bosniac Albanian Unknown Macedonian
Total	     73.4  7.5	 5.9	 8.6	 15.8	 30.9	     4.7
Convicted     2.4  0.3	 0.1	 0.0	  0.2	  1.8	     0.0
Dismissed    29.4  0.0	 0.8	 1.0	  0.0	 27.3	     0.2
Ongoing       4.0  0.2	 2.3	 0.0	  0.9	  0.4	     0.2
Acquitted    28.8  5.5	 0.5	 6.0	 12.7	  0.4	     3.8
Died	      3.6  1.0	 1.2	 0.0	  0.8	  0.4	     0.2
Unknown       2.7  0.0	 0.7	 0.7	  0.7	  0.3	     0.2
Transferred   1.4  0.1	 0.0	 0.6	  0.2	  0.3	     0.1
At Large      1.1  0.4	 0.4	 0.1	  0.1	  0.1	     0.0

Where we have added the numbers by row and by column into the "Total" row and column. The total deviation is 73.4 which should be compared with a chi-square variable with 47 degress of freedom (for 6 ethinicities x 8 trial status - 1). The p-value is 0.00819 so appears to be a significant deviation. The trouble is that the single largest deviation comes from the 4 "dismissed" cases of "unknown" ethnicity, and also from the fact that there are a lot of cells with expected numbers of observations below 1 which makes the approximation (observed - expected)^2/expected a very bad approximation to the true likelihood function, which is what we're estimating here.

So, what we need to do is to aggregate the "unknown" (I choose to do this with the smallest category, "Macedonian" into "other") and also the "al large" - which I will aggregate with the "Ongoing".

Most economists teach a theoretical framework that has been shown to be fundamentally useless. -- James K. Galbraith