Kappa, Cohens
Get help from research-coaches.com

If the road is not yet clear to you,

we will show you the way how to get through.

Research Coaches World Wide

Kappa, Cohen's

Cohen's Kappa is a coefficient of agreement between two raters or - more general - between two variables. The maximum score is 1 meaning that a total agreement is reached. There is no minimum.


Suppose two specialists in behaviour have to classify busy children into the categories PPDNOS, ADHD, None or Both. Each child in the entity gets scores from both specialists. Between these two specialist a cross table can be made, something like this:

Cohen's Kappa

It is obvious that there is some agreement, but it is not a total agreement. The phi-coefficient does not fit because it is not a 2 x 2 table. Cramer’s V might fit, but we are interested in the agreement and not in the amount outside the diagonal. And Kendall’s tau is a coefficient of agreement too, but is applicable to ordinal data. These data aren’t ordinal. We are looking for a coefficient that is only concerned about the diagonal. The coefficient who is applicable is Cohens Kappa.


The computation of Cohens Kappa is not too difficult. It takes three steps.

Step 1: Compute the percentage of agreement, that is the numbers on the diagonal divided by the total. In this case it is ( (17 + 23 + 22 + 29) / 120 =) 75.8%.


Step 2: Create an expectation table in percentages for the cells on the diagonal. The expected percentage is the row-total times the column-total divided by the grand total. For PPDNOS this is 30 * 22 / 120 = 5.5.

Kappa, Cohen's

The percent of agreement in this table is ( 5.5 + 6.7 + 6.5 + 11.7 =) 30.4%. Complementary the percentage off the diagonal is 69.6%.


Step 3: compute the kappa-coefficient by:

Formula of Cohen's kappp

In this case it is: ( ( 75.8 – 30.4 ) / 69.6 = ) 0.65.


Interpreting the Kappa-coefficient


If Kappa is less than 0, it is a poor score.

A Kappa between 0 and 20 is said to be slight.

Between 21 and 40 it is fair.

Between 41 and 60 it is moderate.

Between 61 and 80 it is substantial.

Above 80 is an almost perfect score.

So in our example the computed kappa of .65 is substantial.


Related topics to Cohen's Kappa:


All these tests are clearly explained in in our SPSS-tutorials.

SPSS-tutorials of Research Coaches


Return to the index of this dictionary

Mission

My goal is to teach you how to conduct good research.

Because:

Good research provides you with better information.
With better information, you can make better decisions.
With better decisions, you can create a healthier, wealthier and freer world,
for people, fauna and flora, for current and future generations.

That is why I think it is important that  you know how to do your research well.