Package org.dkpro.statistics.agreement.coding

Class BennettSAgreement

java.lang.Object

org.dkpro.statistics.agreement.AgreementMeasure

org.dkpro.statistics.agreement.coding.CodingAgreementMeasure

org.dkpro.statistics.agreement.coding.BennettSAgreement

All Implemented Interfaces:

ICodingAgreementMeasure, IAgreementMeasure, IChanceCorrectedAgreement

public class BennettSAgreement
extends CodingAgreementMeasure
implements IChanceCorrectedAgreement

Implementation of Bennett et al.'s S (1954) for calculating a chance-corrected inter-rater agreement for two raters. The measure assumes a uniform probability distribution for all raters and categories. The measure is equivalent to Janson and Vegelius's (1979) C and Brennan and Prediger's (1981) kappa_n.

References:

• Artstein, R. & Poesio, M.: Inter-Coder Agreement for Computational Linguistics. Computational Linguistics 34(4):555-596, 2008.
• Bennett, E.M.; Alpert, R. & Goldstein, A.C.: Communications through limited response questioning. Public Opinion Quarterly 18(3):303-308, 1954.
• Brennan, R.L. & Prediger, D.: Coefficient kappa: Some uses, misuses, and alternatives. Educational and Psychological Measurement 41(3):687-699, 1981.
• Janson, S. & Vegelius, J.: On generalizations of the G index and the phi coefficient to nominal scales. Multivariate Behavioral Research 14(2):255-269, 1979.

Author:

Christian M. Meyer

Field Summary

Fields inherited from class org.dkpro.statistics.agreement.coding.CodingAgreementMeasure

study

Constructor Summary

Constructors

Constructor	Description
BennettSAgreement(ICodingAnnotationStudy study)	Initializes the instance for the given annotation study.

Method Summary

Modifier and Type	Method	Description
double	calculateExpectedAgreement()	Calculates the expected inter-rater agreement that assumes a uniform distribution over all raters and annotations.

Methods inherited from class org.dkpro.statistics.agreement.coding.CodingAgreementMeasure

calculateObservedAgreement, doCalculateItemAgreement, ensureTwoRaters, warnIfMissingValues

Methods inherited from class org.dkpro.statistics.agreement.AgreementMeasure

calculateAgreement

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.dkpro.statistics.agreement.IAgreementMeasure

calculateAgreement

Methods inherited from interface org.dkpro.statistics.agreement.IChanceCorrectedAgreement

calculateObservedAgreement

Constructor Details

BennettSAgreement

public BennettSAgreement(ICodingAnnotationStudy study)

Initializes the instance for the given annotation study. The study may never be null.

Method Details

calculateExpectedAgreement

public double calculateExpectedAgreement()

Calculates the expected inter-rater agreement that assumes a uniform distribution over all raters and annotations.

Specified by:

calculateExpectedAgreement in interface IChanceCorrectedAgreement

Overrides:

calculateExpectedAgreement in class AgreementMeasure

Throws:

java.lang.NullPointerException - if the annotation study is null.

java.lang.ArithmeticException - if there are no annotation categories.