Package org.dkpro.statistics.agreement

Interface IChanceCorrectedAgreement

All Superinterfaces:
IAgreementMeasure
All Known Implementing Classes:
BennettSAgreement, CohenKappaAgreement, FleissKappaAgreement, RandolphKappaAgreement, ScottPiAgreement
public interface IChanceCorrectedAgreement
extends IAgreementMeasure
Generic interface to be implemented by all inter-rater agreement measures that perform a chance correction and thus distinguish between observed agreement and expected agreement. The basic idea is that if raters made a random decision on each annotation unit, then the final inter-rater agreement should be zero. See also IChanceCorrectedDisagreement for the analogous definition of chance corrected measures with observed and expected disagreement.

References:
  • Artstein, R. & Poesio, M.: Inter-Coder Agreement for Computational Linguistics. Computational Linguistics 34(4):555-596, 2008.
Author:
Christian M. Meyer
See Also:
IAgreementMeasure, AgreementMeasure, IChanceCorrectedDisagreement

  • Method Details

    • calculateObservedAgreement

      double calculateObservedAgreement()
      Returns the observed agreement of an annotation study. The observed agreement is basically the proportion of annotation units that the raters agree on divided by the number of units in the given study.

    • calculateExpectedAgreement

      double calculateExpectedAgreement()
      Returns the expected agreement of an annotation study. The expected agreement is the proportion of agreement that would be expected by chance alone. The expected agreement should be equal to the observed agreement if each rater makes a random decision for each unit.