Package org.dkpro.statistics.agreement

Interface IChanceCorrectedDisagreement

All Superinterfaces:
IAgreementMeasure
All Known Implementing Classes:
KrippendorffAlphaAgreement, WeightedKappaAgreement
public interface IChanceCorrectedDisagreement
extends IAgreementMeasure
Generic interface to be implemented by all inter-rater agreement measures that perform a chance correction and thus distinguish between observed disagreement and expected disagreement. The basic idea is that if raters made a random decision on each annotation unit, then the final inter-rater agreement should be zero. See also IChanceCorrectedAgreement for the analogous definition of chance corrected measures with observed and expected agreement.

References:
  • Artstein, R. & Poesio, M.: Inter-Coder Agreement for Computational Linguistics. Computational Linguistics 34(4):555-596, 2008.
Author:
Christian M. Meyer
See Also:
IAgreementMeasure, DisagreementMeasure, IChanceCorrectedAgreement

  • Method Details

    • calculateObservedDisagreement

      double calculateObservedDisagreement()
      Returns the observed disagreement of an annotation study. The observed disagreement is basically the proportion of annotation units that the raters disagree on divided by the number of units in the given study.

    • calculateExpectedDisagreement

      double calculateExpectedDisagreement()
      Returns the expected disagreement of an annotation study. The expected disagreement is the proportion of disagreement that would be expected by chance alone. The expected disagreement should be equal to the observed disagreement if each rater makes a random decision for each unit.