How to use krippendorff - 4 common examples
To help you get started, we’ve selected a few krippendorff examples, based on popular ways it is used in public projects.
Secure your code as it's written. Use Snyk Code to scan source code in minutes - no build needed - and fix issues immediately.
pln-fing-udelar / fast-krippendorff / sample.py View on Github 
def main():
print("Example from http://en.wikipedia.org/wiki/Krippendorff's_Alpha")
print('')
reliability_data_str = (
"* * * * * 3 4 1 2 1 1 3 3 * 3", # coder A
"1 * 2 1 3 3 4 3 * * * * * * *", # coder B
"* * 2 1 3 4 4 * 2 1 1 3 3 * 4", # coder C
)
print('\n'.join(reliability_data_str))
print('')
reliability_data = [[np.nan if v == '*' else int(v) for v in coder.split()] for coder in reliability_data_str]
print("Krippendorff's alpha for nominal metric: ", krippendorff.alpha(reliability_data=reliability_data,
level_of_measurement='nominal'))
print("Krippendorff's alpha for interval metric: ", krippendorff.alpha(reliability_data=reliability_data))
print('')
print('')
print("From value counts:")
print('')
value_counts = np.array([[1, 0, 0, 0],
[0, 0, 0, 0],
[0, 2, 0, 0],
[2, 0, 0, 0],
[0, 0, 2, 0],
[0, 0, 2, 1],
[0, 0, 0, 3],
[1, 0, 1, 0],
[0, 2, 0, 0],def main():
print("Example from http://en.wikipedia.org/wiki/Krippendorff's_Alpha")
print('')
reliability_data_str = (
"* * * * * 3 4 1 2 1 1 3 3 * 3", # coder A
"1 * 2 1 3 3 4 3 * * * * * * *", # coder B
"* * 2 1 3 4 4 * 2 1 1 3 3 * 4", # coder C
)
print('\n'.join(reliability_data_str))
print('')
reliability_data = [[np.nan if v == '*' else int(v) for v in coder.split()] for coder in reliability_data_str]
print("Krippendorff's alpha for nominal metric: ", krippendorff.alpha(reliability_data=reliability_data,
level_of_measurement='nominal'))
print("Krippendorff's alpha for interval metric: ", krippendorff.alpha(reliability_data=reliability_data))
print('')
print('')
print("From value counts:")
print('')
value_counts = np.array([[1, 0, 0, 0],
[0, 0, 0, 0],
[0, 2, 0, 0],
[2, 0, 0, 0],
[0, 0, 2, 0],
[0, 0, 2, 1],
[0, 0, 0, 3],
[1, 0, 1, 0],
[0, 2, 0, 0],
[2, 0, 0, 0],
[2, 0, 0, 0],[0, 0, 0, 0],
[0, 2, 0, 0],
[2, 0, 0, 0],
[0, 0, 2, 0],
[0, 0, 2, 1],
[0, 0, 0, 3],
[1, 0, 1, 0],
[0, 2, 0, 0],
[2, 0, 0, 0],
[2, 0, 0, 0],
[0, 0, 2, 0],
[0, 0, 2, 0],
[0, 0, 0, 0],
[0, 0, 1, 1]])
print(value_counts)
print("Krippendorff's alpha for nominal metric: ", krippendorff.alpha(value_counts=value_counts,
level_of_measurement='nominal'))
print("Krippendorff's alpha for interval metric: ", krippendorff.alpha(value_counts=value_counts))[2, 0, 0, 0],
[0, 0, 2, 0],
[0, 0, 2, 1],
[0, 0, 0, 3],
[1, 0, 1, 0],
[0, 2, 0, 0],
[2, 0, 0, 0],
[2, 0, 0, 0],
[0, 0, 2, 0],
[0, 0, 2, 0],
[0, 0, 0, 0],
[0, 0, 1, 1]])
print(value_counts)
print("Krippendorff's alpha for nominal metric: ", krippendorff.alpha(value_counts=value_counts,
level_of_measurement='nominal'))
print("Krippendorff's alpha for interval metric: ", krippendorff.alpha(value_counts=value_counts))krippendorff
Fast computation of the Krippendorff's alpha measure.
