You gave the same answer. B for four of the five participants. So you accepted 80% of the opportunities. Your approval percentage in this example was 80%. The number of your pair of workshops may be higher or lower. Unfortunately, the limit amounts may or may not estimate the amount of random agreement in uncertainty. It is therefore doubtful that the reduction in the estimate of the agreement provided for by the kappa statistics is truly representative of the amount of the coincidence-advice agreement. In theory, the pre (e) is an estimate of the approval rate when advisors advise each position and guess with rates similar to marginal shares, and when the advisors were totally independent (11). None of these hypotheses is justified, so there are wide differences of opinion on the use of Kappa among researchers and statisticians. Subsequent extensions of the approach included versions that could deal with «under-credits» and ordinal scales.  These extensions converge with the intra-class correlation family (ICC), which allows us to estimate reliability for each level of measurement, from the notion (kappa) to the ordinal (or ICC) at the interval (ICC or ordinal kappa) and the ratio (ICC). There are also variations that may consider the agreement by the evaluators on a number of points (for example.B. two people agree on the rates of depression for all points of the same semi-structured interview for a case?) as well as cases of raters x (for example.
B how do two or more evaluators agree on whether 30 cases have a diagnosis of depression, yes/no a nominal variable). A good example of concern about the importance of Kappa`s results is a paper that compares visual detection of abnormalities in biological samples by humans with automated detection (12). The results showed only a moderate agreement between human and automated advisors for kappa (n-0.555), but the same data showed excellent percentage match of 94.2%. The problem with interpreting the results of these two statistics is: how should researchers decide whether advisors are reliable or not? Do the results indicate that the vast majority of patients receive accurate laboratory results and therefore correct medical diagnoses or not? In the same study, the researchers selected a data collector as the standard and compared the results of five other technicians to the standard. While sufficient data to calculate a percentage agreement are not included in the document, Kappa`s results have been only moderate. How does the lab head know if the results are of good quality with few discrepancies between trained laboratory technicians or if there is a serious problem and a need for training? Unfortunately, Kappa`s statistics do not provide enough information to make such a decision. In addition, a Kappa can have a confidence interval so wide that it contains everything from good to bad chord. Cohen`s original (1960) Kappa is in some cases prejudiced and is only suitable for fully cross-referenced designs with exactly two coders.