From: Maximum likelihood estimation of reviewers' acumen in central review setting: categorical data
 |  |  | Reviewer 1 |  | |
---|---|---|---|---|---|
 | Observed Response | 1 | 2 | 3 | 4 |
True Response | 1 | 0.78 | 0.12 | 0.09 | 0.00 |
 | 2 | 0.15 | 0.85 | 0.00 | 0.00 |
 | 3 | 0.49 | 0.15 | 0.30 | 0.07 |
 | 4 | 0.13 | 0.02 | 0.09 | 0.76 |
 |  |  | Reviewer 2 |  | |
 | Observed Response | 1 | 2 | 3 | 4 |
True Response | 1 | 1.00 | 0.00 | 0.00 | 0.00 |
 | 2 | 0.42 | 0.52 | 0.03 | 0.03 |
 | 3 | 0.32 | 0.15 | 0.32 | 0.20 |
 | 4 | 0.00 | 0.00 | 0.07 | 0.93 |
 |  |  | Reviewer 3 |  | |
 | Observed Response | 1 | 2 | 3 | 4 |
True Response | 1 | 0.79 | 0.15 | 0.00 | 0.06 |
 | 2 | 0.08 | 0.88 | 0.00 | 0.04 |
 | 3 | 0.22 | 0.15 | 0.38 | 0.26 |
 | 4 | 0.32 | 0.04 | 0.05 | 0.60 |
 |  |  | Reviewer 4 |  | |
 | Observed Response | 1 | 2 | 3 | 4 |
True Response | 1 | 0.62 | 0.21 | 0.01 | 0.15 |
 | 2 | 0.14 | 0.76 | 0.06 | 0.03 |
 | 3 | 0.00 | 0.29 | 0.58 | 0.13 |
 | 4 | 0.02 | 0.00 | 0.06 | 0.93 |
 |  |  | Reviewer 5 |  | |
 | Observed Response | 1 | 2 | 3 | 4 |
True Response | 1 | 0.82 | 0.06 | 0.06 | 0.07 |
 | 2 | 0.30 | 0.68 | 0.00 | 0.02 |
 | 3 | 0.51 | 0.13 | 0.36 | 0.00 |
 | 4 | 0.25 | 0.04 | 0.13 | 0.58 |