Intra- and inter-rater reliability of goniometric finger range of motion using a written protocol

Research design

We used an observational, descriptive study design to examine the intra- and inter-rater reliability of a new protocol for goniometric measurement of finger motions. The risk of bias of the present study was assessed using the COSMIN checklist (Reliability: relative measures) in the supplementary tables. The Kitasato University School of Medicine and Hospital Ethics Committee (2020-027) approved this study.

Participants

The participants were recruited from among the staff members of the hospital where the first author was employed. The exclusion criteria were as follows: (1) history of musculoskeletal condition, such as arthritis, orthopedic conditions involving the upper limbs, (2) neurological, (3) psychiatric conditions, and (4) an unstable general condition due to other complications.

Evaluator

Finger ROM was assessed by three occupational therapists (TN, CM, HT) with ≥5 years of clinical experience in the field of physical disabilities (Rater A/B/C, mean years of experience: 8.3 years).

Procedure

We developed a measurement protocol manual that was based on “Joint Range of Motion Indication and Measurement Methods” by the Japanese Orthopaedic Association and Japanese Society of Rehabilitation Medicine (5) and Meas­urement Evaluation for PT/OT: ROM Measurement, Second Edition (2). To ensure uniformity of the measurement method used, raters received a 15-minute course on the contents of the written protocol and trained for 15 minutes individually using the measurement manual.

Each participant was seated in a chair facing the table with the assessed side of the arm placed on the table. The forearm position was 0-degree rotation with a 20-degree wrist dorsiflexion. A sheet of paper with a diagram of the basic fixed axis was placed under the arm as a guide (Fig. 1A). The goniometer was placed from the dorsal side of the hand with the long handle (with fixed axis) on the basic axis and the short handle (with meter printed) on the moving axis (Fig. 1B). The thumb was measured first, followed by the index, middle, ring, and little fingers. Measurement of each finger followed the order of metacarpophalangeal (MP), proximal interphalangeal (PIP), and distal interphalangeal (DIP) joints.

First, active (voluntary) movement was measured with the accompanying verbal instruction, “Please bend XX joint of your XX finger utmost, without moving your wrist.” If other fingers were flexed at the same time, the raters instructed the participant to “try to move only your XX (targeted) finger.” Second, passive ROM was measured in the same order, with the instructions, “Please relax and let me bend your XX finger’s XX joint to the maximum.” While measuring the MP joint, extreme flexion of the interphalangeal (IP) joint was avoided, and it kept its natural orientation. The MP and DIP joints were straightened (0-degree flexion/extension) during PIP joint measurement. When the DIP joint was measured, the MP joint was straightened (0-degree flexion/extension) with the PIP joint flexed at 70-90 degrees.

Last, active composite movements of all finger flexion positions were performed following the same orders. The instructions were, “Please bend all fingers utmost without moving your wrist.” The thumb was placed closely over the basal phalange of the index finger to avoid interfering with ROM of the other fingers. If the goniometer could not fully contact the joint, we allowed measurement on a line parallel to the basic axis and axis of movement.

All three raters measured all participants twice with at least 24-hour interval to test intra-rater reliability. For inter-rater reliability, the dates of assessment were distributed so a participant was not assessed by more than one rater on the same day. Before each assessment, it was confirmed with the participants that there had been no injury or change in hand function since the last assessment. Assessment was conducted individually in a separate room to ensure the other raters were blinded, and discussion or comparison between rates was strictly prohibited.

Data analysis

Intraclass correlation coefficients (ICCs) were used for the relative reliability analysis (ICC (1,1) for intraclass reliability, ICC (2,1) for inter-rater reliability) (12). R (version 4.0.2) was used for the statistical analysis. We used heatmaps to summarize the reliability data because the study included a large number of values, based on 366 ICC calculations. Heatmaps were also of great value for presentations based on the shape of the hand. However, because heatmaps alone did not include all necessary information, we provide ICC precision data for more in-depth interpretation (Supplementary Table). In addition, minimal detectable change (MDC) was calculated for absolute reliability. The standard error of measurement (SEM) was used to calculate an MDC value with the following formula: . A SEM value was calculated as (square root of the error variance) (13).

Participant demographic characteristics

Twenty healthy adults were included in this study; no participants met the exclusion criteria and no data were missing. The mean ± standard deviation age of the participants was 36.4 ± 10.9 years (33.8 ± 8.3 years) for males and 40.3 ± 13.1 years for females, 40% were female, and 90% were right-handed (Tab. 1).

Relative intra- and inter-rater reliability

Figure 2-5 presents the results for the heatmap of intra-rater reliability of each rater and inter-rater reliability among the three raters. A darker red color indicated a higher ICC value; a lighter color indicated a lower ICC value. Both intra- and inter-rater reliability values showed no trends between dominant and non-dominant hand, type of movement, finger, or joint. Rater C’s heatmap tended to be lighter than that of rater A or B. Reliability results varied among the different raters. Compared with intra-rater reliability (Figure 2-4), ICC values for inter-rater reliability were generally low (Figure 5). Detailed ICC information, including precision data, is presented in the supplementary tables.

Absolute intra- and inter-rater reliability

Both intra- and inter-rater reliability values showed no clear trends between dominant and non-dominant hand, type of movement, finger, or joint. Absolute reliability varied depending on the different evaluators, but in many cases MDC fitted between 10 and 15. Compared with intra-rater reliability, MDC values for inter-rater reliability were generally high. Detailed MDC and SEM information, including precision data, is presented in the supplementary tables.

Relative intra-rater reliability

Heatmap analysis revealed a constant dark red color that indicated the presence of a relatively high intra-rater reliability. There were only a few differences in reliability, depending on the type of movement (active, passive/composite), dominant or non-dominant hand, and each finger and each joint. Lewis et al (10) examined intra-rater reliability of the MP, PIP, and DIP joints of the middle finger of the dominant hand in 20 healthy adults. The raters were 10 therapists using Rolyan goniometers to measure both active and passive movement. The ICC values ranged from 0.43 to 0.99. The rater with the highest reliability had ICC values of 0.84-0.99; the rater with the lowest reliability had ICC values of 0.43-0.84. In this study, rater A had the highest reliability (ICC 0.66-0.90 for active composite movement). Thus, the results of this study had acceptable reliability. A certain degree of intra-rater reliability was achieved because we developed a measurement protocol and used raters who were trained to ensure good reproducibility.

Relative inter-rater reliability

For inter-rater reliability, heatmap analysis revealed lighter red color than intra-rater reliability that indicated inter-rater reliability was relatively low compared with the ICC values of intra-rater reliability. Similar results for low inter-rater reliability for finger ROM measurements, compared with intra-rater reliability, have been published (9,10,14). Lewis et al (10) found that inter-rater reliability is lower than intra-rater reliability with ICC values in the range of 0.35-0.85. They also found that errors in ROM angle were due to biarticular muscles and short DIP joints. Ellis et al (14) found that inter-examiner measurements are less reliable than intra-examiner measurements for the comparative reliability of finger ROM measurements using goniometry and wire tracing. They included the amount of force applied to the goniometer, the accuracy of alignment during goniometer application, and identification of anatomical landmarks as reasons for inconsistent measurement outcomes with respect to errors in goniometer measurements. Short et al (15) mentioned that the size of the rater’s body (height difference) may affect the interpretation of goniometer readings. In our study, the maximum palm lengths of each rater varied from 19.5, 17.5, and 16.3 cm (average 17.8 cm), and the hand size of each participant also varied. Handling difficulties due to differences in the body structure of both raters and participants may have affected measurement consistency.

Absolute reliability

Measurement error was considered as absolute reliability. Even if the interpretation of relative reliability was acceptable, the results of absolute reliability may not be clinically acceptable. However, rather than clearly judging it to be “clinically unusable,” we would like to recommend that medical professionals leave it to the “system” for interpreting ROM. The Mayo Wrist Score (16) is a good example of a practice that takes this approach. In section 3 of the assessment (regarding ROM), the assessment is based on an ordinal scale in increments of approximately 25%, with emphasis on % normal. Even if the ROM is clinically acceptable in terms of relative reliability, medical clinicians should pay attention to the results of this study, which show that the results cannot be interpreted in terms of absolute reliability at 2-degree and 5-degree increments.