Absolute reliability and concurrent validity of hand held dynamometry and isokinetic dynamometry in the hip, knee and ankle joint: systematic review and meta-analysis

2.1 Search strategy

A search was performed for relevant studies from 1987 up to and including November 2016 related to the psychometric properties of muscle dynamometry. For this, several bibliographic databases were extensively explored, including Medline, Embase, CINAHL, and the ISI Web of Science. The following words and combinations thereof were included as search terms: dynamometry, muscle strength, power or torque, isokinetic device, machine or instrument, reliability, validity, inter class or interclass, intra class or intra-class, inter rater or intra rater, inter tester, intra examiner, sensitivity or specificity, and gold standard. This database search was complemented by manually checking the bibliographies of identified papers for relevant key authors and journals. The search strategy was guided by a trained librarian.

2.2 Study selection

The inclusion criteria for studies assessed in this review were 1) included asymptomatic participants; 2) evaluated the dominant side of participants using isometric or concentric contractions (60°/s angular velocity) with either a HHD or ID in any of the joints of interest (i.e. hip, knee, or ankle); and 3) considered the following psychometric properties for the HHD and ID: a) Absolute reliability, expressed as the SEM and LOA for within subject variability between trials, and b) Concurrent validity, expressed as the inter device correlation coefficient (ICC) and 95% confidence interval (CI).

The exclusion criteria for this review were 1) studies published in a language other than English; 2) studies that analyzed only relative reliability (i.e. ICC) but not absolute reliability; 3) concurrent validity was expressed as Pearson correlation instead of ICC.

2.3 Data extraction and quality assessment

Two independent reviewers (CCH and JF) screened the abstracts/titles of the publications found in the databases. After initial selection, the reviewers then further analyzed each paper based on inclusion/exclusion criteria. Each criterion was graded on a yes/no basis. If discrepancies existed between reviewers regarding a particular paper meeting a criterion, the ratings were compared and discussed until a consensus was reached.

The following characteristics were extracted from studies analyzing the absolute reliability of hand-held and isokinetic dynamometry: i) author, ii) publication year, iii) sample characteristics, iv) type of dynamometer used, v) joint and movement studied, vi) assessment position, vii) type of muscle contraction, viii) SEM and LOA expressed as absolute values and as percentages of the mean value between peak muscle assessments (i.e. tests 1 and 2). The following characteristics were extracted from studies analyzing concurrent validity between HHDs and IDs: i) author, ii) publication year, iii) sample characteristics, iv) type of dynamometer used, v) ID used as gold standard, vi) joint and movement studied, vii) assessment position and viii) ICC_2,1 based on a 2-way random effects repeated measures analysis of variance model.

3.1 Data synthesis and analysis

Studies investigating similar outcomes (i.e. LOA, ICC) and those providing clear quantitative data were grouped, evaluated for heterogeneity, and pooled if possible. A meta-analysis was performed to quantify the pooled absolute reliability (i.e. LOA for within subject variation) of hand-held and isokinetic dynamometry in assessing muscle force in the hip, knee, and ankle joints.

Concurrent validity was quantified and expressed as the inter-machine ICC for the HHD as compared to isokinetic testing, the gold standard for assessing muscle strength. ICC was based on a 2-way random effects repeated measures analysis of variance model with absolute agreement. Munro’s scale was selected to determine the level of agreement between devices, where 0.0-0.25 represented little correlation; 0.26-0.49 low correlation; 0.50-0.69 moderate correlation; 0.70-0.89 high correlation; and 0.90-1.0 very high correlation [23].

The Stata 13 software was used to pool effects and construct the forest plots for all comparisons. This analysis used a 95% CI. A test for heterogeneity was performed using a Chi-square test (p < 0.10). If clinical heterogeneity existed in the study population or intervention, the DerSimonian and Laird Random Effects Model of Pooling was used based on the assumption of inter-study variability, thus providing more conservative estimates of the true effect. The SEM results were not pooled since, as a measure of variability, 95%CI cannot be computed. However, SEM data were described in terms of minimal, maximum, and range of numbers between studies.

4.1 Characteristics of the studies

4.2 Methodological quality of the studies

4.3 Meta-analysis results

Seventeen studies were included in the meta-analysis (Figure 1). For HHDs, absolute reliability assessments, kilograms (kg) was considered as unit of measure while Newton*meter (Nm) was considered for IDs analysis. Concurrent validity was expressed as the ICC in all selected studies. Only studies that presented HHD peak torque in Nm were considered, thereby facilitating comparison to ID studies.

Position

Body position when performing a strength assessment is relevant for results. For example, Edouard et al [41] reported that the reliability of ID shoulder rotator strength assessments is dependent on shoulder position (i.e. frontal or scapular plane with 45° or 90° of abduction). When evaluating commonly referenced texts on the topic of manual muscle testing [42, 43], it becomes clear that there is a fundamental lack of consensus for patient and practitioner positions. Nevertheless, when standardized techniques are used, the inter- and intra-rater reliability of manual testing prominently improves in healthy populations [44, 45].

The results of the present review further highlight the lack of consistency for position when assessing muscle strength. The hip abduction studies, for example, varied between standing [11, 25] and a supine position [9, 26]. Likewise, knee flexion was assessed while either seated [26, 27] or in a prone position [9]. When evaluating the reliability of HHDs, these different positions may reduce result accuracy. The highest heterogeneity (I² = 67.3) and upper LOA limit (35.2%) for hand-held dynamometry was found for hip flexion assessments, as compared to other hip movements. This could be influenced by the different assessment positions (e.g. standing [11], supine [9] and seated [26]), as compared to hip extension assessments, which only used a prone position. The upper LOA limits for knee flexion and extension were only 10.6% and 9.3%, respectively, showing accuracy in measuring peak torque, as compared with IDs. It is probable that this lower LOA was obtained due to all assessed studies using the same sitting position, as well as to IDs having good stabilization systems for assessments in this joint.

Evaluator and muscle group strengths

The reliability of the HHD test is known to increase when the rater is stronger than the subject [1]. For example, the knee extensors are a very strong muscle group and require a strong practitioner for accurate testing [13, 46]. This is congruent with the higher LOAs observed in HHDs testing knee extension (43.3%) and ankle plantar flexion (62.6%), as compared to the LOAs observed in IDs for knee extension (9.3%) and ankle plantar flexion (23.8%), where evaluator muscle force is not a factor (Tables 7 and 8). These inter-device differences were the highest among all the joints and movements included for assessment, demostrating that muscle group strength is one of the most relevant factors affecting reliability in strength-related studies.

Fixation system

The strap system applied for body fixation and patient comfort during testing [47] may contribute noise to the measurements that can alter subject performance. Burnham [48] published that if the patient is not well stabilized during HHD testing, other muscles will be involved with the process and affect reliability. HHDs do not have their own stabilization straps, and the stabilization procedure is usually unclear or not described at all in studies. Alfuth [49] reported fair ICC values (0.58, 0.82) for ankle inversion and moderate values (0.77, 0.87) for ankle eversion using a HHD. Although these are not strong muscle groups, difficulties in stabilizing the ankle joint may have contributed to this low reliability. The present systemic review found that the highest upper LOA limits using ID were for hip abduction (22.03%) and flexion (25.31%), as well as for ankle plantar flexion (23.89%) (Table 8). Difficulties in stabilizing the lumbopelvic region in hip assessments and the ankle may also be important factors affecting absolute reliability when using an ID. It is likely that if IDs would have better stabilization systems for these strength assessments, lower LOA values would be obtained.

Devices

There is still controversy on the reliability between isokinetic devices. While Thompson [50] suggested no difference between the Biodex and Cybex IDs for the knee flexors, Gross [51] demonstrated that knee flexion tests performed on the Cybex ID reached higher peak torque values.

Biological factors

While it could be possible to control all of the aforementioned factors, variations between measurements would still exist for each subject due to biological factors. This is the result of changes in mental or physical states between trials, which is equally applicable for the tester or the person assessed [40].

Limits of agreement for decision making

The LOA is a stringent decision limit for establishing improvement/deterioration in peak muscle force or torque following rehabilitation post-injury or as part of a strengthening program in a healthy individual. High heterogeneity between subjects exists for many measurements in sports medicine, as in the case of peak muscle force. Therefore, SEMs and, consequently, LOAs are high [52]. Experts in sports medicine rehabilitation, consider 10% to be a clinically relevant improvement or deterioration in muscle force [39]. In practice, one criterion for a return to sports is peak musle strength deficits under 10% of the contralateral extremity. This small, but clinically relevant, difference was only detected during knee flexion (−6.43,10.61%) and knee extension (−2.62, 9.26) using IDs. All remaining assessments using an ID, as well as full strength assessments using HHDs, showed LOA values higher than the desired 10%. As previously mentioned, this means that these devices are not able to accurately detect these clinically meaningful changes.

Concurrent validity (ICC) between HHDs and IDs

A very high correlation was found for the hip adductors and flexors. A high correlation was found for the hip abductors and extensors, and a moderate correlation existed for the knee flexors and extensors, as well as ankle plantar and dorsiflexion. The highest CI was recorded for the knee extensors and ankle plantar and dorsiflexion. Lower concurrent validity and wider CI appears in joints and movements that show higher LOAs assessed by HHDs.

Methodological elements

Considering the Cosmin checklist [22], the included studies were classified from fair and poor methodologically. Final Cosmin classifications are determined by the lowest score in any of the analyzed items. As a result, only three studies [16, 24, 32] ranked as fair in relation to absolute reliability, while the remaining were scored as poor. The major methodological flaw of these studies was related to sample size. To obtain a score of good, at least 50 subjects should be recruited. This sample size was not achieved by any study. Additionally, no study indicated if dropouts existed or how data loss was managed. Another crucially important item is the independence of measurements. Terwee et al [37] recommends at least one week between trials to ensure independence, a time met by only 8 studies.

Four studies [11, 26, 35, 36] reviewed the concurrent validity between HHDs and IDs, as expressed through an inter-device ICC correlation. Two of them [11, 36] were scored as poor because of a low sample size (< 20). One study [26] was found methodologically fair, [26] while the last was good. [35] The gold standard was always recognized or assumed.

Publication bias

A potential bias is the omission of non-English publications. There was no attempt to identify unpublished studies and doctoral thesis in this area.

Strengths and limitations

To our knowledge, this is the first systematic review and meta-analysis focusing on the absolute reliability and concurrent validity of HHDs and IDs. A comprehensive search was performed for all relevant published research over a wide range of years (1965-2016). There was restricted information regarding concurrent validity due to the scarcity of studies providing adequate data, and thus, the generalizability of the results is limited. Finally, the quality of the included studies rated from mostly poor to fair, which impeded subgroup analyses based on quality.