Introduction

For persons with tetraplegia after cervical spinal cord injury (SCI), the loss of hand function poses a significant functional deficit.1 Restoring the highest possible level of arm and hand function is, therefore, an important goal in the rehabilitation of this group.2 In the Hoensbroeck Rehabilitation Centre, this importance is translated into an approach to end therapeutic interventions only when no more improvement or progress is observed. However, it proved to be very difficult to objectify progress in a clinically meaningful way and to use test scores to answer questions like: Is this level of functioning a satisfying result in this phase of treatment? Is the current level of functioning to be seen as the maximum that can be reached? Tests like the Jebsen test3 and the Grasp-Release test 4 only rate the time used to accomplish certain tasks and thus do not provide the information that is necessary to answer these questions. Other tests like the Sollerman test are performance based, but are only in part useful to rate the specific characteristics of the tetraplegic hand.5, 6 At the moment, there is no superior test available that has been developed for people with tetraplegia, which is valid within the range of C5 to T1 spinal cord lesions.7

Development of the Van Lieshout test (VLT)

Van Lieshout developed a standardized protocol for the assessment of arm and hand function of persons with cervical SCI. This protocol evolved over a 10-year period into an assessment instrument, the VLT.8 Basic arm and hand function modalities: positioning and stabilizing the arms; development of the opening and closing of the ‘function hand’; grasp and release; and manipulation using thumb and fingers were made explicit in 19 tasks. Based on extensive patient observations, standards of excellence were made explicit for all 19 tasks. The possible ways of performance of each task were described in six hierarchical levels, resulting in a score from 5, the highest level of accomplishment, down to 0, representing that accomplishment of the task is not possible at all. The score valuing principles of performance were, ranging from low to high level of performance:

  1. 1

    bilateral task performance, combined with unsupported sitting balance is valued higher than a task performance in which the patient needs to use the back of the wheelchair to maintain sitting balance;

  2. 2

    a single-handed task performance scores higher than a bimanual performance;

  3. 3

    an active grasp function scores higher than a passive, reciprocal hand function;

  4. 4

    a spontaneous, unprepared task performance scores higher than a performance that needs to be prepared; and

  5. 5

    a visible fluent and smooth task performance scores higher than a slow, clumsy performance.

In this original ‘clinical’ version, complications like spasms or contractures that hamper performance can be scored and a satisfaction-with-performance score is given by the person with SCI for each task.8 Administration of the VLT provides a detailed and standardized assessment of tetraplegic hand function that allows therapeutic goal setting and monitoring of progress. Such an assessment takes about 60–90 min. A description of each task is given in Appendix A. The scoring of the task ‘forward reaching’ is given in Appendix B as an example of the task-specific scoring guidelines of the VLT.

The measurement concept of the VLT

Functional deficits can be measured on the levels of impairments of structure and function, of activity limitations, and of participation restrictions as defined in the international classification of functioning, disability and health (ICF).9 Grasping and moving of objects are on the level of activities (D440 and D445). However, the level of activities in the ICF covers a broad range of activities, from basic activities like grasping and moving of objects (D440 and D445) to complex activities like dressing or grooming (D520 and D540). Therefore, it is useful to make a distinction between basic activities and complex activities. The level of basic activities is comparable to the concept of functional limitation in Nagi's model of disablement as discussed by Marino and Stineman.10 The measurement concept of the VLT is, therefore, best situated at the level of basic activities (ICF classification) or functional limitation (Nagi classification).

Development of a VLT-SV

We expected that the VLT could not also be useful as a performance measure of arm/hand function for research purposes, if the total administration time could be substantially reduced. Therefore, a VLT-SV was developed. The VLT-SV includes 10 of the 19 tasks, and the level of performance of each task is scored. The total VLT-SV score is the mean of the item scores, ranging from 0 (worst arm/hand function) up to 5 (best arm/hand function). Administration time of the VLT-SV is 25–35 min.

Development of the VLT-SV was based on data of a convenience sample of 24 persons with tetraplegia.11 Items were selected that were characterized by: (a) high item–rest correlations; (b) no strongly skewed score distribution; and (c) manipulation of objects. This last criterion was applied to ensure that the VLT-SV would be a valid test of arm/hand function at the level of basic activities. The items selected are indicated by an asterisk in Appendix A. The VLT-SV showed excellent internal consistency (Cronbach's alpha=0.91). Correlations between the total scores of the VLT and VLT-SV were between 0.90 and 0.93 for the left hand, right hand, dominant hand, nondominant hand, best hand, and worst hand, indicating that the short version represents the complete VLT very well.11 In this short version, complications that hamper performance and satisfaction with performance of the person involved are not scored; the research version is a test of functional ability only.

Research questions

This paper concerns the reliability and validity of the VLT-SV. Research questions were:1 What is the internal consistency reliability of the VLT-SV?2 What is the inter-rater reliability of the VLT-SV?3 and What is the criterion validity of the VLT-SV? To answer these questions, two separate studies were performed.

Methods

Both studies were approved by the medical ethical committees of both centres and all individuals gave informed consent.

Study 1: Inter-rater reliability

Design and subject selection

A convenience sample of 12 persons with tetraplegia after cervical SCI participated. Subject characteristics are displayed in Table 1. The VLT-SV administered twice with a period of 7 days between both measurements. Raters were two trained occupational therapists who were blinded to the outcomes of each other's measurements. Patients of course could not be blinded, but were instructed not to give information to the therapist about the procedures or results of the previous measurement. The order in which both therapists measured the subjects was randomized to avoid systematic differences owing to learning effects.

Table 1 Demographic and injury characteristics of subjects with tetraplegia after SCI in the two studies
Full size table

Measures

In study 1, only the VLT-SV was administered. Data about personal and injury characteristics were retrieved from medical files.

Analyses

Agreement at item level between both raters was examined using weighted kappa. Kappa indicates agreement corrected for chance. A kappa value below 0.40 indicates ‘poor’ agreement, between 0.40 and 0.60 ‘fair’, between 0.60 and 0.75 ‘good’, and above 0.75 ‘excellent’ agreement.12 Agreement between total scores of both raters was examined using intraclass correlations (ICC).13 ICC also indicates agreement corrected for chance. The two-way random effects model for absolute agreement was used. An ICC above 0.70 is called ‘sufficient’ and above 0.80 ‘good’.13 In addition to the ICC, Bland–Altman plots were used to analyse the relationship between agreement and the height of the scores. The ‘limits of agreement’ were computed, defined as ±2 SDs of the difference score, which indicates the minimum difference between scores exceeding chance.14

Study 2: Validity

Design and subject selection

A cross-sectional study was performed involving two SCI centres in The Netherlands. Persons were invited who had tetraplegia after cervical SCI for at least 2 years. Fifty-five out of 60 persons were willing to participate in study 2 (response 92%). Demographic and injury characteristics are displayed in the right column of Table 1.

Measurements took place in the rehabilitation centre and were performed by trained occupational therapists.

Measures

Information about subject and injury characteristics was retrieved from medical files.

The motor level and completeness of the SCI were assessed following the American Spinal Injury Association (ASIA) guidelines.15 Additionally, the international classification for surgery of the hand in tetraplegia was assessed.16

Arm/hand function at the level of basic activities was measured using the VLT-SV and the Grasp-Release test (GRT).4 The GRT was designed to assess use of the Freehand system, but the test is used in other tetraplegic populations as well.7 Validity and some evidence of sensitivity to change were established.17 The test contains three objects to be handled (eg grasp, move, and release) with the lateral grasp and three objects to be handled with the palmar grasp. For the present study, the total number of correct manipulations over three sessions was used as the score for the left and right hands, respectively.

Assessment at the level of complex activities was carried out with two scales of the functional independence measure (FIM), the most often used measure for functional independence in medical rehabilitation. The FIM is sensitive to the level of lesion in tetraplegia and for improvement during clinical rehabilitation.18 It has excellent inter-rater reliability.19 Each FIM item is scored on a seven-point scale, from total assistance up to complete independence. Two of the seven FIM scales were used that are specifically relevant for arm/hand function: the self-care scale (FIM-SC; five items) and the transfer scale (FIM-TR; three items).

Analyses

The Kolmogorov–Smirnov statistic was used to test whether or not the distribution of the total VLT-SV score deviated from the normal distribution. The internal consistency reliability of the VLT-SV total score was tested using Cronbach's alpha. Criterion validity is demonstrated if scores on a new instrument show close correlations with scores on an existing instrument of good reputation measuring the same construct. The GRT was the criterion measure because the GRT is an established measure that is assumed to measure hand function at the same level, that is, basic activity, as the VLT-SV. Strong correlations between VLT-SV and GRT scores (>0.60) were expected. The other scores of hand function are at different levels, body function (ASIA; international classification) and complex activities (FIM-SC and FIM-TR), and therefore lower correlations between the VLT-SV and these scores were expected. As most measures were of ordinal level, Spearman's rank–order correlations were used.

Results

Study 1

Inter-rater reliability of the total score of the VLT-SV was excellent with ICC 0.98 (95% confidence interval (CI) 0.94–0.99) and 0.99 (95% CI 0.94–1.00) for the left and right hand, respectively. Weighted kappa's of the item scores were 0.74–0.99 for nine out of 10 items. Only the inter-rater reliabilities of the item ‘writing’ lagged behind (0.47 and 0.48), although these were still ‘fair’.

From the Bland–Altman plot (not shown) it can be concluded that the agreement between scores of both raters was independent of the height of their mean score. The limits of agreement was about ±0.5 points, indicating that an increase of 0.5 points on the VLT-SV represents an improvement of functioning exceeding measurement error.

Study 2

Table 2 shows the mean item scores and total scores of the VLT-SV. For the left hand, three subjects obtained the minimum possible score of 0 and the highest score reached was 4.9. For the right hand, one subject obtained the minimum score (0) and two subjects obtained the maximum score (5). The distribution of total scores for the left and right hand did not deviate significantly from the normal distribution (Z=0.443, P=0.990 and Z=0.714, P=0.687). The Spearman correlation between the left- and right-hand scores of the VLT-SV was 0.50.

Table 2 Item and total scores of the VLT-SV in persons with tetraplegia after SCI (N=55)
Full size table

The mean item scores in Table 3 showed that the difficulty of the items varied from relatively easy items like the arch task to relatively difficult items like opening a bottle. Most items were performed slightly better with the right hand than with the left hand, a difference that was most pronounced for the three items that can be performed with assistance of the contralateral hand: writing, lighting a match, and opening a bottle.

Table 3 Spearman's correlations between the VLT-SV and injury characteristics, International classification, GRT, and FIM (N=55)
Full size table

Cronbach's alpha was good for both the left and right hand (0.88 and 0.94) and most item–rest correlations were above 0.60. Only the left-hand scores of the three bimanual items showed relatively poor correlations with the rest of the items, although these were still above the usual threshold of 0.30.

In Table 3, the Spearman correlations between the VLT-SV and the other measures used are displayed (Table 3). The correlations between the VLT-SV and the GRT scores were very strong for both the left and right hand (0.87 and 0.90), indicating very good criterion validity (Figure 1). Correlations between the VLT-SV scores and scores on the other measures were, as expected, lower than correlations between the VLT-SV and the GRT. However, eight out of 10 correlations were still strong (>0.60). In line with the correlations in Table 3, the distribution of VLT-SV scores for each separate level of the international classification showed that relationships between VLT-SV scores and international classification were stronger for the right hand than for the left hand (Figure 2).

Figure 1
figure 1

Relationships between total scores on the GRT and the VLT for the left arm (left side) and right arm (right side) (N=55)

Full size image
Figure 2
figure 2

Distribution of VLT-SV total scores in persons with tetraplegia after SCI for different levels of the international classification for surgery of the hand in tetraplegia for the left arm (left side) and right arm (right side) (N=55)

Full size image

Discussion

This study showed very good inter-rater reliability and criterion validity of the VLT-SV. Only one item, ‘writing’, showed less favourable results. Information of the raters indicated confusion about the way this item should be scored. To resolve this confusion, the item was renamed ‘pen grip’ in the current version of the test and an adaptation of the scoring guidelines was made, stressing that the scoring of this item is not dependent on the readability of the result but on the type of hand grip used to write. The item and total scores fell well within the scoring range of the instrument, indicating that the difficulty of the test is well adjusted to this patient group. An interesting finding is the better performance of the right hand compared to the left hand for all items, especially for the three items of bimanual execution of the task. This difference reflects the right-handedness of most of our subjects.

The very high Spearman correlations between the VLT-SV and the GRT total scores suggest that both tests reveal more or less equivalent results. Administration of the VLT-SV takes less time than administration of the GRT and might be more motivating for the patient because it does not consist of as many repetitions as possible of a few manipulations. During administration of the VLT-SV or its original version, patient and rater or therapist together search for the best (highest rated) possible way of task execution. The strong correlation between VLT-SV scores and arm/hand impairment according to the international classification gave additional proof of the validity of the VLT. The distribution of VLT-SV scores for each separate level of the international classification was, however, hampered by the small number of subjects (11 levels and 55 subjects). A larger group of patients would probably give a better impression of the relationship between arm/hand impairment and VLT-SV score.

Many tests are available that measure arm/hand function by rating levels of accomplishment, use of aids, difficulty, speed, or satisfaction with performance of various tasks.3, 4, 20, 21, 22, 23, 24 Use of these measures results in a total score or subscores for several categories of activities. An advantage of the VLT-SV is that it also provides detailed information about how the tasks of the test are carried out, which might make it more useful to establish treatment goals.

This study suffers from several limitations. Firstly, the samples of both studies were small, especially the sample of study 1. Larger-scale studies are needed to confirm our results. However, the probability of finding substantially worse figures of validity and reliability is small. In study 1, the lower boundaries of both 95% CIs for the ICC were well above 0.90, indicating that the chance of finding an ICC below thresholds (<0.80) in the population is very small. Study 2 showed very good criterion validity (correlations 0.87 and 0.90), and for this study, it can also be stated that the probability of finding less than good criterion validity figures in the population (correlations <0.60) will be very small. Secondly, the VLT was developed in the Dutch language. English and German versions are available from the author (for more information about the Van Lieshout test please contact George van Lieshout at: The Institute for Rehabilitation Research, PO Box 192, 6430 AD Hoensbroek, The Netherlands. E-mail: g.vlieshout@irv.nl. Website: www.irv.nl/vlt), but have to be tested yet.

Future studies into the psychometric qualities of the VLT-SV are in progress. As part of a longitudinal study, the dimensionality, hierarchical structure, and sensitivity to change of this measure will be examined. Merging results from different studies, VLT-SV norm scores for persons with different lesion levels will be presented in the future. Nevertheless, the results of this study already show that the VLT-SV seems to be a useful measure of hand function in persons with tetraplegia.