CROSS-VALIDATION OF THE FUNCTIONAL EVALUATION

IN A WHEELCHAIR (FEW) INSTRUMENT WITH CONSUMER GOALS

Tamara Mills^{1, 2}, MS, OTR/L, Margo B. Holm^{1, 2}, PhD, OTR/L, ABDA, Elaine Trefler¹, MEd, OTR/L, ATP, Mark Schmeler¹, MS, OTR/L, ATP, Shirley Fitzgerald¹, PhD, Michael Boninger^{1, 3}, MD, Mary Ellen Buning¹, PhD, OTR/L, ATP, Nigel Shapcott¹, M.Sc.

¹Dept. of Rehabilitation Science & Technology, University of Pittsburgh

²Dept. of Occupational Therapy, University of Pittsburgh

³Dept. of Physical Medicine & Rehabilitation, University of Pittsburgh

ABSTRACT

A new outcome measurement tool is being systematically developed and validated in three phases over a 5 year period. The Functional Evaluation in a Wheelchair (FEW) instrument has been developed to investigate the effectiveness of seating-mobility interventions on the functional performance of individuals using wheelchairs. In Phase 1, the FEW, Beta Version 1.0 was developed based on consumer input, and consumers validated its 10 categories. In Phase 2, the test-retest reliability of the FEW, Beta Version 1.0, and cross-validation of its items with consumer goals were conducted. The results of Phases 1 and 2 are discussed in this paper.

BACKGROUND

Outcomes are necessary in the field of wheelchair seating and mobility technology to benefit consumers, clinicians, and third-party payers. Outcomes allow wheelchair users to assure their improved or maintained function resulted from receiving the most appropriate technology. Clinicians need quantifiable outcomes to validate the purpose and significance of seating-mobility interventions, and third-party payers need outcomes to justify the efficacy of clinicians’ recommendations for seating-mobility technology users. For this reason, the FEW was designed to quantify consumer function related to seating-mobility interventions (1). The FEW is expected to provide objective data to demonstrate the effectiveness of seating-mobility technology in meeting consumer needs, and justify the costs and benefits of prescribed seating-mobility interventions to third-party payers.

METHOD

PHASE 1

Ten manual and ten power wheelchair users (see Table 1) with either a progressive or non-progressive condition were interviewed by trained researchers using a modified version of a client-centered outcome measure. Consumers were asked to report and rank order their self-care, productivity, and leisure goals based on perceived importance for function in their seating-mobility system (1). A 154 item bank was created, and then frequency of response and weighted value of the ranked goals (10 = highest priority, 1 = lowest priority) were calculated to yield a weighted rank order (WRO). First, using a consensus approach the items were sorted into 21 categories, and then reallocated to 10 categories based on a review of other functional assessment tools and literature searches of seating-mobility technology (1). “The number of categories was reduced for the purpose of creating a new outcome measure that is concise, and easy to administer by clinicians, and reflects factors viewed by consumers as most affecting function in their seating-mobility system” (1). Next, 17 of the 20 Phase 1 participants validated the following 10 new categories based on the importance (10 = highest priority, 1 = lowest priority) for function in their seating-mobility system: accessing task surfaces, transfers, human-machine interface (i.e., consumers’ ability to do what they want to do in their wheelchair), architectural barriers, transportation-securement, reach, wheelchair accessories, natural barriers, transportation-portability, and transportation-accessibility (2). These 10 categories were used to develop the first version of the FEW, and then consumers’ self-administered the FEW and provided feedback on the content, wording, and scoring system. The wording of the first version of the FEW was “My seating-mobility system allows me to,” and the scoring system ranged from 6 = completely agree to 1 = completely disagree. Based on consumers’ feedback and consensus of the research team, the current FEW, Beta Version 1.0 consists of 10 questions in the following format: My wheelchair/scooter allows me to … easily (a) operate it, (b) transfer from surface to surface, (c) use accessories, (d) do tasks at different surface heights, (e) reach, (f) get around indoors, (g) get around outdoors, (h) ride public transportation, (i) secure it during transportation, and (j) stow it in a vehicle. Consumer responses on the FEW, Beta Version 1.0 are scored using a 7 point scale with a reverse ranking system: 6 = completely agree, 5 = mostly agree, 4 = slightly agree, 3 = slightly disagree, 2 = mostly disagree, 1 = completely disagree, and 0 = does not apply to me. Data analyses were conducted using SPSS.

PHASE 2

Test-Retest Reliability of the FEW, Beta Version 1.0

A total of 40 manual or power wheelchair users (see Table 1) with a non-progressive condition are being recruited from the University of Pittsburgh Medical Center (UPMC), Center for Assistive Technology (CAT). Currently, 30 consumers have consented to participate. The test-retest process involved an initial administration of the FEW, Beta Version 1.0 to CAT consumers by trained researchers. Next, consumers were asked to self-administer the FEW, Beta Version 1.0 4-7 days after the initial administration, and return the instrument by mail. Four consumers have not returned the self-administered FEW. Data analyses (n = 25) were conducted using SPSS.

Cross-Validation of FEW Items

The FEW, Beta Version 1.0 items were cross-validated with consumer goals from an internet-based (IB) study database, a telerehabilitation (TR) study database, and the CAT medicalized (CAT-M) and CAT consumer-reported (CAT-C) clinical databases. The purpose of the IB study was to examine factors related to wheelchair decision-making and the effects of consumer exposure to WheelchairNet, a website designed to inform consumers, clinicians and others regarding wheelchair use (3). Seventy-one consumers (see Table 1) reported mobility, self-care, productivity, and leisure goals on a pretest and posttest life goals questionnaire (3). In the TR study, the seating-mobility needs of 20 wheelchair users (see Table 1) were evaluated in mobility assessments using a TR and “in person” method to examine the reliability and limits of video conferencing systems in maintaining appropriate standards of care (4). Each consumer received two TR and two “in person” mobility assessments over a one week period by four different seating-mobility clinicians (two occupational therapists and two physical therapists) (4). Thus, consumers’ seating-mobility goals were collected for this study from a total of 80 evaluations. Next, the CAT-M sample (see Table 1), consisting of clinical goals for 20 consumers obtained from the therapist evaluation/prescription letter, and the CAT-C sample (see Table 1), goals individually reported by 20 consumers to therapists and recorded on an evaluation form, were systematically (CAT-M every 10^th medical record, CAT-C every 3^rd medical record) selected from the CAT clinical databases. Next, frequency data from the four databases were collected by a consensus approach each time a FEW item captured a consumer goal. If a consumer goal reflected more than one FEW item, then each item was coded as appropriate. For example, a consumer goal to use public transportation for community mobility would receive a frequency code for the following FEW items: ride public transportation, wheelchair securement, and outdoor mobility. However, a consumer goal to allow caregivers to operate a wheelchair to avoid tipping over would not be reflected in a FEW item, and thus would be added as a new category to ensure documentation of all consumers’ goals. For the clinical goals of the CAT-M and CAT-C sample and the pretest and posttest responses of the IB sample, duplicate goals for an individual consumer were coded only once. Cross-validation of the FEW, Beta Version 1.0 items with consumer goals from two research study databases and two clinical databases examined the strength of the FEW items, and identified new content items to be included in Beta Version 2.0.

Table 1. Demographics of the Phase 1, Test-Retest, and Cross-Validation Sample

	Phase 1	Test-Retest	IB	TR	CAT-M	CAT-C
	n = 20	n = 26	n = 71	n = 20	n = 20	n = 20
Age mean (min, max)	47.80 (23,87)	45.54 (19, 67)	42.66 (22, 63)	42.20 (**)	47.25 (31, 80)	54.95 (20, 83)
Gender	M = 10 F = 10	M = 15 F = 11	M = 35 F = 36	M = 10 F = 10	M = 11 F = 9	M = 10 F = 10
Race	White = 18 Black = 2	White = 22 Black = 3 Asian = 1	**	**	**	**
Diagnosis	(n=5) Spinal cord injury (SCI) (n=4) Cerebral Palsy (CP) (n=3) Spina Bifida (SB) (n=3) Post polio (n=2) Multiple Sclerosis (MS) (n=3) Other diagnoses	(n=7) CP (n=6) SCI (n=2) SB (n=11) Other diagnoses	(n=44) SCI (n=10) CP (n=4) SB (n=3) Post Polio (n=3) Arthritis (n=7) Other diagnoses	(n=4) CP (n=4) Head injury (n=12) Other diagnoses	(n=4) CP (n=2) Stroke (n=2) MS (n=12) Other diagnoses	(n=3) CP (n=3) MS (n=14) Other diagnoses
Type of wheelchair (w/c)/ mobility device	(n=10) Manual (n=10) Power	(n=16) Manual (n=10) Power	(n=30) Manual (n=24) Power (n=16) Both (n=1) Scooter	(n=13) Power (n=7) Manual	(n=9) Manual (n=5) Power (n=5) Cane/walker (n=1) Scooter	(n=13) Manual (n=5) Power (n=1) Scooter (n=1) Walker
Years of w/c use mean (min, max)	27.9 (3, 56)	19.76 (1, 52)	18.68 (4, 52)	**	**	**
Age of w/c (years) mean (min, max)	**	6.72 (6 months, 26)	**	**	6.42 (1, 20)	5.54 (1 month, 10)

** Not included in database.

RESULTS

The Phase 1 results are included in Table 2. Based on the sorting and reallocation of the 154 item bank to 10 categories, consumers perceived (WRO value) accessing task surfaces (579), architectural barriers (510), and reach (358) as most important, and transportation-portability (25), transportation-securement (27), and transportation-accessibility (75) as least important for function in their seating-mobility systems (1). Validation of the 10 FEW categories by seating-mobility users yielded (means) architectural barriers (7.00), transfers (7.00), and accessing task surfaces (6.41) as the highest priority, and transportation-securement (4.88), natural barriers (4.18), and accessories (4.00) as the lowest priority (1). For the self-administered FEW results (means), more consumers overall agreed their wheelchair allowed them to do what they wanted to do (4.65), transfer from surface to surface (4.59), and use public transportation (4.29) (1). The Phase 2 test-retest reliability results yielded an ICC (2,k) = 0.93, [CI = .84, .97; p < .001]. Table 3 included the cross-validation results of the 10 FEW, Beta Version 1.0 items with consumer goals from four databases, and 15 additional categories of goals not captured by the instrument.

Table 2. Comparison of Consumer Responses by Items, FEW categories, and Self-Administered FEW (1)

Category	WRO Sum by Items	Rank of Items	Prioritization of FEW Categories (means/medians)	Rank of FEW Categories	Self-Administered FEW Scores Mean (SD)	Rank of FEW scores
	(n = 20)		(n = 17)		(n = 17)
Accessing task surfaces	579	1	6.41/6.00	3	3.65 (1.50)	9
Architectural barriers	510	2	7.00/8.00	1	4.24 (1.35)	4
Reach	358	3	6.35/7.00	4	3.94 (1.30)	7
Human-machine interface	181	4	5.06/4.00	7	4.65 (1.32)	1
Transfers	164	5	7.00/7.00	2	4.59 (1.37)	2
Accessories	127	6	4.00/4.00	10	4.18 (1.74)	5
Natural barriers	83	7	4.18/3.00	9	3.47 (1.50)	10
Transportation-accessibility	75	8	5.47/5.00	5	4.29 (1.49)	3
Transportation-securement	27	9	4.88/5.00	8	3.82 (1.74)	8
Transportation-portability	25	10	5.18/5.00	6	4.12 (1.80)	6

Table 3. Frequency of Consumer Goals with the FEW Items and New Categories in Four Samples

FEW, BETA VERSION 1.0 ITEMS														NEW CATEGORIES
	Operate wheelchair (w/c)	Transfer		Accessories		Surface height access	Reach	Indoor mobility	Outdoor mobility	Public transportation (trans.)	Secure w/c for trans.	Stow w/c for trans.	# Captured by FEW	W/C comfort	Drive from w/c/ adaptive controls	Personal self-care in w/c	W/C caregivers can operate	W/C reliability/ maintenance	Steps	W/C safety	W/C fit/accommodation*	W/C aesthetics	W/C efficiency	W/C features*	Pressure distribution	Postural control	Increase independence in self-care	Increase sitting time & tolerance	# Not captured by FEW	Total Goals
IB	59		40	15	39		40	44	56	27	13	21	354	25	6	12	2	15	5	17	5	3	5	0	0	0	0	0	95	449
TR	80		15	37	2		2	29	40	8	16	22	251	22	0	0	0	3	1	5	75	0	0	193	0	0	0	0	299	550
CAT-M	16		12	10	7		9	17	15	10	13	11	120	12	0	4	6	16	1	18	13	0	0	19	12	18	0	0	119	239
CAT-C	15		1	0	2		2	17	12	0	2	1	52	7	1	0	0	0	0	9	1	0	1	6	3	5	8	3	44	96

* New categories captured in FEW, Beta Version 2.0.

DISCUSSION

Phase 1 and Phase 2 findings indicated wheelchair users have unmet needs relating to function and quality of life in their seating-mobility systems. Categorization of the 154 item bank into 21 categories, and then into 10 FEW categories was primarily based on the utilization of an item-fit approach and clinical expertise and consensus of the research team. “Based on a review of the interviews, we concluded that consumers considered these items as a priority for functional performance based on the interaction between themselves, the technology, and the environment” (1). Comparison of the rank order variability between consumers’ prioritization of the FEW categories and their self-administered FEW scores revealed that transfers, transportation-portability, and transportation-securement received identical ranks, and human-machine interface, accessories, and transportation-accessibility increased in rank order. Thus, Phase 1 participants’ responses changed when validating the importance of seating-mobility goals for wheelchair users in general to identifying individual goals for their perceived function in a seating-mobility system. Overall, the Phase 1 findings supported the need to further study the ability of the FEW to quantify the needs of seating-mobility users to improve or maintain function and increase quality of life.

The Phase 2 test-retest reliability results indicated that the FEW, Beta Version 1.0 demonstrated a high level of stability in the measurement of seating-mobility goals over time. The cross-validation revealed the 10 FEW, Beta Version 1.0 items would have captured 79%, 46%, 50%, and 54% of consumers’ goals in the IB, TR, CAT-M, and CAT-C samples, respectively. The FEW item, operate w/c, was the top consumer goal in the IB and TR samples, but no other consistent pattern was reflected for another FEW item or new category. The high frequency of “w/c fit/accommodation” and “w/c feature” (see Table 3) goals, particularly in the TR and CAT-M samples, has influenced the reformatting of the FEW question for the Beta Version 2.0 to: “My wheelchair/scooter features allow me to”. Utilization of this wording and addition of the new categories, “w/c fit/accommodation” and “w/c feature,” in Phase 2 would have led to capturing 80%, 94% 64%, and 61% of consumers’ goals in the IB, TR, CAT-M, and CAT-C samples, respectively. Overall, the emergence of 15 new categories not addressed in the FEW will lead to their inclusion in FEW, Beta Version 2.0. Cross validation of the FEW, Beta Version 1.0 items with goals identified by four diverse samples of seating-mobility system users has further validated the FEW items, and generated 15 new categories for consideration for Beta Version 2.0.

Future research plans include cross-validation of the FEW items with goals from larger samples of seating-mobility users, and a clinical trial to examine the capacity of the FEW to detect changes in function following acquisition of new seating-mobility technology.

REFERENCES

1. Mills, T., Holm, M. B., Trefler, E., Schmeler, M., Fitzgerald, S., & Boninger, M. (in press). Development and consumer validation of the Functional Evaluation in a Wheelchair (FEW) Instrument. Disability and Rehabilitation.

2. Mills, T., Holm, M. B., Trefler, E., Schmeler, M., Fitzgerald, S., & Boninger, M. (2001). Development of an outcome measure tool for wheelchair seating & mobility interventions: A work in progress. Proceedings of the RESNA 2001 Annual Conference, 245-247.

3. Buning, M. E. (2001). Unpublished Doctoral Dissertation, University of Pittsburgh, April 18, 2001.

4. Shapcott, N., Boninger, M., Cooper, R., Cohen, L., Cooper. R., & Fitzgerald, S. (2001). Determining the efficacy of POTS telerehabilitation for wheelchair prescription. Proceedings of the RESNA 2001 Annual Conference, 169-171.

5.

ACKNOWLEDGEMENTS

This study and the IB study are funded by the National Institute on Disability & Rehabilitation Research, U.S. Dept. of Education, Grant# H133E990001. The TR study was funded by the U.S. Dept. of Veterans Affairs, VA Rehabilitation Research & Development Service, Grant # B2159TC.

CORRESPONDENCE

Tamara Mills, MS, OTR/L

Rehabilitation Science and Technology

School of Health and Rehabilitation Sciences, University of Pittsburgh

5044 Forbes Tower, Pittsburgh, PA 15260

tamst75@pitt.edu