Does the modified shuttle test exhibit a ceiling effect in healthy and cystic fibrosis children and adolescents?

Introduction

Cystic fibrosis (CF) is a hereditary, autosomal recessive disease caused by a mutation in the gene responsible for encoding the cystic fibrosis transmembrane conductance regulator protein (CFTR). The absence or dysfunction of this protein results in a multisystemic disease, leading to obstruction in secretory glands (1). As the disease progresses, exercise capacity declines due to a multifactorial etiology (2), including chronic infection (3), lung function impairment (4), peripheral muscle dysfunction (5), and ventilation impairment (6). This reduced exercise capacity is associated with a higher risk of hospitalization for pulmonary exacerbations (7) and a poorer prognosis leading to increased mortality (8).

International guidelines strongly recommend including exercise capacity assessments as a standard component in regular evaluations for individuals with CF (9). The gold standard for assessing exercise capacity is the cardiopulmonary exercise test (CPET), despite its logistical limitations such as testing time, space, cost, and the need for specialized expertise (10). Cost-effective alternatives encompass field tests like the six-minute walking test (6MWT) and the shuttle tests (9). While the 6MWT has been thoroughly studied, the physiological responses vary when compared to a CPET, mainly because the former involves submaximal effort, especially in patients with low-severity disease. In contrast, the shuttle tests stand out as validated incremental protocols designed to assess maximal exercise capacity (11).

The original shuttle test consisted of 12 levels (ST-12) of progressively increasing speed, covering a 10-m distance until the subject could no longer maintain the pace or experienced fatigue, dyspnea, or signs of alertness (12). The ST-12 was developed for adults with chronic obstructive pulmonary disease and later validated for application in children with CF (13). Given that the ST-12 did not elicit a maximal response in patients with minimal disability, it was modified by adding three levels and allowing patients to run (14–16). Despite this expansion, the 15-level modified shuttle test (MST-15) may still be a submaximal test for some patients, limiting its applicability for those with high exercise capacity. Subsequently, a novel version, the 25-level modified shuttle test (MST-25), has been developed (17,18).

To date, only two reports have identified a ceiling effect for the MST-15. The first, a conference abstract, reported a ceiling effect in 6% of adult CF patients and 31% of healthy peers (17). The second found that 40% of children and adolescents with CF reached the 15th level of the MST-15 (18). However, these findings are based on either the authors’ clinical experience (17) or small sample sizes (18), which may introduce bias into the results. Our hypothesis is that the impact of the ceiling effect of the MST-15 in children and adolescents is relatively low, even within a cohort encompassing both CF and healthy individuals. Therefore, this study aims to provide a description of the percentage of healthy and CF children and adolescents reaching the ceiling of the MST-15. New evidence on the topic may help to guide healthcare professionals in choosing adequate tests to evaluate exercise capacity. Secondarily, we examine the associations between MST-15 distance and demographic, anthropometric, and lung function data.

Methods

A retrospective cross-sectional study was conducted on previously collected data from the Pediatric Physical Activity Laboratory. The sample of healthy participants from our database comprised volunteer children and adolescents of both genders from public and private schools in Southern Brazil, who met the following inclusion criteria: (i) age between 6 and 19 years, (ii) body mass index (BMI) between the 5th and 85th percentile (19), (iii) absence of chronic or acute neurological, orthopedic, respiratory, cardiac, or endocrine diseases contraindicating participation in school physical education, and (iv) forced expiratory volume in 1 second/forced vital capacity (FEV₁/FVC) above the lower limit of the reference values (20). Patients of both sexes diagnosed with CF were recruited from databases of two specialized CF centers, as a part of clinical assessments or annual reviews, based on the following inclusion criteria: (i) confirmed CF diagnosis through genetic testing, (ii) age between 6 and 19 years, and (iii) regular follow-up at two specialized CF centers. Patients were excluded if they exhibited signs of hemodynamic instability (altered blood pressure or heart rate (HR) responses), exacerbation of respiratory symptoms within the last 30 days (increased cough and expectorated sputum, changes in secretion, a decline in lung function by more than 10%), and/or osteoarticular or musculoskeletal changes that could interfere with the performance of the test. None of the participants were under use of CFTR modulator therapy.

Demographic (age and gender) and anthropometric (weight, height, BMI) data were collected from all participants. Weight measurement was conducted in an upright position using a calibrated digital scale (G-tech, Glass 1 FW, Rio de Janeiro, Brazil, or 110 F, Welmy, São Paulo, Brazil) with a precision of 100 g. Height was obtained with a portable stadiometer (AlturaExata, TBW, São Paulo, Brazil), accurate to 1 mm, and participants were barefoot. BMI was calculated and expressed in kg/m². For participants with CF, additional clinical and genetic data were presented. Pseudomonas aeruginosa chronic colonization was defined as the persistent presence of the bacterium in the oropharyngeal swab or sputum culture for at least 6 months or in three consecutive collections (21).

Lung function assessments were performed through spirometry using a KOKO spirometer (Louisville, CO, USA), following the criteria established by the American Thoracic Society-European Respiratory Society (ATS/ERS) (22). The parameters under evaluation encompassed FVC, FEV₁, the FEV₁/FVC ratio, and forced expiratory flow between 25% and 75% of FVC (FEF_25–75%). Data were presented in absolute values, z-scores, and as a percentage of predicted values derived from an international reference equation (23).

The MST-15 was conducted following the guidelines outlined by the ATS/ERS (24). Participants were advised to abstain from vigorous physical activity, avoid consuming caffeine within the 24 hours leading up to the test, ensure a minimum of 8 hours of sleep the night before, and have a light meal. A 10-m circuit was demarcated by two cones positioned at a 9-m distance, leaving half a meter on each side to account for the change of direction. Participants navigated the circuit at a pace signaled by an acoustic cue. The initial speed was set at 0.5 m/s and increased by 0.17 m/s at each level until the completion of the test. Initial instructions and encouragement during the test adhered to standardized protocols. The test was finished under the following circumstances: the participant failed to reach the cone on two consecutive occasions, the participant did not sustain the required speed, the participant reached the maximum distance of 1,500 m, or the participant exhibited peripheral oxygen saturation (SpO₂) below 85%. Prior to the commencement and upon completion of the test, measurements were taken for HR, SpO₂ (Nonin®, Minneapolis, USA), blood pressure (BIC sphygmomanometer, Itupeva, Brazil), and the modified Borg scale score for dyspnea (rated from 1 to 10). HR and SpO₂ were continuously monitored throughout the test. The distance covered in the test was calculated by counting the total number of laps and expressed in meters and as a percentage of predicted values (25).

The normality of the data was assessed using the Kolmogorov-Smirnov test, and the results were reported by presenting the mean and standard deviation or median and interquartile range, according to distribution. Categorical variables were expressed as absolute and relative frequencies. Comparisons between groups were performed using the Student’s t-test or the Mann-Whitney U-test. A linear regression analysis was conducted to assess the association of sex, age, BMI (z-score), and FEV₁ (z-score) with the distance covered in the MST-15. For each variable, β-coefficients were presented along with their 95% confidence intervals and the associated p-values. Additionally, a correlation analysis was performed to examine the relationship between age and distance covered in the MST-15 for both the CF group and the healthy group, with p-values reported. Pearson correlation coefficients were categorized as low (r = 0.0 to 0.3), moderate (r = 0.3 to 0.7), or high (r = 0.7 to 1.0). Scatter plots illustrating the linear relationship between age and distance covered in the MST-15 were also presented for both groups. In all cases, significance was considered when p ≤ 0.05.

Results

The healthy participant group included 286 volunteers, comprising 140 males with a mean age of 11.5 ± 3.3 years. Spirometry parameters indicated normal lung function, as expected (FEV₁ 103.0 ± 16.8% of predicted and FVC 98.9 ± 16.6% of predicted). The CF sample comprised 70 individuals, with a mean age of 11.9 ± 4.4 years, consisting of 47 males. Genotyping revealed that over half of the participants had a heterozygote F508del mutation (57.1%), and 12 were chronically colonized by P. aeruginosa. Spirometry assessments indicated a mild decline in lung function, with a mean FEV₁ of 77.2 ± 23.9% of the predicted, while FVC was 83.8 ± 20.9% of the predicted. The characteristics of the study sample are shown in Table I.

In the MST-15, healthy participants achieved a mean level of 12.7 ± 1.5, covering a distance of 1065.9 ± 232.9 m (110.2 ± 23.5% of the predicted), while participants with CF attained an average level of 10.5 ± 2.1 and covered a distance of 760.5 ± 261.3 m (71.2 ± 21.2% of the predicted). All parameters related to the evaluation of exercise capacity using the MST-15 are presented in Table II.

A ceiling effect for the MST-15 was observed in 19 (6.6%) participants in the healthy group and in only 1 (1.4 %) patient in the CF group. For healthy participants, we conducted a subgroup analysis by age, categorizing individuals into the following age ranges: 6–10, 11–13, 14–16, and 17–19 years. Our findings showed that the ceiling effect was age-related, with higher occurrences in older participants, while no cases were observed in those under 10 years. Figure 1 illustrates the percentage of healthy participants exhibiting the ceiling effect on the MST-15 across each age group. This analysis was not performed in the CF group, as only one participant, an 18-year-old male, reached the 15th level. In addition, the correlation analysis between age and distance covered in the MST-15 (Fig. 2) revealed a high correlation for the healthy group (r = 0.777; p < 0.001) and a moderate correlation for the CF group (r = 0.538; p < 0.001).

Table III displays the association of the MST-15 distance with demographic, anthropometric, and lung function variables, using a univariate linear regression analysis. The analyses revealed significant associations between age and MST-15 distance for both healthy (β-coefficient = 53.6, p < 0.001) and CF groups (β-coefficient = 32.1, p < 0.001). Additionally, we identified significant associations between sex and MST-15 distance for the healthy group (β-coefficient = 107.0, p < 0.001) and between FEV₁ and MST-15 distance for the CF group (β-coefficient = 31.0, p = 0.05).

Discussion

The results obtained in the present study reveal that 1.4% of CF participants and 6.6% of healthy children and adolescents reached the end of the 15th level of the MST-15. Our findings further indicate that the ceiling effect is age-dependent, with no evidence of a ceiling effect observed in participants younger than 10 years and a maximum of 25% in healthy participants aged 17 to 19.

The ERS recently updated its guidance and standard operating procedures for functional exercise testing in CF. The ERS now recommends using the MST-15 for patients with moderate to severe lung disease, and the MST-25 for those with mild-to-moderate lung disease, considering the possibility that some individuals may complete the MST-15 without reaching their exercise capacity limits (9). Our findings suggest that, even among healthy participants with preserved lung function, the 15 levels of the MST-15 are sufficient to assess exercise capacity in children under 10 years. However, consideration should be given to extending the MST-15 as children transition into adolescence, as a ceiling effect (up to 25%) may occur. It is important to note that our cohort did not include healthy or CF adults. Therefore, the incidence of the ceiling effect in this population warrants further investigation to enhance understanding of its impact on the MST-15 within a broader demographic.

Our results contrast with a recent study reporting that 40% of children and adolescents with CF reached the maximum level in the MST-15, indicating a high ceiling effect (18). While no clear or definitive factors explain these discrepancies, some considerations may shed light on the observed variations in the proportion of children and adolescents demonstrating a ceiling effect across our study and that previous report (18). Differences in sample characteristics could play a role, as a smaller sample size (n = 20 vs. n = 70) might not capture the broader clinical variations seen in typical practice. Additionally, it is possible that a higher exercise capacity (mean MST-25 distance of 1408 ± 298 m) and the introduction of new CFTR modulator therapy in 55% of the sample could have contributed to the observed high exercise performance (18).

Various demographic and clinical variables demonstrated associations with the distance covered in the MST-15. In both healthy children and those with CF, age was significantly associated with the distance covered during the test, with a greater effect observed in the healthy group compared to the CF group. As expected, the simultaneous growth in height and muscle mass with age leads to a corresponding increase in exercise capacity. These findings align with previous reports indicating that age is a variable contributing to the prediction of exercise capacity in pediatrics (26-28). Our findings also demonstrate that, in healthy subjects, age is a key determinant of exercise performance, exhibiting a more consistent linear relationship with the distance covered. In contrast, the weaker association between age and distance in the CF participants is likely due to disease-related factors that may overshadow the linear relationship between age and exercise capacity. In the healthy group, sex was also associated with the MST-15 distance, which can be explained by boys generally having greater muscle mass compared to girls (29). However, sex was not associated with MST-15 performance in the CF group, contrary to a previous study that indicated sex plays a role in predicting exercise capacity in patients with CF (28). The fact that sex was not associated with the distance achieved in the CF group, along with the lower associations between age and MST-15 distance compared to the healthy group, could be attributed to the greater impact of lung function on exercise capacity in children and adolescents with CF. This aligns with previous findings that report a moderate to strong correlation between FEV₁ and MST-15 (14, 15, 30), while other studies show a moderate correlation only among patients with FEV₁ < 67% of predicted (27, 28). It has been suggested that exercise intolerance in patients with CF may depend more on FEV₁ in those with severe lung disease, while in those with mild-to-moderate lung disease, the limitation may be more related to the magnitude of ventilatory responses to exercise (30). Surprisingly, the β-coefficient for BMI (z-score) did not demonstrate significant association with the distance covered in the MST-15 for either the healthy or the CF group. This result contradicts a study of prediction equations for MST-15 distance in children and adolescents, where sex, age, and BMI accounted for 48% of the variability in MST-15 performance (25).

In this rapidly changing landscape for CF, characterized by the increasing implementation of CFTR modulator therapy, it may be needed to reevaluate the applicability of field tests, including the MST-15. The use of CFTR modulator therapy could potentially lead to a long-term improvement in functional capacity and could significantly increase exercise capacity beyond the expected (31, 32). It has been observed that many patients with CF, as part of their disease management, engage in a significant amount of physical activity, often surpassing even their healthy peers (33). High levels of physical activity contribute to preserving exercise capacity in children and adolescents with CF (34), despite exhibiting lower ventilatory efficiency and reduced respiratory reserve during exertion (6). In individuals with mild-to-moderate lung disease who followed exercise recommendations before starting CFTR modulator therapy, lung function and ventilatory responses to exercise are likely to normalize. This could lead to a significant enhancement in exercise capacity, prompting the utilization of MST-25 whenever CPET is not available. Therefore, the MST-25 could be considered an alternative to the MST-15 for managing potential unpredictable increases in exercise capacity, especially in adolescents. Nonetheless, it is also important to highlight that CFTR modulator therapy is not available worldwide, as well as there are patients ineligible or considered as non-responders for the use of the therapy (35).

Field tests will continue to have an important value in assessing functional exercise performance, especially when CPET is not available or recommended. MST-15 has demonstrated excellent reliability (14-16); a good correlation between distance covered and peak oxygen consumption (VO₂peak), HR, and breathlessness assessed in CPET (36); a minimal clinically important difference (MCID) of 97 m (37); responsiveness to antibiotic therapy (15); and predictability of risk for hospitalization (38). Although MST-25 has also shown good reliability and correlation with VO₂peak in children with CF (18), further research is required to define its psychometric properties, responsiveness, and MCID before standardizing its usage, as well as larger sample size studies allowing results to be generalizable to the CF population. While other 20-m shuttle tests are available (13, 39), they are typically regarded as impractical in most clinical settings.

Limitations

A limitation of this study is the lack of measurements regarding the physical activity levels of the participants, hampering our ability to assess the influence of this variable on MST-15 performance. Additionally, none of the participants with CF had initiated CFTR modulator therapy, preventing us from examining the treatment’s influence on the ceiling effect of the MST-15.

Conclusion

The results obtained in the present study indicate that the ceiling effect on the MST-15 is age-dependent, with no occurrences observed in children under 10 years and a gradual increase as participants age. Furthermore, our findings indicate that MST-15 efficiently evaluates functional exercise capacity in most children and adolescents with CF. Until further evidence to support the use of alternative field tests are available, the MST-15 remains a valid option for assessing functional exercise capacity, particularly in younger children, when the gold standard is unavailable or not recommended.

Disclosures

Conflict of interest: The authors state that there are no conflicts of interest to declare.

Financial support: This work was supported by Coordenação de Aperfeiçoamento de Pessoal de Nivel Superior – Brazil (CAPES) – Finance Code 001, and Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq).

Authors’ contributions: MAB: Conceptualization, Methodology, Formal Analysis, Writing – original draft; FSP: Methodology, Writing – original draft; FMV: Investigation, Writing – review and editing; MAB: Investigation, Writing – review and editing; EdSA: Investigation, Resources, Writing – review and editing; MVFD: Formal Analysis, Resources, Supervision, Funding Acquisition, Writing – review and editing.

Data availability statement: Data available on request: The data presented in this study are available on request from the corresponding author. The data are not publicly available due to confidentiality reasons.

Ethics approval statement: The performance of tests was approved by the Research Ethics Committees of all involved institutions, with the assigned protocol numbers 52583416.5.0000.5336, 54142716.8.3001.5119, and 29351620.6.2006.5336.

Patient consent statement: All legal guardians and participants aged at least 18 signed an informed consent form, while children and adolescents under the age of 18 signed an assent form.