Use of a maturity model for facilitating the introduction of CAR T-cell therapy—Results of the START CAR-T project

Introduction

Chimeric antigen receptor (CAR) T-cell therapy is the most advanced immunotherapeutic strategy currently available for the treatment of hematologic malignancies (1). This personalized strategy involves several steps including the isolation of autologous T cells from the patient, their modification via retroviral transduction with a CAR construct, the expansion of the CAR-expressing T cells, and, finally, the infusion of the modified cells back into the patient (2,3). CARs are fusion proteins that contain an extracellular antibody-derived domain for the recognition of a specific antigen on the surface of tumor cells (e.g., CD19 on malignant B cells), an intracellular primary signaling domain leading to T-cell activation, and an intracellular costimulatory domain to promote effector T-cell functions (2,3).

Due to the novelty and complexity of CAR T-cell therapies, their manufacturing and adoption in clinical practice impose several new regulatory and organizational challenges. The European Medicines Agency (EMA) classifies these therapies as Advanced Therapy Medicinal Products (ATMPs), which can be administered only in centers with documented experience with hematopoietic stem cell transplantation (HSCT) and novel immunotherapies (4). The Joint Accreditation Committee ISCT (International Society for Cellular Therapy)-Europe (JACIE) has developed a series of quality standards for CAR T-cell therapy administration (5). In terms of personnel and infrastructure, qualified centers should comprise a multidisciplinary team and several highly coordinated units (hematology, transfusion, intensive care, emergency care, and neurology) (4).

The quality of healthcare is influenced by the maturity of the processes used to develop it (6). A Maturity Model (MM) can be useful for the assessment and improvement of organizational processes. MMs are used as an evaluative and comparative basis for improvement, designed to assess the maturity of a selected domain based on a more or less comprehensive set of criteria (7,8). Maturity, thus, implies an evolutionary progress in the accomplishment of a target—from an initial to a desired or normally occurring end stage (9). MMs are used to assess as-is situations, to guide improvement initiatives and to control progress (10,11). The purpose of an MM can be categorized as descriptive, prescriptive, or comparative (7). A descriptive MM is applied for as-is assessments; a prescriptive MM is applied to guide organizations in the identification of desirable future maturity levels, highlighting how to reach them according to improvement measures; a comparative MM allows benchmarking across industries and regions (7).

The START CAR-T project was conceived to facilitate the introduction of CAR T-cell therapy in Italy. An MM was developed within this project, to evaluate the maturity and performance of Italian centers with regard to CAR T-cell therapy use. This article presents the START CAR-T MM.

Methods

The development of the START CAR-T MM started with the definition of the scope and domain to which the MM would be targeted and applied (7). In particular, the identification of the most coherent organizational model for the introduction of CAR T-cell therapy in clinical sites was set as the final scope.

The second phase of MM development, in accordance with de Bruin et al (7), was the design of the model. Due to the novelty of CAR T-cell therapy in Italy, four different Italian clinical centers were involved, which had either a JACIE accreditation or a commercial qualification for CAR T-cell therapy. The selected clinical sites, represented by the directors of the CAR T-cell therapy unit, were among the first centers to adopt this therapy in Italy (Istituto Clinico Humanitas, Milano; IRCCS Ospedale San Raffaele, Milano; Policlinico Umberto I, Roma; and Ospedale San Bortolo, Vicenza). The interaction with these clinical sites allowed to: (1) map the current treatment pathway with CAR T-cell therapy; (2) outline the various therapy administration methods and the patients’ needs; (3) highlight the organizational patterns and the presence of digital solutions to support the management processes. From this interaction and the analysis of CAR T-cell therapy management and pathways of patient care, it was possible to define the MM shown in Figure 1. The proposed MM is divided into three macro-categories: “Organization” that looks at the configuration of practitioners and their interaction methods in the process of CAR T-cell therapy; “Process” that synthesizes activities addressed to define and implement pathways of treatment with CAR T-cell therapies; “Digital support” that investigates the presence of digital solutions to support the management of CAR T-cell therapies. Each macro-category is composed of different sub-components, called variables mutually exclusive and collectively exhaustive, which describe in depth features of each macro-category. The maturity of a clinical site can be investigated by positioning it into one of the four maturity stages that describe variables, from a basic (level 1) to an advanced stage (level 4), defined in accordance with the clinical sites. Along with the variables, a Hub & Spoke Network was also investigated due to the importance of collaborative healthcare networks and the alliance of professionals and/or institutions for providing patients with fair and equal opportunities of access to healthcare (12).

The first macro-category—Organization—reflects the importance of organizational strategies in healthcare systems, due to the increased uncertainty and variety hospitals are facing (13). Variables included in this macro-category are:

• Role formalization, which deals with the presence of medical center staff (e.g., doctors, biologists, nurses) specifically dedicated to the management of the care pathway of patients treated with CAR T-cell therapy and the role conferred to them.
• Coordination mechanism, which investigates the coordination modalities used by the team dedicated to the management of the care pathway of patients treated with CAR T-cell therapy in order to define strategies and share information.
• Skills and experience development, which investigates the type of courses provided, the skills and experience acquired by the personnel, all fundamental elements when new therapies are introduced.
• Physical spaces, which deal with the spaces dedicated to patients treated with CAR T-cell therapy; these spaces could be shared with the transplantation department or could be devoted exclusively to the management of CAR T-cell recipients, throughout the entire process.
• Hub & Spoke Network, which investigates the presence of formal and informal agreements with other clinical sites regarding the delegation of one or more phases of the therapy process.

The second macro-category—Process—addresses the activities implemented in the care pathway of patients treated with CAR T-cell therapy, for example, to limit waste of time and space and improve the management capacity of clinical sites (14). This macro-category is composed of four different variables:

• Procedures and accreditation: to introduce CAR T-cell therapy, Italian clinical sites need to acquire a recognized certification of quality, including the National Transplant Centre certification, the qualification by the provider of the cellular therapy, a regional accreditation, and the JACIE accreditation (or submission to obtain it). Clinical sites could also introduce specialized procedures, coherent and homogeneous with the existing ones.
• CAR-T process management, which addresses the postinfusion monitoring phase that can take place inside the intensive care unit or the transplantation department, while the transfer of patient to the intensive care unit is reserved only to cases with major complications.
• CAR-T process monitoring, which monitors a panel of clinical indicators to evaluate the care of patients treated with CAR T-cell therapy, as requested by current legislations and the JACIE.
• Hub & Spoke Network, which evaluates phases of CAR T-cell therapy that can be delegated to other sites.

Digitalization has become crucial for healthcare organization, not only due to its increasing pervasiveness but also due to its ability to respond to the challenges imposed by innovation, including rationalization of healthcare costs and increase in the quality of processes (15). The last macro-category—Digital support—is composed of the following five variables:

• The presence of support systems for clinical processes such as an Electronic Medical Record and its features, and the digital solutions used to manage CAR T-cell therapy.
• The presence of a clinical corporate repository and the type of documents and data collected by this solution; the grade of interoperability of the repository is also investigated.
• Communication and collaboration tools used within the clinical site to enable the communication between medical staff.
• The ability of clinical sites to supply the Electronic Health Record and European pathology networks (i.e., ProMISe, https://www.ebmt.org/registry/data-collection-car-t-cells) with information about the care pathway of patients treated with CAR T-cell therapy.
• The use of digital solutions within the Hub & Spoke Network that permits interactions between the clinical sites and their Spoke centers.

The four clinical sites were involved also in a first test of the validity of the developed MM. In particular, applying the Plan-Do-Check-Act (PDCA) cycle inspired in 1950 by Deming (16), it was possible to evaluate the capability of the MM to investigate salient organizational, procedural, and digital support features used to manage CAR T-cell therapy and patient care pathways. After the intervention made to improve the validity of the MM, the tool was applied again and discussed within clinical sites to assess its reliability.

Following the test phase, a survey was performed to extend the MM use to a broader number of clinical sites and to evaluate its generalizability. The survey was based on the salient elements of the MM, which were formulated into a questionnaire. Clinical sites were asked to evaluate the maturity of their processes by assigning a score from 1 to 4 to each variable of the MM questionnaire. The survey questionnaire was made available via a dedicated Internet website to 18 Italian clinical sites, selected so as to cover the entire national territory. The sites involved in the survey included public and private clinics, as well as centers at different stages of the accreditation process required for using CAR T-cell therapy. The results of the survey about the MM tool developed within the START CAR-T project were analyzed in the last phase of the project.

Results

The MM developed within the START CAR-T project to evaluate the effectiveness of CAR T-cell therapy centers was submitted to 18 Italian clinical sites. In the following paragraphs, the survey results are discussed according to the MM macro-categories.

Organization

According to the results of the survey, Italian clinical sites have gained a medium-high positioning into different variables of the organizational macro-category (Fig. 2). In particular, clinical sites, with no distinction between those that already used CAR T-cell therapy and those that were concluding the accreditation phase, formalized roles of personnel involved in therapy management. In the majority of cases (77%), a CAR T-cell therapy team, including a director, a coordinator, and a case manager, was defined. As for the presence of coordination mechanisms, 61% of clinical sites defined multidisciplinary meetings, where strategic decisions for CAR T-cell therapy and care pathways were made; only a minority of centers (22%) organized monodisciplinary meetings. In 66% of clinical sites, courses directed to the personnel involved in therapy management were established and a continuous educational program was ongoing; 11% of clinical sites had achieved a level of expertise that allowed them to provide educational courses to other Italian clinical sites. With regard to physical spaces, half of the clinical sites established a clinic for CAR T-cell therapy, where the entire care pathway or part of it took place. The remaining clinical sites had no dedicated spaces, but had a number of beds reserved for patients treated with CAR T-cell therapy.

Digital support

As shown in Figure 4, 83% of the clinical sites had an Electronic Medical Record; 55% had advanced features and mobile technologies, reflecting the increasing importance of ICT in healthcare (18). Notably, 17% of clinical sites were still using paper-based records. Support systems used for CAR T-cell therapy, like prescription and administration, allowed a digital management of these procedures (72%); in 22% of cases the digital tools supported logistic and administrative management. A clinical corporate repository collected the documents generated through the care pathway of CAR T-cell therapy entirely (39%) or partially (22%); 17% of clinical sites integrated an interoperable clinical corporate repository with digital tools. Communication and collaboration instruments used between physicians were unstructured solutions in 55% of sites, while structured and dedicated tools, like the Electronic Medical Record, were used in 39% of sites. Finally, 77% of clinical sites provided all the information about CAR T-cell therapy and follow-up to European pathology networks. By contrast, the supply of the Electronic Health Record was not planned in most clinical sites (61%); if supplied, the record contained the minimal set of relevant information (11%), or this minimal set plus a few additions (22%).

Maturity level of CAR T cell therapy centers

Based on the results of the MM application, the 18 Italian clinical sites had an average level of maturity of 7.0 (maximum level: 8.3, minimum level: 5.6, variance: 2.0), on a scale from 1 to 10 (Fig. 6). In particular, the highest scores were reached in the organization and process macro-categories, which averages respectively 7.8 (maximum level: 10.0, minimum level: 3.1, variance: 1.3) and 8.2 (maximum level: 10.0, minimum level: 4.2, variance: 2.0). These results suggest a greater attention in the definition of organizational and procedural aspects, observed in the identification of roles within the CAR T-cell therapy unit, the coordination of activities, and the definition of specialized procedures for the care pathway of patients, in particular for the postinfusion monitoring phase. Nevertheless, improvements in these macro-categories were still possible, including the creation of dedicated spaces and delivery of training courses about CAR T-cell therapy. Digital supports available at the clinical sites were limited, with an average score of 6.4 (maximum level: 9.7, minimum level: 2.5, variance: 0.2): only a proportion of centers owned an Electronic Health Record, a Clinical Data repository, or communication instruments able to support clinical management of CAR T-cell therapy and patient care pathway. Finally, even if the development of a Hub & Spoke network, with an average score of 5.8 (maximum level: 8.3, minimum level: 4.2, variance: 1.5) was recognized as crucial by all clinical sites, improvements are needed especially in the formalization of accordance and communication between centers, and the development of collaboration tools.

Conclusions

The MM developed within the START CAR-T project may support centers that use CAR T-cell therapy in monitoring and improving management aspects, and in developing greater awareness about how organizational and procedural aspects, along with digital support, are fundamental to make this novel treatment as accessible and sustainable as possible.

As specified by de Bruin et al (7), the model will evolve as the domain knowledge and model understanding broadens and deepens. The application of the START CAR-T MM over the next years will permit to measure the improvements achieved by the various clinical sites and, at the same times, will point out what changes should be made to the MM in order to maintain it up-to-date and capable of supporting clinical site growth and new healthcare challenges (e.g., COVID-19 management, increased use of CAR T-cell therapy, new regulations). A deeper analysis of Spoke sites by the START CAR-T MM may also be needed: Spoke sites will be crucial for improving patient care pathways, as some of the steps of CAR T-cell therapy (bridging chemotherapy, follow-up) could be performed at Spoke sites conveniently located for patients. In the future, Spoke sites will also be likely to alleviate the work burden on Hub sites caused by the steadily increasing number of patients who will receive CAR T-cell therapy.

Acknowledgments

The authors would like to thank the following for their collaboration in this study project: Maurizio Martelli, COU Hematology, AOU Policlinico Umberto I, Rome, Italy; Federica Sorà and Simona Sica, Department of Diagnostic Imaging, Oncological Radiotherapy and Hematology, COU Hematology and Hematopoietic stem cell transplant, Fondazione Policlinico Universitario Agostino Gemelli-IRCCS, Rome, Italy; Vincenzo Perriello and Brunangelo Falini, Institute of Hematology, Ospedale S. Maria della Misericordia, Perugia, Italy; Mauro di Ianni and Valentina De Thomasis, Department of Oncology and Hematology, Ospedale “Santo Spirito”, Pescara, Italy; Enrico Orciuolo and Francesco Caracciolo, UO Ematologia, AOU Pisana, Ospedale Santa Chiara, Pisa, Italy; Ilaria Cutini and Riccardo Saccardi, Department of Cellular Therapies and Transfusion Medicine, A.O.U Careggi, Florence, Italy; Marco Ruggeri, Cell Therapy and Hematology, San Bortolo Hospital, Vicenza, Italy; Emanuele Angelucci, Hematology and Transplant Center, IRCCS Ospedale Policlinico San Martino, Genova, Italy; Massimo Martino, Department of Hemato-Oncology and Radiotherapy, Grande Ospedale Metropolitano “Bianchi-Melacrino-Morelli”, Reggio Calabria, Italy; Maurizio Musso, Dipartimento oncologico, Unità Operativa di Oncoematologia e TMO, Ospedale La Maddalena, Palermo, Italy; Alessandro Indovina, UO Unità Trapianto di Midollo Osseo (UTMO), P.O. V. Cervello, Palermo, Italy; Marco Ladetto and Francesco Zallio, SC Ematologia, AON SANTI Antonio e Biagio e Cesare Arrigo, Alessandria, Italy; Giovanni Grillo and Roberto Cairoli, Hematology, ASST Grande Ospedale Metropolitano Niguarda, Milan, Italy; Mara Magri, Alessandro Rambaldi and Silvia Ferrari, Hematology and Bone Marrow Transplant Unit, ASST Papa Giovanni XXIII, Bergamo, Italy; Alessandro Busca and Roberto Freilone, Department of Hematology, AOU Cittá della Salute e della Scienza di Torino, Presidio Molinette, Turin, Italy; Domenico Russo, Alessandro Turra and Mirko Farina, Chair of Hematology, Bone Marrow Transplant Unit, ASST-Spedali Civili Brescia, Brescia, Italy; Armando Santoro, Humanitas University, Pieve Emanuele (Milan), IRCCS Humanitas Research Hospital, Humanitas Cancer Center, Rozzano (Milan), Italy; Consuelo Corti and Fabio Ciceri, Hematology and Bone Marrow Transplantation Unit, IRCCS San Raffaele Scientific Institute, Milan, Italy.

Editorial support was provided by Health Publishing & Services Srl and was supported by Novartis Farma, Italy.

Disclosures

Conflict of interest: SB has nothing to disclose. MGC has nothing to disclose. ADR has nothing to disclose. EF reports to have received provisions, payment/honoraria for manuscript writing, support for travel, and medical writing/documents (property of Novartis Group) from Novartis Farma. LG has nothing to disclose. PL reports to have received provisions, payment/honoraria for manuscript writing, support for travel, and medical writing/documents (property of Novartis Group) from Novartis Farma. MCT reports payment or honoraria for lectures, presentations, speakers bureaus, manuscript writing, or educational events from Gilead and to have participated on a Data Safety Monitoring Board/Advisory Board for Novartis, Gilead, and BMS, outside the submitted work.

Financial support: This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.