Towards standardization of economic evaluation research in the youth psychosocial care sector: A broad consultation in the Netherlands

  • Silvia M.A.A. Evers Department of Health Service Research, Care and Public Health Research Institute (CAPHRI), Maastricht University, Maastricht and Centre for Economic Evaluation and Machine Learning, Trimbos Institute, Netherlands Institute of Mental Health and Addiction, Utrecht - The Netherlands
  • Carmen D. Dircksen Department of Clinical Epidemiology and Medical Technology Assessment, Care and Public Health Research Institute (CAPHRI), Maastricht University Medical Center, Maastricht University, Maastricht - The Netherlands
Keywords: Adolescence, Adolescent psychiatry, Cost-benefit analysis, Expert opinions, Quality of life, Review


Introduction: Stakeholders are increasingly interested in the societal impact of psychosocial interventions in the youth sector, in terms of costs and quality of life, as well as in outcomes research. The aim of this broad consultation was to reach consensus regarding the steps to be undertaken to set a research agenda for the Netherlands Organisation for Health Research and Development (ZonMw) programme.

Methods: The broad consultation consisted of an eight-step procedure, including the conceptualization of a consultation document consisting of a scoping review of (mainly) international opinion/methodological literature and an inventory of existing Dutch guidelines and manuals for economic evaluation, a written consultation procedure among a broad range of stakeholders, and a consultation meeting with these stakeholders.

Results: In total 21 documents were included in the scoping review. A total of 24 stakeholders participated in the written consultation procedure and 14 stakeholders during the consultation meeting. The methodological issues and challenges, which were ranked in the top 5 by the stakeholders, are (i) outcome measurement, (ii) outcome identification, (iii) cost valuation, (iv) outcome valuation, and (v) time horizon/analytical approach. The existing guidelines and manuals provided guidance for some, but not all, issues and challenges.

Discussion and Conclusion: This broad consultation has contributed to a research agenda for the ZonMw programme, which will in the long run lead to the standardization of economic evaluations in this sector in the Netherlands and methodological improvement of economic evaluations in the Dutch youth sector.


Drost RMWA, Paulus ATG, Evers SMAA. Five pillars for societal perspective. Int J Technol Assess Health Care. 2020;36(2):72-74.

Drost RMWA, van der Putten IM, Ruwaard D, Evers SMAA, Paulus ATG. Conceptualizations of the societal perspective within economic evaluations: a systematic review. Int J Technol Assess Health Care. 2017;33(2):251-260.

Gold MR, Siegel JE, Russell LB, Weinstein MC. Cost-effectiveness in health and medicine. New York: Oxford University Press 1996.

Tricco AC, Lillie E, Zarin W, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467-473.

Dirksen CD, Evers SMAA. Broad consultation as part of the standardization of economic evaluation research in the youth sector. Maastricht: Maastricht University 2016.

Coghill D, Danckaerts M, Sonuga-Barke E, Sergeant J; ADHD European Guidelines Group. Practitioner review: quality of life in child mental health - conceptual challenges and practical choices. J Child Psychol Psychiatry. 2009;50(5):544-561.

Dirksen CD, Evers SMAA, Joore M. Notitie Economisch evaluatieonderzoek binnen het programma jeugd. Maastricht: academisch ziekenhuis Maastricht/Universiteit Maastricht 2007.

Dirksen CD. Kosteneffectiviteit van jeugdzorg. ZonMW, 21 April 2015.

Drost RM, Paulus AT, Ruwaard D, Evers SM. Inter-sectoral costs and benefits of mental health prevention: towards a new classification scheme. J Ment Health Policy Econ. 2013;16(4):179-186.

Drotar D. Validating measures of pediatric health status, functional status, and health-related quality of life: key methodological challenges and strategies. Ambul Pediatr. 2004;4(4):358-364.

Evers S. Kosteneffectiviteit voor interventies in de jeugdgezondheidszorg: resultaten en methodologische uitdagingen. Rotterdam: National Congres Volksgezondheid 2015.

Frick KD, Ma S. Overcoming challenges for the economic evaluation of investments in children’s health. Acad Pediatr. 2009;9(3):136-137.

Griebsch I, Coast J, Brown J. Quality-adjusted life-years lack quality in pediatric care: a critical review of published cost-utility studies in child health. Pediatrics. 2005;115(5):E600-E614.

Bouwmans CAM, Schawo SJ, Hakkaart-van Roijen L. Handleiding Vragenlijst TiC-P voor kinderen. Rotterdam: Erasmus Universiteit Rotterdam, iMTA 2012.

Homer JF, Drummond MF, French MT. Economic evaluation of adolescent addiction programs: methodologic challenges and recommendations. J Adolesc Health. 2008;43(6):529-539.

Kilian R, Losert C, Lark AL, McDaid D, Knapp M. Cost-effectiveness analysis in child and adolescent mental health problems: an updated review of literature. Int J Ment Health Promot. 2010;12:45-57.

Knapp M. Economic evaluations and interventions for children and adolescents with mental health problems. J Child Psychol Psychiatry. 1997;38(1):3-25.

Knapp M, Evans-Lacko S. Health economics. In: Thapar A, et al. Rutter’s child and adolescent psychiatry. Chichester, UK: Wiley-Blackwell 2015.

Koot HM. Challenges in child and adolescent quality of life research. Acta Paediatrica. 2002;91(3):265-266.

Matza LS, Swensen AR, Flood EM, Secnik K, Leidy NK. Assessment of health-related quality of life in children: a review of conceptual, methodological, and regulatory issues. Value Health. 2004;7(1):79-92.

Pal DK. Quality of life assessment in children: a review of conceptual and methodological issues in multidimensional health status measures. J Epidemiol Community Health. 1996;50(4):391-396.

Petrou S. Methodological issues raised by preference-based approaches to measuring the health status of children. Health Economics. 2003;12(8):697-702.

Prosser LA, Hammitt JK, Keren R. Measuring health preferences for use in cost-utility and cost-benefit analyses of interventions in children - theoretical and methodological considerations. Pharmacoeconomics. 2007;25(9):713-726.

Thorn JC, Coast J, Cohen D, et al. Resource-use measurement based on patient recall: issues and challenges for economic evaluation. Appl Health Econ Health Policy. 2013;11(3):155-161.

Ungar WJ. Challenges in health state valuation in paediatric economic evaluation are QALYs contraindicated? Pharmacoeconomics. 2011;29(8):641-652.

Wallander JL, Schmitt M, Koot HM. Quality of life measurement in children and adolescents: issues, instruments, and applications. J Clin Psychol. 2001;57(4):571-585.

Woolderink M, Lynch FL, van Asselt AD, et al. Methodological considerations in service use assessment for children and youth with mental health conditions; issues for economic evaluation. Adm Policy Ment Health. 2015;42(3):296-308.

Zavala SK, French MT, Henderson CE, Alberga L, Rowe C, Liddle HA. Guidelines and challenges for estimating the economic costs and benefits of adolescent substance abuse treatments. J Subst Abuse Treat. 2005;29(3):191-205.

Bouwmans CAM, Schawo SJ, Jansen DEMC, Vermeulen K, Reijneveld M, Hakkaart-van Roijen L. Handleiding Vragenlijst Intensieve Jeugdzorg: Zorggebruik en productieverlies. Rotterdam: Erasmus Universitiet Rotterdam 2012.

Delwel GO. Leidraad voor Uitkomstenonderzoek ‘ten behoeve van de beoordeling doelmatigheid intramurale geneesmiddelen’ Op 1 december 2008 vastgesteld en uitgebracht aan de Minister van Volksgezondheid, Welzijn en Sport. Delwel: College voor zorgverzekeringen 2008.

Drost R, Paulus ATG, Ruwaard D, Evers SMMA. Handleiding intersectorale kosten en baten van (preventieve)interventies. Maastricht: Universiteit Maastricht 2014.

Romijn G, Renes G. Algemene leidraad voor maatschappelijke kosten-batenanalyse. Den Haag: Centraal Planbureau/Planbureau voor de Leefomgeving 2013.

Pomp M, Schoemaker CG, Polder JJ. Themarapport Volksgezondheid Toekomst Verkenning (VTV). Op weg naar maatschappelijke kosten-baten analyses voor preventie en zorg. Bilthoven: Rijksinstituut voor Volksgezondheid en Milieu 2014.

Ravens-Sieberer U, Wille N, Badia X, et al. Feasibility, reliability, and validity of the EQ-5D-Y: results from a multinational study. Qual Life Res. 2010;19(6):887-897.

Wille N, Badia X, Bonsel G, et al. Development of the EQ-5D-Y: a child-friendly version of the EQ-5D. Qual Life Res. 2010;19(6):875-886.

Zorginstituut Nederland. Kostenhandleiding: Methodologie van kostenonderzoek en referentieprijzen voor economische evaluaties in de gezondheidszorg. Diemen: Zorginstituut Nederland 2015.

Zorginstituut Nederland. Richtlijn voor het uitvoeren van economische evaluaties in de gezondheidszorg. Diemen: Zorginstituut Nederland 2015.

Mierau JO, Kann-Weedage D, Hoekstra PJ, et al. Assessing quality of life in psychosocial and mental health disorders in children: a comprehensive overview and appraisal of generic health related quality of life measures. BMC Pediatr. 2020;20(1):329.

Netherlands Youth Institute.

How to Cite
Evers S, Dircksen C. Towards standardization of economic evaluation research in the youth psychosocial care sector: A broad consultation in the Netherlands. GRHTA [Internet]. 14Dec.2020 [cited 22Jan.2021];7(1):117-23. Available from:
Mental Health Economics: the Netherlands experience