James Ryan, AstraZeneca; Tanja Podkonjak, Takeda; Aikaterini Fameli, GSK; Markos Dintsios-Charalabos, Bayer; Mairead Kearney, Merck; Tilman Krueger, MSD; Veronia Calzada, Sanofi; on behalf of the EFPIA Oncology Platform and the EFPIA HTA Methods Group

Outcomes are central to health technology assessments (HTA) and the upcoming EU Joint Clinical Assessment (JCA). In addition to measuring the clinical and health benefit for patients, their measurement also determines the relative benefit of a medicine. This guides not only value considerations at a national level, but also health care system delivery and clinical practice, and patient-clinician decision making. To inform the new upcoming JCA, the Co-ordination Group on HTA (HTACG) published the ‘Guidance on outcomes for joint clinical assessments’ in June 2024¹.

As health technology developers (HTD), we are pleased to see that the guidance allows for the inclusion and assessment of outcomes requested by Member States during scoping. This is critical as different health care systems use and value outcomes differently as part of their national HTA processes. The guidance provides flexibility regarding how those outcomes should be measured and analysed allowing indirect comparisons across different trials, a requirement for the JCA due to varying clinical practice across the EU. Furthermore, for outcomes for which there is immature data, we welcome the opportunity for the HTD to demonstrate a relationship with other outcomes that are available, often referred to as surrogacy.

One of the key components of the guidance is the inclusion of patient-reported outcomes (PROs) and, where appropriate, observer-reported outcomes. These measures are critical in capturing dimensions that matter to patients (and their carers), including health-related quality of life, symptoms, and psychological, social and physical functioning. More relevant for JCA than economic based health state utility index scores, these measures represent the dimensions reported directly by the patients and carers themselves, and we look forward to seeing all Member States incorporate these outcomes into their national decision-making. While patient experience data (PED) is used to support the acceptance of the PRO measures, additional patient experience and insight data about patient preferences² and the impact of current treatments and symptoms on their daily life is not requested. With the upcoming reflection paper from the EMA on PED, we hope that the inclusion of such data is incorporated in future method guidelines.

Finally, we welcome the expansion of the guidance regarding core outcome sets for diseases. A multi-stakeholder approach, including patients and healthcare professionals, will lead to a more comprehensive and relevant outcomes at the HTA, clinical and most importantly, patient level.

There are, however, several aspects of the guidance which are worthy of further consideration.

Retaining European consistency and recognising the relevance of clinical endpoints

Unlike the other methods guidance on comparators³ and populations⁴, the outcome guidance tries to restrict the outcomes that Member States should request as part of the PICO scoping process. This inclusion in a methods guidance is contradictory given the Regulation itself says that outcomes should not be ranked⁵.

More specifically, the guidance indicates that when final outcome data, the preferred outcome noted in the guidance, is not available, surrogate or intermediate outcomes (often known as non-OS clinical endpoints by the European Medicines Agency (EMA))⁶ “may be acceptable if there is evidence of a strong association or correlation of effects on the surrogate or intermediate outcome with the effect on the long-term or final outcome.” In addition, while balancing the needs of the Member States and their right to ask for clinical endpoints, the guidance states “only surrogate outcomes for which validity has previously been clearly established should be requested where possible.”

While our experience is that a variety of clinical outcomes beyond survival are regularly requested and used by the majority of national HTA bodies, this guidance attempts to steer Member States away from asking for these. We welcome that the guidance allows clinical outcomes if requested, and their acceptability remains a Member State responsibility, but it is a missed opportunity to not explicitly include, as standard, all the endpoints used within the registrational clinical trial and by the EMA. This would ensure alignment between the two key procedures at a European level - regulatory assessment and JCA,⁷ enabling consistency with clinical guidelines and clinical practice⁸. Ironically, clinical is at the heart of the phrase ‘joint clinical assessment’, but clinical outcomes are not in this outcome methods guide².

By suggesting that the only value of clinical and intermediate outcomes can be as proven surrogates for final outcomes, the guidance risks incentivising HTDs to delay time to EMA submission versus other global regions. This would delay rather than improve access for European patients. As recommended in the EMA’s ‘Guideline on the evaluation of anticancer medicinal products in man’ it is the combination of all outcomes, including final, patient centred and clinical, as well as the magnitude of their effect, that is important for decision-making⁶. All endpoints should therefore be treated equally within a JCA, and we look forward to seeing clinical endpoints explicitly incorporated into the next outcome guidance update.

Levels of surrogacy

Where a requested outcome is not available, establishing its relationship (or surrogacy) with an available outcome is welcomed.  However, the Level 1 of evidence of surrogacy in the guidance would be unachievable for many innovative oncology medicines, ATMPs and rare diseases, which are challenged by dynamic treatment landscapes and confounding on final outcomes. Establishing such evidence may take many years, if it is even possible.

More specifically, the guidance acknowledges that there is no universally accepted threshold for the establishment of sufficient correlations between outcomes. However, it then suggests a correlation of 0.85 as ‘high’. The source of this threshold value is a 2015 study by Prasad and colleagues⁹, and while applied slightly differently, Prasad based it on IQWiG’s guidance – the value judgement of one Member State¹⁰.

While we welcome that the guidance covers different levels of surrogacy evidence, acknowledges different frameworks, and leaves the final decision to each Member State, assessors should not base level of surrogacy on this threshold as it is not universally accepted and is rarely achievable in practice. Instead, assessors should apply flexibility to assess the appropriateness of evidence within the disease and treatment context on a case-by-case basis. This approach would aid national decision-makers in a more informative and pragmatic manner where decisions on which patients can access a new medicine matter.

Disproportionate safety analyses

We note that the final guidance has now incorporated the need to provide adverse events analyses according to system organ class and preferred terms. This creates significant analysis burden for developers¹¹ and resource implications for assessors. From our experience, this level of detail is typically only requested by Germany. Recent research has shown that in Germany 77% of the analyses are not used during the assessment or appraisal process.¹² By going beyond main adverse event analyses typically requested by EMA and most HTA bodies, and adding many hundreds of additional analyses, we are concerned that this will distract from the impact and usability of the JCA report by national decision-makers. Further, there are methodological challenges: the more analyses are performed on safety, the higher the chance of statistical signals which are artefacts. Such needs should therefore be requested as complementary analyses by the Member States.

Summary

The final guidance on outcomes for JCA is a product of collaboration and experience from HTA practitioners across Europe and input from stakeholders, including developers themselves. While we discuss our key concerns above, we remain optimistic that the flexibility included in the guideline is applied by the assessors in a pragmatic approach. Looking into the future, we hope the second iteration of the guidance will incorporate our recommendations as well as pragmatic learnings from the first years of the JCA process, for the benefit of European patients.

1. European Commission, Member State Coordination Group on HTA. Guidance on outcomes for joint clinical assessments. 2024. Accessed August 20, 2024. https://health.ec.europa.eu/publications/guidance-outcomes-joint-clinical-assessments_en
2. PREFER Consortium. PREFER Recommendations - Why, when and how to assess and use patient preferences in medical product decision-making. Published online 2022. doi:10.5281/ZENODO.6592304
3. European Commission, Member State Coordination Group on HTA. Methodological Guideline for Quantitative Evidence Synthesis: Direct and Indirect Comparisons. 2024. Accessed November 6, 2024. https://health.ec.europa.eu/latest-updates/methodological-guideline-quantitative-evidence-synthesis-direct-and-indirect-comparisons-2024-03-25_en
4. European Commission, Member State Coordination Group on HTA. Guidance on reporting requirements for multiplicity issues and subgroup, sensitivity and post hoc analyses in joint clinical assessments. 2024. Accessed November 6, 2024. https://health.ec.europa.eu/publications/guidance-reporting-requirements-multiplicity-issues-and-subgroup-sensitivity-and-post-hoc-analyses_en
5. European Commission. Regulation (EU) 2021/2282 on Health Technology Assessment. 2021. Accessed August 20, 2024. https://health.ec.europa.eu/health-technology-assessment/regulation-health-technology-assessment_en
6. European Medicines Agency; Committee for Medicinal Products for Human Use (CHMP). Guideline on the evaluation of anticancer medicinal products in man. EMA/CHMP/205/95 Rev.6. Published online 2019.
7. Niehaus I, Dintsios CM. Confirmatory versus explorative endpoint analysis: Decision-making on the basis of evidence available from market authorization and early benefit assessment for oncology drugs. Health Policy. 2018;122(6):599-606. doi:10.1016/J.HEALTHPOL.2018.03.017
8. Baltes N, Icks A, Dintsios CM. Treatment response in hemato-oncology in the context of the German early benefit assessment of drugs compared to clinical practice. J Evid Based Med. 2023;16(4):451-454. doi:10.1111/JEBM.12575
9. Prasad V, Kim C, Burotto M, Vandross A. The Strength of Association Between Surrogate End Points and Survival in Oncology: A Systematic Review of Trial-Level Meta-analyses. JAMA Intern Med. 2015;175(8):1389-1398. doi:10.1001/JAMAINTERNMED.2015.2829
10. Validity of surrogate endpoints in oncology. Institute for Quality and Efficiency in Health Care: Executive Summaries. Published online November 21, 2011. Accessed November 12, 2024. https://www.ncbi.nlm.nih.gov/books/NBK198799/
11. van Engen A, Krüger R, Parnaby A, et al. The Impact of Additive PICOs in a European Joint Clinical Health Technology Assessment. Value in Health. 2024;0(0). doi:10.1016/j.jval.2024.07.024
12. Verband Forschender Arzneimittelhersteller, AMS Advanced Medical Services. New requirements for AMNOG-dossiers: Investigation of considered evaluations in the context of the benefit assessment by IQWiG and G-BA. Published online 2021.