The HTA Regulation (HTAR) requires that Joint Clinical Assessments (JCAs) describe the certainty of a health technology’s effects by considering the strengths and limitations of the available evidence. The Member States HTA Coordination Group's guidance on the Validity of Clinical Studies¹ (focused on defining, classifying, and assessing the certainty of study results leading to statistical analyses from single studies) and on Multiplicity, Subgroup, Sensitivity, and Post Hoc Analyses² (addressing how to handle multiplicity issues and complementary analyses such as specific subgroup, post hoc, and sensitivity analyses) stress that JCAs should remain descriptive at the European level to avoid overstepping on national decision-making. EFPIA welcomes that both guidance documents align with the HTAR’s requirement that JCA reports remain factual, without judgments or recommendations on the technology's clinical value, target population, or place within treatment strategies.

While rigorous scientific standards are essential for JCA development, it is also important to avoid overly complex frameworks that could hinder clarity and practical use at the national level. As health technology developers prepare their JCA dossiers, concerns persist that these guidelines might not be sufficiently tailored to diverse therapeutic needs and real-world constraints. This could create uncertainties that impede timely and consistent market access, ultimately impacting patient care.

Balancing Validity Standards with Practical Application

The guidance on the Validity of Clinical Studies promotes principles to ensure that data within JCA dossiers meets high scientific standards, emphasizing the inclusion of evidence that reflects the diverse needs of Europe’s healthcare systems while allowing flexibility in interpretation at the member state level based on national needs and healthcare settings.

The guidance highlights the need for careful handling of single-arm trials with external controls to ensure the reliability of evidence. However, by seemingly discouraging reliance on practical solutions for challenging conditions like rare diseases, the guidance risks imposing standards that are difficult to meet outside of ideal scenarios. A balanced approach would prioritize transparency about evidence limitations while recognizing contextual factors, ensuring that real-world challenges in HTA are addressed without diminishing evidence rigor.

Furthermore, while the guideline opens the door for adaptive trials and real-world evidence (RWE) to support validity, these approaches are constrained by high thresholds for scientific rigor that may be difficult to meet uniformly across therapeutic areas. In rapidly evolving fields like oncology, where there is a high demand for innovative evidence types and advanced methodologies in comparative effectiveness analyses, pharmaceutical companies may find the guidelines’ requirements overly rigid, potentially creating obstacles to timely patient access.

Similarly, the treatment-switching adjustments recommended within the guidelines offer helpful insights but fall short of providing a flexible framework that accommodates the nuanced realities of clinical practice. Striking the balance between methodological stringency and the evolving nature of clinical evidence remains a challenge.  Without greater clarity and a more adaptable framework, pharmaceutical companies risk preparing extensive submissions without a clear understanding of how well their evidence will align with diverse Member State expectations.

Statistical Complexity: Enhancing Rigor or Overburdening the Process?

The Multiplicity, Subgroup, Sensitivity, and Post Hoc Analyses guidance introduces advanced statistical standards to address key considerations in JCA dossiers. While scientific rigor is necessary, the stringent requirements for multiplicity adjustments and subgroup analyses could add significant complexity to the assessment process. Excessive amounts of additional exploratory analyses or overly rigid multiplicity adjustments may result in a dossier that, despite its volume, lacks the flexibility to support nuanced clinical decisions at the national level.

For example, subgroup analysis is highly valued, particularly in precision medicine, yet the guidance’s preference for basing the credibility of a subgroup result on a treatment-subgroup interaction test may constrain its utility. The guidance risks limiting the meaningful insights that subgroup data can provide—especially where there is a pressing need to understand differential treatment effects among subpopulations. However, the inclusion of the ICEMAN criteria for assessing the credibility of subgroup results is a positive step forward. These criteria offer a more rigorous and nuanced approach than methods previously applied by some HTA bodies, such as IQWiG, which rely heavily on the p-value of the interaction term with an unadjusted 0.05 threshold. The ICEMAN criteria promote a structured evaluation of subgroup analyses, enhancing their credibility and potential utility for decision-making in cases where understanding subgroup effects is critical.

Additionally, sensitivity and post hoc analysis requirements, while intended to bolster evidence credibility, may lead to redundancies. Pharmaceutical companies, already contending with tight timelines, may face the burden of conducting analyses that, while methodologically sound, do not necessarily contribute meaningfully within the JCA framework and may have limited impact in national HTA processes. Without clearer criteria, there is a risk that the extensive statistical work required—particularly for post hoc analyses—may not consistently deliver meaningful contributions to the EU-level assessment or decisions at the national level. The high volume of post hoc analyses potentially requested by HTA bodies often adds to the analytical burden without necessarily enhancing the dossier’s relevance or readability. Focusing on analyses strictly needed for decision-making would help ensure that dossiers remain both usable and informative. Without this focus, companies may be required to produce extensive supplementary analyses that could delay dossier preparation and, ultimately, the availability of essential therapies to patients.

Moving Toward Clarity and Collaboration

As the industry prepares to adapt to the evolving HTA landscape, we urge the HTA Coordination Group to work closely with all stakeholders to refine the methodological guidance documents in ways that address these nuanced concerns. Greater flexibility and clearer guidance would enable  the industry to meet the JCA’s high standards more effectively while delivering evidence that is both scientifically rigorous and practically valuable.

The upcoming ISPOR EU 2024 conference offers a timely platform to foster dialogue on these topics. Members of the EFPIA HTA Methods Working Group look forward to engaging with the HEOR community, aiming to shape these guidelines into a robust yet workable framework that will truly serve the needs of European patients and healthcare systems.

Authors’ Affiliations:

KK, AP, NV are employees of Bristol Myers Squibb. AK is employee of Pfizer. AM is an employee of Astellas Pharma Europe Ltd. JR is an employee of Astra Zeneca.

References:

1. Guidance on Validity of Clinical Studies, Adopted on 19 September 2024 by the HTA CG pursuant to Article 3(7), point (d), of Regulation (EU) 2021/2282 on Health Technology Assessment
2. Guidance on reporting requirements for multiplicity issues and subgroup, sensitivity and post hoc analyses in joint clinical assessments, Adopted on 10 June 2024 by the HTA CG pursuant to Article 3(7), point (d), of Regulation (EU) 2021/2282 on Health Technology Assessment