Aggregated results of valid responses from HTA implementation survey (scorecard)
| Consent | Yes | No |
|---|---|---|
| n (%) | n (%) | |
| Hereby, I accept that my anonymous answers can be aggregated and used in scientific presentations and publications | 21 (100%) | 0 (0%) |
| Consent | Yes | No |
|---|---|---|
| Hereby, I accept that my anonymous answers can be aggregated and used in scientific presentations and publications | 21 (100%) | 0 (0%) |
| Question | Current status | Preferred status |
|---|---|---|
| n (%) | n (%) | |
| 1. HTA capacity-building | ||
| a) Education | ||
| No training | 5 (23.8%) | 0 (0.0%) |
| Project-based training and short courses | 16 (76.2%) | 0 (0.0%) |
| Permanent graduate program with short courses | 0 (0.0%) | 7 (33.3%) |
| Permanent graduate and postgraduate program with short courses | 0 (0.0%) | 14 (66.7%) |
| 2. HTA funding | ||
| a) Financing critical appraisal of technology assessment | ||
| No funding for critical appraisal of technology assessment reports or submissions | 17 (85.0%) | 1 (4.8%) |
| Dominantly private funding (e.g. submission fees) by manufacturers for the critical appraisal of technology assessment reports or submissions | 3 (15.0%) | 9 (42.9%) |
| Dominantly public funding for critical appraisal of technology assessment reports or submissions | 0 (0.0%) | 11 (52.4%) |
| b) Financing health technology assessment (i.e. HTA research) | ||
| No public funding for technology assessment; private funding is not needed or expected | 17 (81.0%) | 1 (4.8%) |
| No or marginal public funding for research in HTA; private funding is expected | 4 (19.0%) | 0 (0.0%) |
| Sufficient public funding for research in HTA; private funding is also expected | 0 (0.0%) | 12 (57.1%) |
| HTA research is dominantly funded from public resources | 0 (0.0%) | 8 (38.1%) |
| 3. Legislation on HTA | ||
| a) Legislation on the role of the HTA process and recommendations in the decision-making process | ||
| No formal role of HTA in decision-making | 11 (52.4%) | 1 (4.8%) |
| Dominantly international HTA evidence is taken into account in decision-making | 9 (42.9%) | 0 (0.0%) |
| International and additionally local HTA evidence is taken into account in decision-making | 1 (4.8%) | 10 (47.6%) |
| Local HTA evidence is mandatory in decision-making | 0 (0.0%) | 10 (47.6%) |
| b) Legislation on organizational structure for HTA appraisal | ||
| There is no public committee or institute for the appraisal process | 14 (66.7%) | 1 (4.8%) |
| A committee is appointed for the appraisal process | 5 (23.8%) | 0 (0.0%) |
| The committee is appointed for the appraisal process with the support of academic centers and independent expert groups | 1 (4.8%) | 2 (9.5%) |
| A public HTA institute or agency is established to conduct a formal appraisal of HTA reports or submissions | 1 (4.8%) | 1 (4.8%) |
| Public HTA institute or agency is established to conduct a formal appraisal of HTA reports or submissions with the support of academic centers and independent expert groups | 0 (0.0%) | 8 (38.1%) |
| Several public HTA bodies are established without central coordination of their activities | 0 (0.0%) | 0 (0.0%) |
| Several public HTA bodies are established with central coordination of their activities | 0 (0.0%) | 9 (42.9%) |
| 4. Scope of HTA implementation | ||
| a) Scope of technologies (multiple choice) | ||
| HTA is not applied to any health technologies | 11 (52.4%) | 1 (4.8%) |
| Pharmaceutical products | 10 (47.6%) | 17 (81.0%) |
| Medical devices | 3 (14.3%) | 18 (85.7%) |
| Prevention programs and technologies | 2 (9.5%) | 19 (90.5%) |
| Surgical interventions | 1 (4.8%) | 17 (81.0%) |
| Other scope of technologies | 0 (0.0%) | 3 (14.3%) |
| b) Depth of HTA use in pricing and/or reimbursement decision of health technologies | ||
| HTA is not applied to any health technologies | 12 (57.1%) | 1 (4.8%) |
| Only new technologies with significant budget impact | 7 (33.3%) | 0 (0.0%) |
| Only new technologies | 1 (4.8%) | 0 (0.0%) |
| New technologies + revision of previous pricing and reimbursement decisions | 1 (4.8%) | 20 (95.2%) |
| 5. Decision criteria | ||
| a) Decision categories (multiple choice) | ||
| None of the below categories are applied | 6 (28.6%) | 0 (0.0%) |
| Unmet medical need | 4 (19.0%) | 11 (52.4%) |
| Healthcare priority | 4 (19.0%) | 12 (57.1%) |
| Assessment of therapeutic value | 5 (23.8%) | 15 (71.4%) |
| Cost-effectiveness | 8 (38.1%) | 17 (81.0%) |
| Budget impact | 7 (33.3%) | 17 (81.0%) |
| Other decision categories | 0 (0.0%) | 1 (4.8%) |
| b) Decision thresholds | ||
| Thresholds are not applied | 17 (81.0%) | 1 (4.8%) |
| Implicit thresholds are preferred | 3 (14.3%) | 3 (14.3%) |
| Explicit soft thresholds are applied in decisions | 1 (4.8%) | 12 (57.1%) |
| Explicit hard thresholds are applied in decisions | 0 (0.0%) | 5 (23.8%) |
| c) Multi-criteria decision analysis | ||
| No explicit multi criteria decision framework is applied | 20 (100.0%) | 1 (4.8%) |
| Explicit multi criteria decision framework is applied | 0 (0.0%) | 20 (95.2%) |
| 6. Quality and transparency of HTA implementation | ||
| a) Quality elements of HTA implementation (multiple choice) | ||
| None of the below quality elements are applied | 18 (85.7%) | 1 (4.8%) |
| Published methodological guidelines for HTA/economic evaluation | 2 (9.5%) | 11 (52.4%) |
| Regular follow-up research on HTA recommendations | 1 (4.8%) | 7 (33.3%) |
| A checklist to conduct a formal appraisal of HTA reports or submissions exists but not available for public | 0 (0.0%) | 9 (42.9%) |
| A published checklist is applied to conduct a formal appraisal of HTA reports or submissions | 0 (0.0%) | 16 (76.2%) |
| b) Transparency of HTA in policy decisions | ||
| Technology assessment reports, critical appraisal and HTA recommendation are not published | 20 (95.2%) | 1 (4.8%) |
| HTA recommendation is published without details of technology assessment reports and critical appraisal | 1 (4.8%) | 1 (4.8%) |
| Transparent technology assessment reports, critical appraisals and HTA recommendations | 0 (0.0%) | 19 (90.5%) |
| c) Timeliness | ||
| HTA submission and issuing recommendation have no transparent timelines | 19 (95.0%) | 0 (0.0%) |
| HTA submissions are accepted/conducted following a transparent calendar, but issuing recommendation has no transparent timelines | 1 (5.0%) | 3 (14.3%) |
| HTA submissions are accepted continuously and issuing recommendation has transparent timelines | 0 (0.0%) | 18 (85.7%) |
| 7. Use of local data | ||
| a) Requirement of using local data in technology assessment | ||
| No mandate to use local data | 14 (73.7%) | 1 (4.8%) |
| The mandate of using local data in certain categories without the need for assessing the transferability of international evidence | 4 (21.1%) | 3 (14.3%) |
| The mandate of using local data in certain categories with the need for assessing the transferability of international evidence | 1 (5.3%) | 17 (81.0%) |
| b) Access and availability of local data | ||
| Limited availability or accessibility to local real-world data | 15 (71.4%) | 0 (0.0%) |
| Up-to-date patient registries are available in certain disease areas, but payers’ databases are not accessible for HTA doers | 4 (19.0%) | 1 (4.8%) |
| Payers’ databases are accessible for HTA doers, patient registries are not available or accessible in the majority of disease areas | 0 (0.0%) | 3 (14.3%) |
| Up-to-date patient registries are available in certain disease areas and payers’ databases are accessible for HTA doers | 2 (9.5%) | 17 (81.0%) |
| 8. International collaboration | ||
| a) international collaboration, joint work on HTA (joint assessment reports) and national/regional adaptation (reuse) (multiple choice) | ||
| No involvement in joint work; and no reuse of joint work or national/regional HTA documents from other countries | 19 (100.0%) | 1 (5.0%) |
| Active involvement in joint work (e.g. Eunet HTA Rapid REA, full Core HTA) | 0 (0.0%) | 5 (25.0%) |
| National/regional adaptation (reuse) of joint HTA documents | 0 (0.0%) | 7 (35.0%) |
| National/regional adaptation (reuse) of national/regional work performed by other HTA bodies in other countries | 0 (0.0%) | 18 (90.0%) |
| b) International HTA courses for continuous education on HTA | ||
| Limited interest in (1) developing/implementing of and (2) participating at international HTA courses | 18 (100.0%) | 1 (4.8%) |
| Interest only in regular participation at international HTA courses | 0 (0.0%) | 2 (9.5%) |
| High interest in (1) developing/implementing of and (2) participating at international HTA courses | 0 (0.0%) | 18 (85.7%) |
| Question | Current status | Preferred status |
|---|---|---|
| No training | 5 (23.8%) | 0 (0.0%) |
| Project-based training and short courses | 16 (76.2%) | 0 (0.0%) |
| Permanent graduate program with short courses | 0 (0.0%) | 7 (33.3%) |
| Permanent graduate and postgraduate program with short courses | 0 (0.0%) | 14 (66.7%) |
| No funding for critical appraisal of technology assessment reports or submissions | 17 (85.0%) | 1 (4.8%) |
| Dominantly private funding (e.g. submission fees) by manufacturers for the critical appraisal of technology assessment reports or submissions | 3 (15.0%) | 9 (42.9%) |
| Dominantly public funding for critical appraisal of technology assessment reports or submissions | 0 (0.0%) | 11 (52.4%) |
| No public funding for technology assessment; private funding is not needed or expected | 17 (81.0%) | 1 (4.8%) |
| No or marginal public funding for research in HTA; private funding is expected | 4 (19.0%) | 0 (0.0%) |
| Sufficient public funding for research in HTA; private funding is also expected | 0 (0.0%) | 12 (57.1%) |
| HTA research is dominantly funded from public resources | 0 (0.0%) | 8 (38.1%) |
| No formal role of HTA in decision-making | 11 (52.4%) | 1 (4.8%) |
| Dominantly international HTA evidence is taken into account in decision-making | 9 (42.9%) | 0 (0.0%) |
| International and additionally local HTA evidence is taken into account in decision-making | 1 (4.8%) | 10 (47.6%) |
| Local HTA evidence is mandatory in decision-making | 0 (0.0%) | 10 (47.6%) |
| There is no public committee or institute for the appraisal process | 14 (66.7%) | 1 (4.8%) |
| A committee is appointed for the appraisal process | 5 (23.8%) | 0 (0.0%) |
| The committee is appointed for the appraisal process with the support of academic centers and independent expert groups | 1 (4.8%) | 2 (9.5%) |
| A public HTA institute or agency is established to conduct a formal appraisal of HTA reports or submissions | 1 (4.8%) | 1 (4.8%) |
| Public HTA institute or agency is established to conduct a formal appraisal of HTA reports or submissions with the support of academic centers and independent expert groups | 0 (0.0%) | 8 (38.1%) |
| Several public HTA bodies are established without central coordination of their activities | 0 (0.0%) | 0 (0.0%) |
| Several public HTA bodies are established with central coordination of their activities | 0 (0.0%) | 9 (42.9%) |
| HTA is not applied to any health technologies | 11 (52.4%) | 1 (4.8%) |
| Pharmaceutical products | 10 (47.6%) | 17 (81.0%) |
| Medical devices | 3 (14.3%) | 18 (85.7%) |
| Prevention programs and technologies | 2 (9.5%) | 19 (90.5%) |
| Surgical interventions | 1 (4.8%) | 17 (81.0%) |
| Other scope of technologies | 0 (0.0%) | 3 (14.3%) |
| HTA is not applied to any health technologies | 12 (57.1%) | 1 (4.8%) |
| Only new technologies with significant budget impact | 7 (33.3%) | 0 (0.0%) |
| Only new technologies | 1 (4.8%) | 0 (0.0%) |
| New technologies + revision of previous pricing and reimbursement decisions | 1 (4.8%) | 20 (95.2%) |
| None of the below categories are applied | 6 (28.6%) | 0 (0.0%) |
| Unmet medical need | 4 (19.0%) | 11 (52.4%) |
| Healthcare priority | 4 (19.0%) | 12 (57.1%) |
| Assessment of therapeutic value | 5 (23.8%) | 15 (71.4%) |
| Cost-effectiveness | 8 (38.1%) | 17 (81.0%) |
| Budget impact | 7 (33.3%) | 17 (81.0%) |
| Other decision categories | 0 (0.0%) | 1 (4.8%) |
| Thresholds are not applied | 17 (81.0%) | 1 (4.8%) |
| Implicit thresholds are preferred | 3 (14.3%) | 3 (14.3%) |
| Explicit soft thresholds are applied in decisions | 1 (4.8%) | 12 (57.1%) |
| Explicit hard thresholds are applied in decisions | 0 (0.0%) | 5 (23.8%) |
| No explicit multi criteria decision framework is applied | 20 (100.0%) | 1 (4.8%) |
| Explicit multi criteria decision framework is applied | 0 (0.0%) | 20 (95.2%) |
| None of the below quality elements are applied | 18 (85.7%) | 1 (4.8%) |
| Published methodological guidelines for HTA/economic evaluation | 2 (9.5%) | 11 (52.4%) |
| Regular follow-up research on HTA recommendations | 1 (4.8%) | 7 (33.3%) |
| A checklist to conduct a formal appraisal of HTA reports or submissions exists but not available for public | 0 (0.0%) | 9 (42.9%) |
| A published checklist is applied to conduct a formal appraisal of HTA reports or submissions | 0 (0.0%) | 16 (76.2%) |
| Technology assessment reports, critical appraisal and HTA recommendation are not published | 20 (95.2%) | 1 (4.8%) |
| HTA recommendation is published without details of technology assessment reports and critical appraisal | 1 (4.8%) | 1 (4.8%) |
| Transparent technology assessment reports, critical appraisals and HTA recommendations | 0 (0.0%) | 19 (90.5%) |
| HTA submission and issuing recommendation have no transparent timelines | 19 (95.0%) | 0 (0.0%) |
| HTA submissions are accepted/conducted following a transparent calendar, but issuing recommendation has no transparent timelines | 1 (5.0%) | 3 (14.3%) |
| HTA submissions are accepted continuously and issuing recommendation has transparent timelines | 0 (0.0%) | 18 (85.7%) |
| No mandate to use local data | 14 (73.7%) | 1 (4.8%) |
| The mandate of using local data in certain categories without the need for assessing the transferability of international evidence | 4 (21.1%) | 3 (14.3%) |
| The mandate of using local data in certain categories with the need for assessing the transferability of international evidence | 1 (5.3%) | 17 (81.0%) |
| Limited availability or accessibility to local real-world data | 15 (71.4%) | 0 (0.0%) |
| Up-to-date patient registries are available in certain disease areas, but payers’ databases are not accessible for HTA doers | 4 (19.0%) | 1 (4.8%) |
| Payers’ databases are accessible for HTA doers, patient registries are not available or accessible in the majority of disease areas | 0 (0.0%) | 3 (14.3%) |
| Up-to-date patient registries are available in certain disease areas and payers’ databases are accessible for HTA doers | 2 (9.5%) | 17 (81.0%) |
| No involvement in joint work; and no reuse of joint work or national/regional HTA documents from other countries | 19 (100.0%) | 1 (5.0%) |
| Active involvement in joint work (e.g. Eunet HTA Rapid REA, full Core HTA) | 0 (0.0%) | 5 (25.0%) |
| National/regional adaptation (reuse) of joint HTA documents | 0 (0.0%) | 7 (35.0%) |
| National/regional adaptation (reuse) of national/regional work performed by other HTA bodies in other countries | 0 (0.0%) | 18 (90.0%) |
| Limited interest in (1) developing/implementing of and (2) participating at international HTA courses | 18 (100.0%) | 1 (4.8%) |
| Interest only in regular participation at international HTA courses | 0 (0.0%) | 2 (9.5%) |
| High interest in (1) developing/implementing of and (2) participating at international HTA courses | 0 (0.0%) | 18 (85.7%) |
Note(s): For single choice questions, each expert chose 1 of the available options for the current status and 1 of the options for preferred status. E.g. for question 1a: an expert chose “No training” in the current status and “Permanent graduate program with short courses” for the preferred status, this means he thinks there are currently no training programs, and he would prefer that in 10 years, there will be permanent graduate programs with short courses
For multiple choice questions, each expert can choose more than one option for the current and preferred status. E.g. for question 4a: an expert chose “pharmaceuticals” and “medical devices” in the current status and “pharmaceuticals”, “medical devices”, “prevention programs” and “surgical interventions” for the preferred status. This means that currently he thinks that HTA is done for medical devices and pharmaceuticals and in the future, he prefers HTA to be done to surgical interventions and prevention programs as well
Source(s): Authors work