Advertisement

Applying Multi-Criteria Decision Analysis (MCDA) Simple Scoring as an Evidence-based HTA Methodology for Evaluating Off-Patent Pharmaceuticals (OPPs) in Emerging Markets

Open ArchivePublished:May 11, 2017DOI:https://doi.org/10.1016/j.vhri.2017.02.001

      Abstract

      Off-patent pharmaceuticals (OPPs) represent more than 60% of the pharmaceutical market in many emerging countries, where they are frequently evaluated primarily on cost rather than with health technology assessment. OPPs are assumed to be identical to the originators. Branded and unbranded generic versions can, however, vary from the originator in active pharmaceutical ingredients, dosage, consistency formulation, excipients, manufacturing processes, and distribution, for example. These variables can alter the efficacy and safety of the product, negatively impacting both the anticipated cost savings and the population’s health. In addition, many health care systems lack the resources or expertise to evaluate such products, and current assessment methods can be complex and difficult to adapt to a health system’s needs. Multicriteria decision analysis (MCDA) simple scoring is an evidence-based health technology assessment methodology for evaluating OPPs, especially in emerging countries in which resources are limited but decision makers still must balance affordability with factors such as drug safety, level interchangeability, manufacturing site and active pharmaceutical ingredient quality, supply track record, and real-life outcomes. MCDA simple scoring can be applied to pharmaceutical pricing, reimbursement, formulary listing, and drug procurement. In November 2015, a workshop was held at the International Society for Pharmacoeconomics and Outcomes Research Annual Meeting in Milan to refine and prioritize criteria that can be used in MCDA simple scoring for OPPs, resulting in an example MCDA process and 22 prioritized criteria that health care systems in emerging countries can easily adapt to their own decision-making processes.

      Keywords

      Introduction

      Delivering effective, universal, and efficient health care is an important policy goal in every country in the world, whether that country is developed or emerging. The fundamental difference between emerging markets and developed markets is a matter of implementation of good manufacturing practice (GMP) standards and bioequivalence. In developed markets, GMP standards have been fully implemented in parallel with the acceptance of bioequivalence, where a ±20% variance is accepted as a standard definition of generic products. Nevertheless, most emerging countries do not yet fully implement bioequivalence or even pharmaceutical equivalence, in which two products have the same active ingredient at the same dose. Thus, different policies for health technology assessment (HTA) need to be considered to make value-based decisions in generic purchasing. This article looks at the current need for such assessment and the use of multicriteria decision analysis (MCDA) simple scoring as a potential solution.
      Because many emerging countries are heading toward universal coverage, quality and affordable off-patent pharmaceuticals (OPPs), which include international nonproprietary name generics, branded generics, and off-patent originators, have a critical role in maximizing the value to the public health system through improved reliability. At the same time, manufacturers who supply drugs of increased quality and quantity should be considered for incrementally similar increased reimbursement. Especially in emerging countries, a value-based HTA for patients treated by OPPs can significantly contribute to improved population health outcomes. Many health systems implement HTA to evaluate patented pharmaceuticals; nevertheless, HTA methodology is seldom applied to OPPs. This lack of HTA methodology for OPPs becomes especially concerning in emerging countries, in which OPPs are used to treat most patients (more than 60%) [].
      Historically, the fundamental assumption of lowest-price policy decisions for OPPs in emerging countries is based on the premise that all OPPs are the same. Given the critical role OPPs play in providing and retaining coverage for populations in emerging countries, this reliance on the assumption, although excluding other criteria such as product quality (i.e., GMP), stringent bioequivalent criteria, value in use (persistence and adherence), clinical outcomes, and additional nondrug costs, may prove to be an inadequate method for providing adequate health care [
      • Bate R.
      • Jin G.Z.
      • Mathur A.
      Does price reveal poor-quality drugs? Evidence from 17 countries.
      ,
      • Kaló Z.
      • Holtorf A.-P.
      • Alfonso-Cristancho R.
      • et al.
      Need for multicriteria evaluation of generic drug policies.
      ]. Therefore, the International Outcomes Research Board, a group of academia and industry experts, has undertaken an initiative to develop an evidence-based HTA methodology for off-patent products and has conducted significant work in this area at both the theoretical and the practical implementation level in emerging countries. From their work, MCDA simple scoring is emerging as a useful approach that can be applied to pharmaceutical pricing, reimbursement, formulary listing, and drug procurement. This method can be adapted easily to suit specific characteristics of individual health care systems, of particular interest to those searching for a sound methodology to evaluate OPPs. In November 2015, a workshop was held at the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) Annual Meeting in Milan to refine and prioritize criteria that can be used in MCDA simple scoring for OPPs, resulting in an example MCDA process and tool that health care systems in emerging countries can adapt to their own decision-making processes.

      Why Lowest-Price Policy Objectives Fall Short

      In lowest-price–driven policies, the assumption that “OPPs are the same” is quite common, especially in developing economies. The off-patent originator has already gone through extensive testing and assessment, and it is presumed that the generic version of that medication will provide the same benefit for lower cost [
      • Kaló Z.
      • Holtorf A.-P.
      • Alfonso-Cristancho R.
      • et al.
      Need for multicriteria evaluation of generic drug policies.
      ,
      • Tawde S.A.
      Generic pharmaceuticals: Is pharmacovigilance required?.
      ,
      • Borgherini G.
      The bioequivalence and therapeutic efficacy of generic versus brand-name psychoactive drugs.
      ]. In a health care system that has limited resources but a desire to provide vital medication to as much of its population as it can, basing decisions on drug acquisition cost can seem like a practical way to get as much “bang for the buck” as possible.
      Unfortunately, the fundamental presumption behind this thinking—that the branded and unbranded generic versions are identical to the originator in active pharmaceutical ingredients (APIs), dosage, consistency formulation, excipients, manufacturing processes, and distribution, for example—is not always the case. These additional variables can greatly affect the efficacy and safety of the product, negatively impacting not just the anticipated cost savings but also the health of the country’s population. Table 1 presents the current status of GMP, pharmaceutical equivalence, and bioequivalence categories considered in generic policy decisions across 14 different emerging countries.
      Table 1Parameters on pharmaceutical equivalence and bioequivalence in emerging countries
      CountryGeneric definitionIn line with international standardRequired GMP meet WHO standard (manufacturer sites)GMP standard implementedNew registration require BE (local production)Mandate BE for in-market local productsIn vitro test performed in routine quality check (local products)
      RussiaYesYesYes (2018)PartiallyYesPartially
      Products >20 y exempted.
      No
      ChinaYesNoYesPartiallyYesPartially
      Started in 2016, 289 molecules by 2018.
      Random
      VietnamYesNoYesPartiallyNo (until 2025)Partially
      Required for 12 molecules.
      Random
      Blacklist if identified.
      IndonesiaYesNoYesPartiallyPartially
      Required for 90 molecules and extended release.
      Partially
      Required for 90 molecules and extended release.
      No
      PhilippinesYesYesYesPartiallyYesPartially
      At product renewal.
      No
      PakistanYesNoYesPartiallyNoNoNo
      EgyptYesNoYesPartiallyYesNoNo
      AlgeriaYesNoYesPartiallyYesNoNo
      IndiaYesYesYesPartiallyYesNoPartially
      ChileYesYesYesPartiallyYesPartially
      At product renewal.
      Partially
      PeruYesYesYesPartiallyNoNoPartially
      Saudi ArabiaYesYesYesPartiallyYesYesYes
      South AfricaYesYesYesPartiallyYesYesYes
      ColombiaYesYesYesYesPartiallyNoPartially
      BE, bioequivalence; GMP, good manufacturing practice; WHO, World Health Organization.
      low asterisk Products >20 y exempted.
      Started in 2016, 289 molecules by 2018.
      Required for 12 molecules.
      § Blacklist if identified.
      || Required for 90 molecules and extended release.
      At product renewal.
      Even when pharmaceutical equivalence and bioequivalence have been achieved, evidence for their therapeutic evidence may be limited [
      • Kaló Z.
      • Holtorf A.-P.
      • Alfonso-Cristancho R.
      • et al.
      Need for multicriteria evaluation of generic drug policies.
      ,
      • Borgherini G.
      The bioequivalence and therapeutic efficacy of generic versus brand-name psychoactive drugs.
      ], and few emerging countries explicitly require bioequivalence and bioavailability studies [
      • Alfonso-Cristancho R.
      • Andia T.
      • Barbosa T.
      • Watanabe J.H.
      Definition and classification of generic drugs across the world.
      ]. In addition, there is a range of bioequivalence, and so one generic drug may actually be closer in bioequivalence to the originator than a second generic drug [
      • Kaló Z.
      • Holtorf A.-P.
      • Alfonso-Cristancho R.
      • et al.
      Need for multicriteria evaluation of generic drug policies.
      ], but if cost is the only criterion considered, the drug that is the “closer match” to the original may be denied. Another factor that may affect the efficacy and safety of generic medications is the type of excipients used, which, although considered inactive substances with no effect on drug action, can sometimes have an effect on drug stability, adverse reactions, or how the active drug is dissolved and absorbed into the body’s systems [
      • Tawde S.A.
      Generic pharmaceuticals: Is pharmacovigilance required?.
      ,
      • Corbert B.
      ]. Biosimilars may have minor structural differences that may be connected to immunogenicity-related adverse effects [
      • Tawde S.A.
      Generic pharmaceuticals: Is pharmacovigilance required?.
      ]. A recent publication further discussed the challenges associated with various definitions around generic pharmaceuticals [
      • Alfonso-Cristancho R.
      • Andia T.
      • Barbosa T.
      • Watanabe J.H.
      Definition and classification of generic drugs across the world.
      ].
      Manufacturing processes can introduce significant variables within OPPs. Investigations into pharmaceutical manufacturing processes across the globe reveal ingredient inconsistencies, sanitation and cross-contamination concerns, misrepresentations, poor storage conditions, and other potentially serious problems with the manufacture and delivery of branded/unbranded generic drugs [
      Institute of Medicine
      Countering the Problem of Falsified and Substandard Drugs.
      ]. Recent recalls in the United States involved issues with container closure systems, foreign particulate matter within injectable medications, and glass delamination within containers/closure systems, as well as contamination issues from compounding pharmacies [
      • Tawde S.A.
      Generic pharmaceuticals: Is pharmacovigilance required?.
      ,
      • Bate R.
      • Jin G.Z.
      • Mathur A.
      • Attaran A.
      Poor quality drugs and global trade: a pilot study.
      ,
      • Gudeman J.
      • Jozwiakowski M.
      • Chollet J.
      • Randell M.
      Potential risks of pharmacy compounding.
      ]. Additional problems include the inability to verify a product’s or ingredient’s source and to confirm proper and sanitary handling throughout the chain of custody [
      • Ventola C.L.
      The drug shortage crisis in the United States: causes, impact, and management strategies.
      ,
      • Fox E.
      • Birt A.
      • James K.
      • et al.
      ASHP guidelines on managing drug product shortages in hospitals and health systems.
      ]. These problems are especially exacerbated during drug shortages, when manufacturers and suppliers must quickly seek alternatives to fulfill demand.
      Clearly, if cost is the highest priority when making decisions on drug policy, the health care system in question may be opening itself to other concerns that could be detrimental to its overall goals of widespread health care at reasonable cost. This is precisely why HTA is used in developing countries to assess the value of OPPs. HTA allows decision makers to evaluate a drug’s value on more criteria than just cost. HTA is, however, seldom used for OPPs, and therefore seldom used by policymakers in emerging countries, where OPPs make up the bulk of their treatment base. Even if HTA methods can help policymakers balance drug acquisition costs with benefits, most emerging countries have limited experience in using such methods.

      Could MCDA Simple Scoring Be a Solution?

      MCDA is a decision-making process in which a set of criteria are defined, ranked in terms of relevance and importance, and evaluated consistently. After a set of criteria have been defined and ranked by stakeholders and decision makers, each alternative is evaluated against the same set of criteria, creating an assessment of value for each alternative. Because it emphasizes relevant criteria and provides a consistent decision basis, MCDA increases the consistency, transparency, and legitimacy of health care decisions over other methods, which are often overly simplified, created ad hoc, focused on only a single facet or goal of health care such as cost-effectiveness, or swayed by political or special-interest motives [
      • Thokala P.
      • Devlin N.
      • Marsh K.
      • et al.
      Multiple criteria decision analysis for health care decision making—an introduction: report 1 of the ISPOR MCDA Emerging Good Practices Task Force.
      ,
      • Marsh K.
      • Lanitis T.
      • Neasham D.
      • et al.
      Assessing the value of healthcare interventions using multi-criteria decision analysis: a review of the literature.
      ,
      • Baltussen R.
      • Niessen L.
      Priority setting of health interventions: the need for multi-criteria decision analysis.
      ]. The implementation of MCDA in health care has recently been growing, with the number of publications addressing MCDA usage in health care decision making ballooning from about a dozen in 2000 to more than 60 in 2011 [
      • Diaby V.
      • Campbell K.
      • Goeree R.
      Multi-criteria decision analysis (MCDA) in health care: a bibliometric analysis.
      ].
      MCDA benefits decision makers by providing a consistent framework for decisions, encouraging the inclusion of various stakeholders from different perspectives, presenting data in a consistent and digestible format, and allowing for the comparison of trade-offs between alternatives. MCDA benefits manufacturers by pointing out data gaps in manufacturers’ research, helping manufacturers simplify and focus communications with decision makers on relevant information, and establishing a common language for discussions with policymakers, regulators, and other interested stakeholders. Because of these demonstrable benefits, decision makers indicate a positive attitude toward MCDA’s potential to improve decision making [
      • Marsh K.
      • Lanitis T.
      • Neasham D.
      • et al.
      Assessing the value of healthcare interventions using multi-criteria decision analysis: a review of the literature.
      ].
      Nevertheless, if an MCDA approach is not carefully designed and applied, it is just as likely to fall prey to inconsistencies and biases as any other method. To complicate matters, various MCDA methodologies exist, all based on different schools of thought and applicable to a wide variety of situations [
      • Thokala P.
      • Devlin N.
      • Marsh K.
      • et al.
      Multiple criteria decision analysis for health care decision making—an introduction: report 1 of the ISPOR MCDA Emerging Good Practices Task Force.
      ,
      • Marsh K.
      • Lanitis T.
      • Neasham D.
      • et al.
      Assessing the value of healthcare interventions using multi-criteria decision analysis: a review of the literature.
      ]. Therefore, methodological guidance for designing, conducting, and implementing MCDA is essential. More than just a listing of criteria to consider, a solid MCDA approach must establish a clear process that transparently extracts and accounts for the needs of all involved stakeholders [
      • Angelis A.
      • Kanavos P.
      Value-based assessment of new medical technologies: towards a robust methodological framework for the application of multiple criteria decision analysis in the context of health technology assessment.
      ]. In 2014, to help decision makers wisely select and implement MCDA methods for their particular needs, the ISPOR established the MCDA Emerging Good Practices Task Force to develop good practice guidelines for implementing MCDA in health care decision making [
      • Thokala P.
      • Devlin N.
      • Marsh K.
      • et al.
      Multiple criteria decision analysis for health care decision making—an introduction: report 1 of the ISPOR MCDA Emerging Good Practices Task Force.
      ,
      • Marsh K.
      • IJzerman M.
      • Thokala P.
      • et al.
      Multiple criteria decision analysis for health care decision making—emerging good practices: report 2 of the ISPOR MCDA Emerging Good Practices Task Force.
      ].
      The task force’s guidelines identify example types of health care decisions in which MCDA can be used, including benefit-risk assessment, HTA, portfolio decision analysis, commissioning decisions/priority setting frameworks, shared decision making, and prioritization of patients’ access to health care. The guidelines also describe the steps involved in conducting an MCDA: defining the decision problem, selecting and structuring criteria, measuring performance, scoring alternatives, weighting criteria, calculating aggregate scores, dealing with uncertainty, and reporting and examining findings [
      • Thokala P.
      • Devlin N.
      • Marsh K.
      • et al.
      Multiple criteria decision analysis for health care decision making—an introduction: report 1 of the ISPOR MCDA Emerging Good Practices Task Force.
      ].
      Even with these guidelines, however, many MCDA methods are considered by some to be overly complex for local decision makers to use, especially those without experience, and can present decision makers with overwhelming amounts of data, or may be too context-specific for decision makers to see how it can apply to their situation [
      • Marsh K.
      • Lanitis T.
      • Neasham D.
      • et al.
      Assessing the value of healthcare interventions using multi-criteria decision analysis: a review of the literature.
      ]. Therefore, the development of a “simple scoring” tool may be an important aid, especially in emerging countries. In 2014, Venhorst et al. [
      • Venhorst K.
      • Zelle S.G.
      • Tromp N.
      • Lauer J.A.
      Multi-criteria decision analysis of breast cancer control in low- and middle- income countries: development of a rating tool for policy makers.
      ] published results of an effort to establish a simple MCDA rating tool for prioritizing breast cancer interventions in low- and middle-income countries. The project used a Delphi study and questionnaires to produce 10 criteria with clear definitions and potential scoring scales for evaluating breast cancer interventions. Following a similar path, the International Outcomes Research Board has spent 2 years designing an MCDA simple scoring methodology that can be adapted for use in various environments to evaluate OPPs. This methodology supports decision making in the broad areas of pricing, reimbursement, formulary listing, drug procurement, prescription, authorization, and research interests, but additional areas of interest are also being investigated. MCDA simple scoring is unique among other MCDA methodologies because a complex set of research principles is translated into a practical set of criteria to weight characteristics important to relevant health policy decisions for emerging markets.

      Tailoring MCDA Simple Scoring for Emerging Markets—A Workshop

      At the ISPOR Annual Meeting in Milan, on November 9, 2015, a workshop was held to elicit preferred criteria that health care systems in emerging countries could use in an MCDA simple scoring format to evaluate medications and determine coverage. The workshop involved 57 health care experts and decision makers from more than a dozen countries, including China, Russia, Egypt, Thailand, Turkey, Indonesia, Kazakhstan, Vietnam, the Philippines, Colombia, Taiwan, Malaysia, Algeria, and Saudi Arabia. The participants represented several professional realms, including academia, ministries of health, drug procurement agencies, and drug pricing and reimbursement agencies.
      Workshop participants were presented with a list of 22 criteria that potentially could be used to evaluate and make policy decisions about medications in a given population. The 22 criteria were originally identified through a literature review that identified publications reporting on the application of MCDA in different decision-making settings. This literature identified several studies in which MCDA was used for priority setting and reimbursement of medical technologies in countries such as Thailand, Hungary, Norway, and Canada [
      • Tony M.
      • Wagner M.
      • Khoury H.
      • et al.
      Bridging health technology assessment (HTA) with multicriteria decision analyses (MCDA): field testing of the EVIDEM framework for coverage decisions by a public payer in Canada.
      ,
      • Thokala P.
      • Duenas A.
      Multiple criteria decision analysis for health technology assessment.
      ,
      • Endrei D.
      • Molics B.
      • Agoston I.
      Multicriteria decision analysis in the reimbursement of new medical technologies: real-world experiences from Hungary.
      ,
      • Mohara A.
      • Youngkong S.
      • Velasco R.P.
      • et al.
      Using health technology assessment for informing coverage decisions in Thailand.
      ,
      • Youngkong S.
      • Baltussen R.
      • Tantivess S.
      • et al.
      Multicriteria decision analysis for including health interventions in the universal health coverage benefit package in Thailand.
      ,
      • Defechereux T.
      • Paolucci F.
      • Mirelman A.
      • et al.
      Health care priority setting in Norway: a multicriteria decision analysis.
      ,
      • Youngkong S.
      • Teerawattananon Y.
      • Tantivess S.
      • Baltussen R.
      Multi-criteria decision analysis for setting priorities on HIV/AIDS interventions in Thailand.
      ,
      • Goetghebeur M.M.
      • Wagner M.
      • Khoury H.
      • et al.
      Bridging health technology assessment (HTA) and efficient health care decision making with multicriteria decision analysis (MCDA): applying the EVIDEM framework to medicines appraisal.
      ]. The goal was to have a broad and comprehensive number of criteria, capturing elements that may be important in different settings, and then present them to emerging market policy stakeholders for final selection. Because there were 22 potential criteria, application of decompositional methodology, which considers stakeholders’ preferences for whole alternatives (such as discrete choice experiment [
      • Hauber A.B.
      • González J.M.
      • Groothuis-Oudshoorn C.G.
      • et al.
      Statistical methods for the analysis of discrete choice experiments: a report of the ISPOR Conjoint Analysis Good Research Practices Task Force.
      ]) would have been too complex. During the workshop, the criteria were presented and discussed, and then participants used an electronic data response system to rank the criteria. Less precision was required for ranking the criteria than for valuing them, and so a simple scoring methodology was selected to rank the importance of each criterion on a numeric scale from 5 (significantly important) to 1 (not important) (see Fig. 1). These criteria were then synthesized to minimize overlap and grouped into four domains: product, manufacturer, service, and cost-effectiveness (economics). The simplicity of this approach minimized the cognitive burden posed to survey participants [
      • Marsh K.
      • IJzerman M.
      • Thokala P.
      • et al.
      Multiple criteria decision analysis for health care decision making—emerging good practices: report 2 of the ISPOR MCDA Emerging Good Practices Task Force.
      ]. Nevertheless, in the future, when precise valuation would be necessary (e.g., for pricing decisions related to OPPs in a given country), more complex methodology for eliciting stakeholders’ preferences might be necessary.
      Fig. 1
      Fig. 1Mean ranking of 22 criteria in MCDA simple scoring. API, active pharmaceutical ingredient; MCDA, multicriteria decision analysis.
      When the results were tallied, the top three criteria regardless of category were drug safety, manufacturing site quality certification, and quality assurance of APIs and production process. Fourteen criteria (as highlighted in Fig. 1) were considered the most important criteria. In the product category, the top three criteria were bioequivalence, level of interchangeability, and pharmaceutical equivalence. In the manufacturer category, the top three criteria were manufacturing site quality certification, quality assurance of API and production process, and supply track record. In the service category, the top three criteria were pharmacovigilance, contribution to national health care priorities, and distribution practices. Finally, in the economic category, the top three criteria were drug safety, clinical efficacy and effectiveness, and direct costs (see Table 2).
      Table 2MCDA simple scoring criteria ranked by importance within each category
      Priority rankingProduct categoryManufacturer categoryService categoryEconomic category
      1 (more important)BioequivalenceManufacturing site quality certificationPharmacovigilanceDrug safety
      2Level of interchangeabilityQuality assurance of APIs and production processContribution to national health care prioritiesClinical efficacy and effectiveness
      3Pharmaceutical equivalenceSupply track recordDistribution practicesDirect costs
      4Indication, formulation, and strengthLocal investmentTechnical assistanceReal-life patient outcomes
      5Excipients, process technology, and delivery systemHistory and sustainability of companyDisease awareness and educationIndirect costs
      6 (less important)Order of entryContinued medical education
      API, active pharmaceutical ingredient.

      Discussion

      To move from a lowest-price policy to a value-based price evaluation through MCDA, decisions makers, especially those in emerging countries, need to understand how to identify and prioritize the criteria that matter most in their population. As workshop participants ranked these 22 criteria, they had the opportunity to consider how these criteria applied to their own unique situations.
      A list of prioritized criteria is an important tool for decision makers. A next step may be to review MCDA evaluations that have been conducted in emerging countries to identify and document additional best practices (building on the ISPOR MCDA Emerging Good Practices Task Force reports [
      • Thokala P.
      • Devlin N.
      • Marsh K.
      • et al.
      Multiple criteria decision analysis for health care decision making—an introduction: report 1 of the ISPOR MCDA Emerging Good Practices Task Force.
      ,
      • Marsh K.
      • IJzerman M.
      • Thokala P.
      • et al.
      Multiple criteria decision analysis for health care decision making—emerging good practices: report 2 of the ISPOR MCDA Emerging Good Practices Task Force.
      ]) so that MCDA simple scoring can be adapted and implemented in many different countries and health systems. For example, in 2015, Hu et al. [
      • Hu S.
      • Zhang Y.
      • He J.
      • et al.
      A case study of pharmaceutical pricing in China: setting the price for off-patent originators.
      ] used MCDA to define a value-based approach to pricing and reimbursement for off-patent originators in China. They used a systematic analysis of current pricing and reimbursement policies and combined a drug price policy review with a quantitative analysis of China’s drug purchasing database. Then the authors used an MCDA approach to interview academic experts and industry stakeholders to identify policy preferences. After identifying 10 differentiating value attributes, they used MCDA to test the impact of three pricing scenarios. Pharmaceutical equivalence, bioequivalence, GMP accreditation of a pharmaceutical company, clinical efficacy and effectiveness, drug safety, patient adherence, drug excipients, process, technology, supply reliability, and manufacturer investment are all key value attributes in their study, which are similar to the criteria in this study.
      The workshop led to a realization that emerging countries that recognize the differences among OPPs are willing to consider value-based MCDA simple scoring in addition to drug acquisition costs. Several important differences between countries were noted, such as availability of data from the manufacturer for the recommended criteria and how these criteria would be scored and then applied to decision making. Specific next steps that were suggested by participants included confirmation of criteria, the development of an MCDA simple scoring framework, and application to decision making by country. Individual country workshops were proposed to accomplish these objectives. Such exercises will further inform the future application of MCDA simple scoring to policy decisions around access and purchase of OPP products within these countries.

      Conclusions

      Emerging countries are in a unique stage of health care system development. A primary goal is to expand access to health care by universal coverage. OPPs play a critical role in the public good; they account for more than 60% of medication access in emerging countries []. Nevertheless, if lowest-price policy is the primary criterion when making drug coverage decisions, then the reasons behind that low price may warrant concern, especially in emerging countries. But the machinations of factoring in a wide-ranging variety of other criteria can be complicated and difficult to navigate without some structure and guidelines.
      Thus, MCDA simple scoring was developed as an HTA methodology to evaluate OPPs, especially in emerging countries in which resources are limited but decision makers still must balance affordability with such disparate factors as drug safety, level interchangeability, manufacturing site quality certification and quality assurance of APIs, clinical efficacy and effectiveness, supply track record, and real-life outcomes. MCDA simple scoring provides an evidence-based HTA methodology for policymakers and payers to make pricing, reimbursement, formulary listing, and drug procurement decision of OPPs, specifically in emerging countries. MCDA simple scoring can easily be adapted to suit national health care priorities. In addition, MCDA simple scoring enables proper stakeholder involvement in defining criteria and in performing the scoring, which encourages policy acceptance and sustainability.
      The 22 criteria presented in the MCDA Simple Scoring ISPOR Educational Forum workshop represent a solid baseline for individual health care systems to build upon, but the criteria are flexible and can be tailored to any health care system’s priorities and developmental stages. Further study and discussion on the implementation of MCDA methods need to be done to tailor the number of categories and the inclusion of criteria, weighting, and scoring in several emerging markets.

      Acknowledgments

      The authors thank the International Outcomes Research Board, the workshop participants at the ISPOR Annual Meeting in Milan in November 2015, and Kelley J. P. Lindberg for her editing assistance for this article.
      Source of financial support: The findings of this study are the result of work supported by Abbott Established Pharmaceutical Division.

      References

        • Bate R.
        • Jin G.Z.
        • Mathur A.
        Does price reveal poor-quality drugs? Evidence from 17 countries.
        NBER Working Paper No. 16854, National Bureau of Economic Research, Cambridge, MAMarch 2011
        • Kaló Z.
        • Holtorf A.-P.
        • Alfonso-Cristancho R.
        • et al.
        Need for multicriteria evaluation of generic drug policies.
        Value Health. 2015; 18: 346-351
        • Tawde S.A.
        Generic pharmaceuticals: Is pharmacovigilance required?.
        J Pharmacovigil. 2014; 2: e124
        • Borgherini G.
        The bioequivalence and therapeutic efficacy of generic versus brand-name psychoactive drugs.
        Clin Ther. 2003; 25: 1578-1592
        • Alfonso-Cristancho R.
        • Andia T.
        • Barbosa T.
        • Watanabe J.H.
        Definition and classification of generic drugs across the world.
        Appl Health Econ Health Policy. 2015; 13: S5-S11
        • Corbert B.
        Cobert’s Manual of Drug Safety and Pharmacovigilance. 3rd ed. Jones & Bartlett Learning, LLC, Sudbury, MA2012
        • Institute of Medicine
        Countering the Problem of Falsified and Substandard Drugs.
        The National Academies Press, Washington, DC2013
        • Bate R.
        • Jin G.Z.
        • Mathur A.
        • Attaran A.
        Poor quality drugs and global trade: a pilot study.
        (September) NBER Working Paper No. 20469, National Bureau of Economic Research, Cambridge, MASeptember 2014
        • Gudeman J.
        • Jozwiakowski M.
        • Chollet J.
        • Randell M.
        Potential risks of pharmacy compounding.
        Drugs R D. 2013; 13: 1-8
        • Ventola C.L.
        The drug shortage crisis in the United States: causes, impact, and management strategies.
        P T. 2011; 36: 740-757
        • Fox E.
        • Birt A.
        • James K.
        • et al.
        ASHP guidelines on managing drug product shortages in hospitals and health systems.
        Am J Health Syst Pharm. 2009; 66: 1399-1406
        • Thokala P.
        • Devlin N.
        • Marsh K.
        • et al.
        Multiple criteria decision analysis for health care decision making—an introduction: report 1 of the ISPOR MCDA Emerging Good Practices Task Force.
        Value Health. 2016; 19: 1-13
        • Marsh K.
        • Lanitis T.
        • Neasham D.
        • et al.
        Assessing the value of healthcare interventions using multi-criteria decision analysis: a review of the literature.
        Pharmacoeconomics. 2014; 32: 345-365
        • Baltussen R.
        • Niessen L.
        Priority setting of health interventions: the need for multi-criteria decision analysis.
        Cost Eff Resour Alloc. 2006; 4: 14
        • Diaby V.
        • Campbell K.
        • Goeree R.
        Multi-criteria decision analysis (MCDA) in health care: a bibliometric analysis.
        Oper Res Health Care. 2013; 2: 20-24
        • Angelis A.
        • Kanavos P.
        Value-based assessment of new medical technologies: towards a robust methodological framework for the application of multiple criteria decision analysis in the context of health technology assessment.
        Pharmacoeconomics. 2016; 34: 435-446
        • Marsh K.
        • IJzerman M.
        • Thokala P.
        • et al.
        Multiple criteria decision analysis for health care decision making—emerging good practices: report 2 of the ISPOR MCDA Emerging Good Practices Task Force.
        Value Health. 2016; 19: 125-137
        • Venhorst K.
        • Zelle S.G.
        • Tromp N.
        • Lauer J.A.
        Multi-criteria decision analysis of breast cancer control in low- and middle- income countries: development of a rating tool for policy makers.
        Cost Eff Resour Alloc. 2014; 12: 13
        • Tony M.
        • Wagner M.
        • Khoury H.
        • et al.
        Bridging health technology assessment (HTA) with multicriteria decision analyses (MCDA): field testing of the EVIDEM framework for coverage decisions by a public payer in Canada.
        BMC Health Serv Res. 2011; 11: 329
        • Thokala P.
        • Duenas A.
        Multiple criteria decision analysis for health technology assessment.
        Value Health. 2012; 15: 1172-1181
        • Endrei D.
        • Molics B.
        • Agoston I.
        Multicriteria decision analysis in the reimbursement of new medical technologies: real-world experiences from Hungary.
        Value Health. 2014; 17: 487-489
        • Mohara A.
        • Youngkong S.
        • Velasco R.P.
        • et al.
        Using health technology assessment for informing coverage decisions in Thailand.
        J Comp Eff Res. 2012; 1: 137-146
        • Youngkong S.
        • Baltussen R.
        • Tantivess S.
        • et al.
        Multicriteria decision analysis for including health interventions in the universal health coverage benefit package in Thailand.
        Value Health. 2012; 15: 961-970
        • Defechereux T.
        • Paolucci F.
        • Mirelman A.
        • et al.
        Health care priority setting in Norway: a multicriteria decision analysis.
        BMC Health Serv Res. 2012; 12: 39
        • Youngkong S.
        • Teerawattananon Y.
        • Tantivess S.
        • Baltussen R.
        Multi-criteria decision analysis for setting priorities on HIV/AIDS interventions in Thailand.
        Health Res Policy Syst. 2012; 10: 6
        • Goetghebeur M.M.
        • Wagner M.
        • Khoury H.
        • et al.
        Bridging health technology assessment (HTA) and efficient health care decision making with multicriteria decision analysis (MCDA): applying the EVIDEM framework to medicines appraisal.
        Med Decis Making. 2012; 32: 376-388
        • Hauber A.B.
        • González J.M.
        • Groothuis-Oudshoorn C.G.
        • et al.
        Statistical methods for the analysis of discrete choice experiments: a report of the ISPOR Conjoint Analysis Good Research Practices Task Force.
        Value Health. 2016; 19: 300-315
        • Hu S.
        • Zhang Y.
        • He J.
        • et al.
        A case study of pharmaceutical pricing in China: setting the price for off-patent originators.
        Appl Health Econ Health Policy. 2015; 13: S13-S20