Talk:Comparison criteria

Health endpoints to consider

How to read discussions

Fact discussion: .
Opening statement: These health endopoints and other criteria should be considered in the comparison: Price of vaccine Invasive pneumococcal disesase (IPD) Otitis media Pneumonia Upper respiratory infections (other than otitis media) Treatment costs Infections caused by Hemofilus influenzae Technical criteria such as service, management and labeling. Closing statement: Criteria or endpoints that should be considered are a) price, b) invasive pneumococcal disease, and c) technical criteria. (A closing statement, when resolved, should be updated to the main page.)
Argumentation: ⇤--#: . An endpoint or criteria should NOT be considered unless there are clear reasons to do so. This is the nature of a tendering process. --Jouni (talk) 13:43, 4 September 2014 (UTC) (type: truth; paradigms: science: attack) Price of vaccine ←--#: . This seems obvious. --Jouni (talk) 13:43, 4 September 2014 (UTC) (type: truth; paradigms: science: defence) ----#: . PCV tender criteria from other Nordic countries show that quality is valued and given greater weight than price also where the winning vaccine is considered the “economic most advantageous”. --Jouni (talk) 13:43, 4 September 2014 (UTC) (type: truth; paradigms: science: comment) ←--#: . See comment Talk:Comparison criteria K3 --Pfizer Oy, 2014-08-27 13:47:39 (type: truth; paradigms: science: defence) IPD ←--1: . Everyone seems to agree that IPD is an important health endpoint to consider. --Jouni (talk) 09:32, 4 September 2014 (UTC) (type: truth; paradigms: science: defence) Otitis media ←--2: . Restricting the assessment of benefits to invasive disease fails to reflect the full benefits of PCV vaccination. --GSK 12:58, 3 September 2014 (UTC) (type: truth; paradigms: science: defence) ⇤--3: . Due to replacement of serotypes in carriage, it is unlikely that otitis media would actually reduce a lot in the long run. --Jouni (talk) 09:32, 4 September 2014 (UTC) (type: truth; paradigms: science: attack) ←--4: . Most of the benefits of PCV are seen in respiratory infection episodes, especially mild upper respiratory infections and other non-invasive disease endpoints, such as hospital-diagnosed and hospital-treated pneumonia, tympanostomy tubes placements, and antimicrobial purchases (largely related to otitis media treatment). This translates to reductions of all healthcare costs, which are dominated by prevention of otitis media related outcomes. Studies demonstrated that the benefits in terms of both episodes and associated costs reduced are comparatively more significant than invasive disease alone. --GSK 12:58, 3 September 2014 (UTC) (type: truth; paradigms: science: defence) ←--5: . Original comments Talk:Comparison criteria GSK2, GSK5 --Jouni (talk) 11:07, 4 September 2014 (UTC) (type: truth; paradigms: science: defence) Pneumonia ←--2: . See argument 2. --Jouni (talk) 09:32, 4 September 2014 (UTC) (type: truth; paradigms: science: defence) Upper respiratory infections (other than otitis media) ←--2: . See argument 2. --Jouni (talk) 09:32, 4 September 2014 (UTC) (type: truth; paradigms: science: defence) Treatment costs ←--6: . impact of PCVs on respiratory diseases (including pneumonia and acute otitis media related endpoints) should be included because these are the major drivers of the cost effectiveness of PCV programmes and the scope of the procurement of PCV is to enable the choice of the economically most advantageous tender. -- GSK 12:58, 3 September 2014 (UTC) (type: truth; paradigms: science: defence) ⇤--3: . See argument 3. --Jouni (talk) 09:32, 4 September 2014 (UTC) (type: truth; paradigms: science: attack) ←--4: . See argument 4. --Jouni (talk) 11:07, 4 September 2014 (UTC) (type: truth; paradigms: science: defence) Infections caused by Hemofilus influenzae ⇤--#: . Vaccine efficacy on non-typeable Haemophilus influenzae cannot be taken into account as comparison criterion because neither of the vaccines have prevention of NTHi in their indication. --Jouni (talk) 13:43, 4 September 2014 (UTC) (type: truth; paradigms: science: attack) ←--#: . See comment Talk:Comparison criteria K5 --Pfizer Oy, 2014-08-27 13:47:39 (type: truth; paradigms: science: defence) Technical criteria such as service, management, and labeling ←--#: . Quality points should be given to several dimensions, not just for number of serotypes in the vaccine, for example: service after delivery, vaccine stability, clear labelling. --Jouni (talk) 13:43, 4 September 2014 (UTC) (type: truth; paradigms: science: defence) ←--#: . See comment Talk:Comparison criteria K3 --Pfizer Oy, 2014-08-27 13:47:39 (type: truth; paradigms: science: defence)

Discussion on comparison criteria

Comment K3 --Pfizer Oy, 2014-08-27 13:47:39 - R upload

PCV tender criteria from other Nordic countries show that quality is valued and given greater weight than price also where the winning vaccine is considered the “economic most advantageous”. Quality points should be given to several dimensions, not just for number of serotypes in the vaccine, for example:

Norway (2011)

Quality 50-70%, price 20-40%, service after delivery 5%. “The offer that achieves the highest point score after adding up the calculated points for each of the weighted criteria for assignment will be considered as the financially most favorable offer.”

Criteria:

• Requirement: It must be possible to store the vaccine at 25°C for at least 48 hours, without altering the vaccine properties. Documentation: stability studies
• The expected number of preventable cases of systemic pneumococcal disease. The efficacy of the vaccine against systemic pneumococcal disease caused by the different serotypes in the vaccine will be of significance. Other relevant conditions, including clinical documentation on the efficacy against otitis caused by S.pneumoniae.
• The best possible safety profile in this particular population. The offered vaccine may be part of a vaccination program where both available pneumococcal vaccines can be used. The prevalence in Norway of the individual disease-developing serotypes will be considered.
• The efficacy and risk/benefit assessment of the vaccines will be assessed on a rough estimate based on the presented documentation. Documentation assessed by competent authorities in the EEA-union will be of particular importance.
• Price pr. dose size is given as one price, which is applicable independently of the delivered dose, i.e. if it is delivered as one dose pack or as 10-dose packs.
• Service after delivery (5%) Service availability and technical assistance will be emphasized

Denmark (2014)

Quality 75%, price 25%. “The framework contract will be awarded on the basis of the award criterion the economic most advantageous tender .

Criteria:

• The effectiveness is assessed on the basis of the vaccine's coverage of invasive pneumococcal diseases in children < 2 years (35%) and the product's direct and indirect coverage of invasive pneumococcal diseases in the rest of the population (25%) in Denmark in 2009 based on the applicable SPC (Summary of Product Characteristics). The tenderer must complete a form concerning the indication of effectiveness of pneumococcal serotypes contained in the vaccine on the basis of the applicable SPC.
• Type and frequency of adverse effects (10%): assessed on the basis of the applicable SPC. As few and least invasive adverse effects as possible are desired.
• The active ingredients/trace elements of the products (5%): it is assessed whether the product is manufactured by or includes materials or trace elements of animal or human origin or other ingredients that may potentially pose a risk to the child/patient.
• The assessment is made on the basis of an applicable SPC, a completed form relating to active ingredients/trace elements enclosed with the tender documents as well as any statement as to no active ingredients and other relevant material, as described in section 10.4. As few active ingredients and trace elements that may potentially pose a risk to the child/the patients as possible are desired.
• The price will be assessed on the basis of the average price for the period 2014-2017. The lowest price possible is desired.

Sweden, Stockholm (2015)
Quality emphasized. If the tender gets 100 points for quality, the price will be multiplied by 1. If, for example, the tender gets 50 quality points, the price will be multiplied by 1,50.

Criteria:

• Clinical efficacy, safety (max 90 points): Range of serotype protection, documentation on efficacy and effectiveness, documentation on risk groups, documentation on antibody response, side effect profile, excipients and preservatives. Very good safety profile, good clinical efficacy, optimal protection in terms of serotype coverage, documentation on risk groups: 90 points:
• Very good safety profile, good clinical efficacy but not optimal protection in terms of serotype coverage, no documentation on risk groups: 20 points Line width, practical management, labelling
• (max 10 points) Size, needles, design (risk of mix up), clear labelling and presence of bar code, storage conditions, handling in general

Sweden, VGR (Region Västra Götaland)
Quality 70%, price 30%

Criteria:

• Clinical efficacy (max 60 p): Number of serotypes, clinical efficacy on individual level, proven herd immunity, approved indications. Based on RCT and/or other studies, official authority statements
  • Very Good:48-60 points
  • Good: 34-47 points
  • Acceptable: 20-33 points
  • Less good: 10-19 points
  • Poor or not gradable: 0-9 points:
• Clinical safety (max 60 p): Side effect profile in SmPC/WHO, RCT and/or other studies, official authority statements. Aluminum content.
  • Very good: 48-60 points
  • Good: 34-47 points
  • Acceptable: 20-33 points
  • Less good: 10-19 points
  • Poor or not gradable: 0-9 points
• Product range (max 10 p) Will be evaluated if relevant, otherwise maximum points will be awarded. *Package size: 0-5 points
• Dosing schedule: 0-5 points
• Practical management (max 30 p) Will be evaluated if relevant, otherwise maximum points will be awarded
  • Storage conditions (e.g. light sensitive, storage in cold, durability):0-10 points
  • Formulation aspects (e.g. dry matter/vial/pre-filled syringe, preparation instructions): 0-5 points
  • Product itself (e.g. manageability at the preparation and administration, packaging design): 0-15 points
• Marking and labelling (max 20 p) The bidder shall submit Mock-ups on the outer packaging and images on the inner packaging to the bid, and blister on all offered products. If need of pharmaceutical samples, the company will be contacted.
  • Barcode (outer and inner packaging): 0-10 points
  • Labelling (outer and inner packaging, readability on the syringe label, removable label): 0-10 points

Comment K4 --Pfizer Oy, 2014-08-27 13:47:39 - R upload

It is appreciable Opasnet is a source of transparency. As JCVI and ACIP do, is it planned to inform the general public by publishing regularly the evaluation and decision processes, including analysis and results, at Opasnet or somewhere else?

Comment K5 --Pfizer Oy, 2014-08-27 13:47:39 - R upload

Vaccine efficacy on non-typeable Haemophilus influenzae cannot be taken into account as comparison criterion because neither of the vaccines have prevention of NTHi in their indication. Moreover, for both vaccines the Summary of Product Characteristics (SmPC) approved by European Medicines Agency states in section 4.4., Special warnings and precautions, that the vaccines do not provide protection against other bacteria: Synflorix: There is insufficient evidence that Synflorix provides protection against pneumococcal serotypes not contained in the vaccine or against non-typeable Haemophilus influenzae. Synflorix does not provide protection against other micro-organism Prevenar 13: Prevenar13 will only protect against Streptococcus pneumonia serotypes included in the vaccine, and will not protect against other microorganism that cause invasive disease, pneumonia or otitis media. Based on the same sentences in SmPCs of the vaccines, the vaccines do not provide protection against other serotypes than those contained in the vaccines. Therefore, adjustability of the serotype composition in the economic assessment (http://en.opasnet.org/w/Economical_assessment ) is purposeless and the user defined option should be removed.

Comment K6 --Pfizer Oy, 2014-08-27 13:47:39 - R upload

According to the information on THL’s webpage, the duration of FinIP, Finnish Invasive Pneumococcal Disease Vaccine trial, is until the end of 2018. Would FinIP follow-up impact the tender? Ref. Lääkärilehti 22.8.2014, page 2017 Budjettileikkuri iskee rokotuksiin.

Question:

• "Benefits refer to the decrease in disease burden caused by large scale use of the vaccine. As a result health care costs decrease and the quality of life attributed to improved health increases."

⇤--GSK1: . Because there are no head-to-head studies which directly compare the effects of the vaccines, this leads to intrinsic uncertainties around the potential differences in the impact of individual formulations, and given that there are much less uncertainties around the cost of the vaccination programme, these two factors should be weighted accordingly.

The selection criteria should also explicitly value the wealth of clinical evidence generated in randomized clinical trials, as well as in post marketing surveillances, including data generated in Finland itself, in addition to the use of modelling to predict the impact of vaccination. Assumptions used in the model should be reflective of this clinical evidence and applicants should be invited to make proposal for these assumptions with appropriate substantiation. -- GSK 12:58, 3 September 2014 (UTC) (type: truth; paradigms: science: attack)

⇤--1: . It is true that there is considerable uncertainty in the predicted impact of individual vaccine formulations. Because of this, extensive uncertainty analyses have been undertaken to identify the most influential assumptions affecting any differential impact of the two currently available conjugate vaccines. These most influential of assumptions concern the expected effectiveness of individual serotypes included in the vaccines. Of note, although there are uncertainties in the estimated QALYs or medical costs, these affect both vaccines in their comparative evaluation.

Importantly, the overall cost effectiveness of pneumococcal conjugate vaccination needs to be understood as well, at least on a broad scale, to appreciate the significance of any (predicted) differences between the impact of the vaccines to be compared.

There is no indication in which respect data specific to Finland had not been taken into account. The model is based on evidence on the impact of large-scale and long-term use of pneumococcal conjugate vaccines, including their proven or inferred effect on (vaccine-type) transmission in Finland and elsewhere. The time-scale chosen for the analysis (i.e. 5-10 years in the future) as well as the fact that the overall impact in the Finnish population is assessed means that direct efficacy in the target population is only part of the answer. In particular, current status data on the epidemiology in any vaccinated population may provide only a ‘snap-shot’ of the underlying trends.

Regarding the request of ‘appropriate substantiation’, these web pages provide the forum for further discussion. {{{3}}} (type: truth; paradigms: science: attack)

Question

• "Benefits are quantified as the expected decrease in invasive pneumococcal disease incidence due to vaccination."

⇤--GSK2: . The Opasnet page entitled Tendering process for pneumococcal conjugate vaccine, section Question/Scope (http://en.opasnet.org/w/Tendering_process_for_pneumococcal_conjugate_vaccine) stipulates: “The preparation of the criteria is based on current knowledge of the impact of pneumococcal conjugate vaccination.”

According to the current knowledge, notably supplemented recently with data generated in a double blind randomized clinical trial in Finland (e.g. Palmu, Lancet, 2013, Palmu Lancet Resp Med, 2013, Palmu ISRAOM, 2013, Kilpi ESPID, 2013), the benefits of reduction of invasive disease are representing a relatively minor fraction of the total infections prevented by PCVs.

It has been demonstrated that majority of reductions is seen in respiratory infection episodes, especially mild upper respiratory infections. This translates to reductions of all healthcare costs, which are dominated by prevention of otitis media related outcomes. (Palmu et al. ESPID, 2014).

Restricting the assessment of benefits to invasive disease fails to reflect the full benefits of PCV vaccination; thus reductions in pneumonia and acute otitis media related outcomes should be taken into account. --GSK 12:58, 3 September 2014 (UTC) (type: truth; paradigms: science: attack)

Rationale:

• "To assess the health benefits of vaccination, the following items needs to be known or assessed: pneumococcal serotypes included in the vaccine..."

⇤--GSK3: . Serotype content alone is an insufficient basis for vaccine comparisons, even if only IPD effects are considered.

Based on efficacy and effectiveness data gathered to date, differences in the protection against overall IPD conferred by PCV10 or PCV13 are not clearly demonstrated.

This is because overall IPD protection results from a combination of vaccine effect against vaccine and vaccine-related serotypes, along with the vaccine effect on non-vaccine-type replacement.

For example, it is not possible to conclude on efficacy against serotype 3 based on the available body of evidence (e.g. Mrkvan, ICAAC 2013, Dagan, CID 2013; Ben-Shimol, CID 2014). In addition, emerging evidence indicates that vaccination with PCV10 is likely to offer significant protection against IPD due to vaccine-related serotypes 6A and 19A (up to 100% and 82% reductions, respectively; Jokinen WSPID 2013, Domingues, Lancet Resp Med 2014) have been observed post- PCV10 introduction to UMVs) and it is unclear whether this protection would substantively differ from that offered by PCV13, because of the absence of efficacy/effectiveness data generated in head-to-head studies.

Therefore, using the pneumococcal serotypes included in the vaccine does not seem to meet the requirements of the public procurement law, which stipulates that the comparison criteria related to the differences must be clearly demonstrated. In fact using pneumococcal serotypes included in the vaccines as a selection criterion would put tenderers on an unequal footing without sufficient scientific basis and would therefore be contrary to the requirement of non-discriminatory treatment in procurement law.The vaccines should be rather evaluated based on the merits of the vaccine effects demonstrated in clinical trials and/or post licensure implementation. -- GSK 12:58, 3 September 2014 (UTC) (type: truth; paradigms: science: attack)

• "The tentative assumption is that the use of either the 10-valent (PCV10) or 13-valent (PCV13) pneumococcal conjugate vaccine (PCV) in the childhood immunization programme significantly reduces the disease incidence attributable to the serotypes included in the vaccine and that this protection extends to the population at large. This assumption may be modified so that the vaccine also reduces disease incidence caused by certain serotypes not included in the vaccine (due to cross-protection). "

⇤--GSK4: . In light of the comments above, as far as possible, the assumption for serotype specific vaccine effectiveness for vaccine and vaccine-related types should be based on available clinical evidence. -- GSK 12:58, 3 September 2014 (UTC) (type: truth; paradigms: science: attack)

• "For other pneumococcal disease endpoints, reliable estimates of the vaccine-preventable disease incidences are not available."

⇤--GSK5: . Although the impact on other than invasive disease endpoints is more difficult to model, assessment of the impact of PCVs on respiratory diseases (including pneumonia and acute otitis media related endpoints) should be included because these are the major drivers of the cost effectiveness of PCV programmes and the scope of the procurement of PCV is to enable the choice of the economically most advantageous tender.

It should be considered that vaccine preventable disease incidence estimates have been reported for a number of non-invasive disease endpoints, including hospital-diagnosed and hospital-treated pneumonia, tympanostomy tubes placements, antimicrobial purchases (largely related to otitis media treatment) in a nation-wide randomized clinical trials conducted in Finland recently (Palmu Lancet Resp Med, 2013, Palmu ISRAOM, 2013, Kilpi ESPID, 2013), and demonstrated that the benefits in terms of both episodes and associated costs reduced are comparatively more significant to invasive disease alone (Palmu, ESPID 2014, poster). -- GSK 12:58, 3 September 2014 (UTC) (type: truth; paradigms: science: attack)