The covid vaccine sceptical community continue to milk self-reporting systems such as the UK Yellow Card system and the US VAERS system to try and support their case that the vaccines are unsafe.
They make two main points:
A) There are far more incidents reported on these systems for the Covid vaccines than for other vaccines.
B) Self-reporting systems typically underreport the number of actual incidents. So the real number of incidents is even higher than the figures that are reported.
For those that are not familiar with them, a quick recap on how these self-reporting systems work and why they are there. I will use the Yellow Card system as an example, but the VAERS system is similar. Anyone can report any incident which they think may be related to any kind of medical intervention. It can be a vaccine but can also be a drug or other intervention. It is commonly used by healthcare professionals to report patient’s incidents (the system asks if the reporting agent is a healthcare professional) but is also used by the public to report their own and other’s incidents. These systems are intended to complement existing precautions such as clinical trials. Their advantages are:
- They can generate very large quantities of data. That may allow competent authorities to detect very rare side effects.
- They can continue to monitor an intervention for many years after any clinical trials have been completed. So that side effects that are only apparent after some years can be detected.
The chief disadvantages are:
- Many incidents that are connected to the intervention will not get reported. This will vary immensely depending on the intervention, the incident, and the context at the time.
- Conversely, as there are no constraints on who reports incidents under what circumstances, many of the incidents that are reported may be unconnected to the treatment. It may be a coincidence that the side effect takes place shortly after the treatment. Or the incident may never have happened, the report could be a hoax, or a misunderstanding.
A lot of the following discussion turns on the difference between a report of an incident that is in fact caused by the vaccine, and a report of an incident that appears to be caused by the vaccine but isn’t, for reasons discussed above. I’ll call the first type of report a true report and the second type a false report. A few points about false reports:
- The term is not meant to imply that false reports are intended to deceive. There may the odd hoax, but the vast majority of false reports are going to be genuine concerns but are actually just coincidences etc.
- Most people making false reports will never know they are false.
- Even healthcare professionals are quite likely to make false reports. Remember they are asked to report suspected side effects so any healthcare professional who errs on the side of caution will be prone to making false reports.
- The Yellow Card system is there to make sure that any rare side effects are picked up, not to assess the quantity of incidents. It is relatively unimportant if there are false reports.
- The MHRA, and others looking at the data, only see reports. They have no way of telling how many are true and how many are false.
Now taking the sceptical points in turn.
A) It is true that there are far more reports for the covid vaccines than for other vaccines. This is true even if you allow for the vast number of people that were vaccinated for covid. The number of reports per dose for the covid vaccines is many times the reports per dose for most other vaccines (some of which were subsequently withdrawn for safety reasons). There are many possible reasons for this. One is that the covid vaccines cause more incidents than other vaccines (and these are mostly true reports). But there are other more prosaic explanations. The most obvious being the blaze of publicity surrounding the covid vaccines and their safety. This would surely encourage many more people to report what they believe to be an incident. Another is the focus on detecting potential safety problems and the Yellow Card system. When I was vaccinated for covid, I was given a leaflet describing the Yellow Card system – the only time this has happened for all the many vaccines I have had in my lifetime. A third is the high proportion of reports from non-healthcare professionals – about 75% in the case of Pfizer. The proportion of reports from non-healthcare professionals has risen over the years but not to anything like this – in 2018 it was about 25%. This means much larger population is potentially reporting than for previous vaccines. And being non-healthcare professionals they are going to be more susceptible to making false reports.
B) There has been concern that self-reporting systems are “underreporting” for some time. There is a lot of evidence that self-reporting systems miss a lot of incidents. As the HART group say:
Underreporting is a well-known phenomenon and has been consistently reported in recent decades: “the median underreporting rate in the 37 studies was 94% (82-98%). There was no significant difference between the median underreporting rates calculated for general practice and hospital studies”, see 2006 review article in ‘Drug Safety’: “Under-reporting of adverse drug reactions : a systematic review”
The article they refer to is behind a paywall but it appears from the abstract that while it covers a wide variety of self-reporting systems for tracking adverse reactions to drugs it does not include self-reporting systems that extend to the public such as the Yellow Card system. The abstract implies that all the studies were in a healthcare setting, either there a or GP surgery. Nevertheless, it is evidence that many/most self-reporting systems miss a lot of incidents.
In the specific context of the Yellow Card system is this 2019 document from the MHRA – which is an appeal to increase the amount of reporting on the Yellow Card system following a decline in 2018 (clearly this is all before covid). The document includes this sentence:
It is estimated that only 10% of serious reactions and between 2 and 4% of non-serious reactions are reported.
This single sentence appears to be the source of most (perhaps all) sceptical claims that Yellow Card reported incidents for covid vaccinations are a gross underestimate of the actual number of incidents. However, it needs careful interpretation. First there are issues around its credibility. Who made the estimate? What was their evidence? Remember that the objective of this document was primarily to encourage healthcare professionals to report – so it was probably phrased to give the impression of emergency. But also what does it really mean? Does it apply only to the Yellow Card system or other systems for tracking reactions? Does it apply equally to all types of intervention?
Most significantly – what does it mean by underreporting? There are two quite different senses in which the Yellow Card system might be underreporting.
- There may be many incidents caused by the vaccines which have not been reported. In other words the true reports are less than the actual number of incidents.
- The total number of reports (true plus false) maybe much less than the actual number of incidents.
Intuitively the first seems very likely to be true. For all sorts of reasons both healthcare professionals and the public may fail to report incidents caused by the vaccine. They may not have detected the incident. They may be too busy to report it. They may not have heard of the yellow card system. They may be wary of using it. And we have the evidence from other types of self-reporting system as suggested by HART. However, there is no way of knowing whether the second is true or false. We simply don’t know how many of the reports are false. And when 75% of the reports are from the public it quite possible that the number is very large.
Which is why the Yellow Card system is useless for estimating the number of incidents caused by the vaccine.
In Moderation 