This scoping review responds to the appeal of the scientific community for collaboration between different entities for pharmacovigilance and active surveillance of coronavirus disease 2019 (COVID-19) vaccines. The objective is to identify, systematically evaluate, and synthesize the best scientific evidence available on the indicators used in pharmacovigilance systems. Our results demonstrate that approximately 50% of the 25 studies used in this review have been carried out in the past 5 years. Of these, only four used the pharmacovigilance indicators proposed by the World Health Organization (WHO). Eighty-seven pharmacovigilance indicators were identified, of which seven (8.0%) related to signal detection. While the WHO advocates signal detection as routine pharmacovigilance, in special situations – such as accelerated clinical studies where adverse events are not yet well known – other indicators related to signal detection appear to be good options for maintaining quality pharmacovigilance and active surveillance in the development of the COVID-19 vaccine. However, the less robust pharmacovigilance systems in low-income countries will necessitate greater involvement of health professionals from public and private sectors, pharmaceutical companies, academic institutions, and the general public, to ensure information security and detection of signals for the COVID-19 vaccine.

COVID-19 (Coronavirus disease 2019) is caused by the SARS-CoV-2 virus, an emerging respiratory pathogen. Issues regarding the main epidemiological, clinical, and virological characteristics and, particularly, the capacity for dissemination are still being discovered. When considering coronavirus diseases (e.g. severe acute respiratory syndrome [SARS] and Middle East respiratory syndrome [MERS]) and the experiences in control and prevention adopted so far, the evidence suggests that COVID-19 is transmitted mainly through the respiratory route (1).

In response to the pandemic and the major public and economic health challenges, a vaccine may be the most effective alternative. In view of this, vaccine development is advancing at a record speed through public and/or private partnerships, with almost 200 vaccine candidates under development or in tests.

To coordinate and guide development and testing, the US Centers for Disease Control and Prevention (CDC) and the Advisory Committee on Immunization Practices (ACIP) convened the Vaccine Safety Technical Working Group COVID-19 (VaST). The objectives of the VaST are to review and interpret the safety data of the pre- and post-approval candidate vaccines against COVID-19 and to provide guidance on the presentation of safety data to ACIP and the general public (2, 3). The World Health Organization (WHO) defines the aims of pharmacovigilance, ‘to detect problems related to the use of medicines and communicate the findings in a timely manner’, and ‘to contribute to the assessment of benefit, harm, effectiveness and risk of medicines, leading to the prevention of harm and maximization of benefit’ (4), among others.

Pharmaceutical industries, marketing authorization holders (MAH), and health authorities have an obligation to monitor all licensed drugs by having pharmacovigilance systems in place to ensure any possible risks are identified in a timely manner to avoid or minimize harm to people.

The demands of a pharmacovigilance system vary according to the quantity, nature, and life cycle of products that the company produces, in addition to the country’s regulatory requirements (5). Not all countries have the capacity or resources to carry out adequate surveillance and rely on data from those who can (6).

Instituto Butantan (IB) is a Brazilian public institution and the largest producer of immunobiologicals, sera, and vaccines in Brazil. As part of the WHO prequalification for its trivalent influenza vaccine, IB implemented an active pharmacovigilance system for the post-marketing safety monitoring of its products by creating a Pharmacovigilance Department within its Clinical Trials and Pharmacovigilance Department. While the pharmacovigilance system met local regulations, improvements were required for WHO prequalification. Among other notes, the need to build performance indicators to assess the IB pharmacovigilance system was evidenced, and based on this finding, it was recommended that the IB pharmacovigilance implements an assessment system through performance indicators, also taking into account the Guideline on Good Pharmacovigilance Practices (GVP) (set of measures drawn up to facilitate the performance of pharmacovigilance) regarding performance indicators as a fundamental way of maintaining the quality of service (4, 7).

Considering this requirement, IB proposed to identify the most adequate performance indicators to manage the effectiveness of its vaccine pharmacovigilance system. This evaluation model converges with the need to strengthen pharmacovigilance systems for new vaccines produced quickly in different populations (6) and corroborates with new information about best practice for evaluating pharmacovigilance systems around the world.

In view of the large number, record time of development, and the wide variety of vaccines for COVID-19 that are being developed globally, it is necessary to establish dynamic and sensitive pharmacovigilance strategies to achieve public trust and support of vaccine policies. Security surveillance systems of well-established local and global policies must be strengthened and will be the cornerstone of monitoring the safety of the COVID-19 vaccine. Confidence in vaccines and, therefore, a successful vaccination program can only be achieved when there is transparency in the decision-making process, awareness of how vaccine safety will be monitored, and timely communication on safety monitoring and balancing the risk-benefit ratio of COVID-19 vaccines (6, 8).

The purpose of this scoping review is to identify, systematically assess, and summarize the best available scientific evidence on which indicators are used in pharmacovigilance for vaccine safety, especially in the context of emergency use, to monitor and communicate safety monitoring findings in a timely manner for better decision-making and follow-up.

This was a scoping review developed by researchers, blinded for review. This review was registered on Open Science Framework in September 2020 (https://osf.io/wyvgd//). In addition, the reporting of this scoping review followed the recommendations of the PRISMA for Scoping Reviews (PRISMA-ScR) (9).

All studies involving pharmacovigilance were eligible. The studies were evaluated with information on pharmacovigilance systems in any part of the world, based on the application of indicators proposed by the WHO or others. Studies that did not use indicators to evaluate pharmacovigilance systems were excluded.

This scoping review follows the recommendations proposed by Arksey and O’Malley (10) and the Jonna Briggs Institute (JBI) (11). It allows for mapping the main concepts/indicators, research areas, and identifying knowledge gaps. For the construction of the research question, the Population, Concept, and Context strategy was used: P: hospitals, pharmaceutical industries, healthcare providers, national regulatory authority, and other established regulatory authorities; C: indicators, assessment, and evaluation of the services and interventions; and C: pharmacovigilance for a scoping review.

To identify potentially relevant documents, the following bibliographic databases were searched from April to July 2020: Cochrane Library (Wiley); Embase (Elsevier); the LILAC’s (BVS), PubMed; CINAHL; Web of Science; Scopus, SciELO; and the Opengrey (https://opengrey.eu) for the grey literature. A manual search was conducted using the references for the primary and secondary studies found in the electronic search. The search strategies were developed by a librarian with experience in health area. The librarian followed the recommendation of Peer Review of Electronic Search Strategies (PRESS) (2015) (12), which consists of a set of recommendations for developing the search strategy and used each electronic database that was searched between April and July 2020. They are presented in Table 1.

The selection and analysis of the studies were carried out by two blinded and independent authors. The first selection was made based on the title and summary of the studies. To manage the studies, we used the Rayyan tool (13). Conflicts were resolved by consensus. After selection according to the inclusion criteria, two reviewers independently analyzed the texts in full in order to identify the relevant outcomes.

The scoping systematic review yielded 720 papers of which 347 were duplicates. After the titles and abstracts had been read by two independent evaluators through the Rayyan online platform, 41 articles were included for the full text to be read. Through this, 25 studies were included, two of which were about vaccine pharmacovigilance and 23 about general pharmacovigilance (14–38). The PRISMA-ScR (9) flowchart is shown in Fig. 1.

The WHO indicators are already validated, and that is the reason why the results of this review will be presented by dividing the indicators found in WHO indicators and non-WHO indicators.

The indicators proposed by the 25 ongoing studies in Table 2 are shown by dividing the studies into two groups: WHO indicators and non-WHO indicators. The years of publication ranged from 2004 to 2020. These studies were conducted in European countries, including Italy, France, Croatia, Switzerland, and in Asian countries, including Indonesia, Malaysia, Philippines, Singapore, Thailand, Later, Brunei Darussalan, Vietnam, Laos, Myanmar, Cambodia, China, Japan, Arabia, and Middle East; in African countries, including Nigeria, South Africa, and Sudan; in Central America countries, including Mexico and Cuba; and the United States of America representing North America. None of the studies were conducted in South America, but one study was searched on all WHO member countries (Table 2).

Although the search found studies from 2004, half of them (60%) were published in the last 5 years, showing a more recent concern related to the quality management of pharmacovigilance systems. The countries that published the most in the last 5 years were the African and European countries, but only the United States has made five publications since 2004.

Of the 25 studies carried out, only four used the pharmacovigilance indicators proposed by the WHO (14–17), of which three were African countries (14, 16–17) and one was from Arab and Eastern Mediterranean countries (15). The objective of the two of these studies was to describe the current pharmacovigilance scenario in their countries; one study aimed to evaluate the pharmacovigilance system of tertiary hospitals, and the other study aimed to evaluate the pharmacovigilance structures, processes, and results in three public health programs.

Table 3 shows the indicators proposed by the studies that differ from WHO pharmacovigilance indicators. Among the indicators found, those related to signal detection are as follows:

Our study responds to the call of Petousis-Harris (6) on the need for collaboration between different robust systems of pharmacovigilance and active surveillance for monitoring and timely communication in the risk-benefit ratio of vaccines against COVID-19. We systematically present the pharmacovigilance indicators published in literature in the last 15 years to the scientific community and propose the most sensitive ones for application in pharmacovigilance and active surveillance systems based on those used in pharmacovigilance systems in different regions of the world.

Our results demonstrate that approximately 50% of the 25 studies used for this scoping review were carried out in the past 5 years, showing a more recent concern related to the quality management of pharmacovigilance systems. Of these, only four used the pharmacovigilance indicators proposed by the WHO. In total, 87 pharmacovigilance indicators were identified, of which seven (8.0%) are related to the detection of signals.

Signal detection is an internationally recommended indicator for the safety monitoring of vaccines and other drugs. The WHO advocates signal detection as an important tool to quickly identify rare signals; the WHO believes that the impact of new and often rare adverse reactions can be minimized as soon as they occur. In special situations, where clinical studies are accelerating and adverse events are not yet well known, signal detection becomes even more important (39, 40). Therefore, in the case of the development of a COVID-19 vaccine, this seems to be an indicator of choice for the quality of pharmacovigilance and active surveillance systems. The ‘number of signals detected in the past 5 years by the pharmacovigilance centre’ is already one of the indicators proposed by the WHO to identify possible adverse events even in the form of a safety signal.

However, to apply this and the other proposed indicators, pharmacovigilance systems must have sensitive tools possible to detect a safety signal as soon as possible, and this will depend on data and notifications. In other words, it will depend on the quality of the information generated by active searching for data and health professionals committing to report the events found. In this context, each national pharmacovigilance center in different regions of the world must be able to provide valid information regarding content, numerator, denominator, scope, data source, and limitations. Therefore, we depend on spontaneous notification and data collection through passive and active pharmacovigilances (41).

To this end, there is a need for committed and active professionals in notification of adverse events. Encouraging spontaneous notification of adverse events and allowing the sharing of information on drug safety are fundamental parts of establishing guidelines for pharmacovigilance in the case of COVID-19. Although spontaneous reporting is always necessary as a sensitive means of identification among those exposed, it is known that additional methods are needed to establish safety profiles and estimate the occurrence rates of adverse events (42), especially in low-middle income countries.

The indicators proposed by the WHO were created from meetings of pharmacovigilance experts with the contribution of several countries. After 32 meetings, the defined indicators were presented to the representatives of the National Centers of the countries participating in the WHO Programme for International Drug Monitoring, categorized into basic, complementary, and public health, thus being distributed to the National Centers. All of them have been validated by the WHO Advisory Committee on Safety of Medicinal Products (ACSoMP) and are already used by some of the articles present in this review. As we observed in this study, an alternative for assessment and validation of the indicators is to use the Delphi method (43).

In countries with few pharmacovigilance systems and functional challenges, there is a lack of evidence, legislation, regulatory framework, and financial support (6). Others still face challenges with low rates of detection and investigation of safety signals, lack of epidemiological tools for active surveillance, and lack of information sharing between countries (44). For Olsson (42), greater involvement of health professionals from the public and private sector, pharmaceutical companies, academic institutions, and the general public is necessary to ensure information security.

Our results demonstrate that the theme of the quality management of pharmacovigilance systems is a recent one, with most of the studies included being published in the last 5 years. Signal detection is an internationally recommended indicator for monitoring vaccine safety, yet it was identified in only 8% of studies, highlighting the need for discussions and consensus on this specific topic. The indicators of a successful vaccination program can only be achieved when there is transparency in decision-making, awareness of how the safety of the vaccine will be monitored, and timely communication on the safety monitoring and risk-benefit ratio of the COVID-19 vaccines.

We highlight that a coordination of post-approval vaccine safety monitoring efforts through indicators, protocols, and signal detection will allow for timely identification and assessment for fast and safe responses.