Feasibility Assessment and Governance of Real-World Data in China

2021-07-27 | Press Releases

Recently, the Guidelines for Real-world Data Used in the Generation of Real-world Evidence (referred to below as the Guidelines) issued by the Center for Drug Evaluation (CDE) of the National Medical Products Administration (NMPA) were offered to the public for comments. The Guidelines comprehensively elaborate the definitions, sources, evaluations, governance, standards, safety compliance, quality assurance, applicability, and other aspects of real-world data (RWD), to provide specific requirements and guiding recommendations for RWD. They provide a guidance on how RWD can be used to generate real-world evidence (RWE) to support drug development and how feasibility assessment and data governance should be done.  


It is critical to convert RWD into the analytical data for clinical study after passing compliance standards. It is also important to determine whether RWD is suitable for generating RWE to support decision-making in drug development and regulatory evaluation. Jun Wang, Deputy Director of the CDE Statistics and Clinical Pharmacology Department, said that when companies consider supporting drug registration using RWE, they must consider many factors first, including the applicability of RWE strategy, real-world study design, RWD relevance, data quality, and statistical analysis methods. We can then consider using  RWE in regulatory decision-making.


Feasibility Assessment of RWD

Due to less stringent quality control in RWD collection and storage, problems like incomplete data and inconsistent standards and models may lead to problems, such that some RWD cannot be used to generate RWE. Only those evaluated as “potentially applicable” can generate RWE through a real-world study (RWS).

These Guidelines supplement the requirements of Guidelines on Real-World Evidence Supporting the Drug Development and Review (Trial) on the relevance and reliability of RWD data. The Guidelines state that the feasibility assessment of RWD should be divided into two stages: the evaluation of data source and the evaluation of governed data.

The first stage of feasibility assessment is to conduct a preliminary evaluation on data source from the perspectives of accessibility, ethical compliance, representativeness, critical variable integrity, sample size, and data source activity status. Only the RWD which meets the basic analysis requirements of the protocol would be selected. If it fails to pass the first-stage , the study can be terminated before data governance to save human, financial, and material resources. Once passing the first-stage, the governance of RWD will take place, followed by the second-stage assessment.


The second-stage evaluation of RWD is based on the relevance and reliability of the data and the data governance mechanism (data standard and general data model) proposed to evaluate and analyze the governed data for RWE generation. The suitability of governed data has two core elements.


The first element is relevance, which evaluates whether RWD is closely related to the clinical questions of interest and whether it contains information that can answer the clinical questions. RWS does not refer to conduct a study based on random data. The critical premise of RWE accepted by regulatory agencies is to identify clinical questions and then conduct a study to answer questions, rather than use the data-oriented study model through data mining. Medical big data and advanced data mining technologies are valuable for providing new perspectives and methods to generate RWE. Nevertheless, such study methods aiming at mining positive signals may generate misleading and biased conclusions. Therefore, when use RWD, regulatory agencies must define clinical questions first prior their assessment on whether RWD contains critical information to answer the questions, such as exposure, outcomes, and other key covariates.


The second element is data reliability. The quality of the data should meet the basic requirements to obtain reliable conclusions. The evaluation should focus on completeness, compliance, accuracy, and transparency (traceability) at the time of data collection and implementing quality control and quality assurance measures during the entire process of data governance.


Data Governance of RWD

After passing the first-stage, we can proceed to RWD governance. To serve specific clinical study objectives, governance of RWD converts raw data into a research database to meet  requirements for statistical analysis. The China REal World Data and Study ALliance (ChinaREAL) had issued technical specifications on how to build a research database using established healthcare data and patient registration databases. , and provided guidance on the construction process and quality evaluation of the database. Data governance is not limited to data extraction, cleaning, and conversion. The Guidelines describe in details data security processing, data extraction, data cleaning, data conversion, data transmission and storage, data quality control, general data models, and real-world data governance proposals.


When doing data governance, it’s critical to have standard process, scientific rigor, high security, and high data quality. Data governance should encompass a complete quality control system to standardize RWD life cycle management, a  secured network environment and an access-controlling mechanism to ensure the data security. It also requires a pre-formulated governance proposal to serve as a foundation to ensure data quality . The data governance proposal should be synchronized with the research project proposal to avoid data-driven governance methods .


Quality control should cover the entire process of data extraction (single data extraction, multiple data source extraction, data desensitization), data cleaning (duplicate data processing, logical check and abnormal data processing, missing data processing ), data conversion (general data model, medical terminology and coding, natural language processing, calculation of derived variables), and subsequent storage, transmission, analysis, and submission. To ensure regulatory compliance, it is critical to conduct quality assurance by preventing, investigating, and correcting data errors or problems.


With the rapid development of information technology and data science, RWE based on diversified real-world data systems has become an essential source of healthcare decision-making (including medical device supervision, catalog development, guideline development, and disease management). Although the current quality of RWD limits its applications, a complete, accurate, and usable research database can be constructed through a safe and scientific feasibility assessment and governance of RWD. In this way, it is possible to generate high-quality RWE recognized by regulatory agencies.