 |
|
||||
|
|||||
| About | Latest Articles | Back-Issues | Articles
|
Search | Instructions | Online Submission | Subscribe | Etcetera | Contact |
| |||||||||||||||||||||
|
An audit of safety reporting in randomized controlled trials over a five-year period in a high impact factor journal M Konwar1, A Mamde1, P Patankar2, UM Thatte1, NJ Gogtay11 Department of Clinical Pharmacology, Seth GS Medical College and KEM Hospital, Mumbai, Maharashtra, India 2 Department of Pharmacology and Therapeutics, Seth GS Medical College and KEM Hospital, Mumbai, Maharashtra, India
Correspondence Address: Source of Support: None, Conflict of Interest: None DOI: 10.4103/jpgm.JPGM_78_21
Keywords: Investigator-initiated trial, P-value, randomized controlled trial, safety outcome
Well-designed randomized controlled trials (RCTs) are the gold standard for evaluating the efficacy and safety of a new treatment and form the corner-stone of evidence-based medicine.[1] RCTs published in high impact factor journals such as the New England Journal of Medicine (NEJM) are a key driver of clinical practice and policy decisions. Reporting of both efficacy and safety in these RCTs is equally important as benefit-risk assessment can be done only when both are present in the manuscript.[2] Evidence shows that safety reporting often takes a backseat relative to the reporting of efficacy. Most studies in literature have only analyzed the safety reporting in terms of adequcacy of reporting of specific adverse effects and their severeity.[3],[4],[5] None of the studies have looked at specifics such as therapeutic area, funding support and type of intervention and phases of trials with regards to safety statistics. The extension of the consolidated standards of reporting trials (CONSORT) statement for harm (2004) gives specific guidelines for the reporting of the harm related results of the clinical trials to address the shortcomings in analysis and reporting of harms. The idea is that the readers can assess benefit-harm ratio in the application of medical interventions.[6] to make informed decisions while treating individual patients. Against this backdrop, the present study was undertaken with the primary objective of evaluating safety reporting (relative to efficacy reporting) of all RCTs published in the NEJM over a five-year period. We chose the NEJM for analysis as it multidisciplinary, weekly and with a large worldwide readership and an impact factor of 74.69 in 2019.[7]
Ethics: The study protocol EC/OA 140/2018 was granted exemption from review on 5th Jan 2018 by the Institutional Ethics Committee (IEC). Search strategy and eligibility criteria: All articles published in the NEJM between 2013 and 2017 were electronically searched and extracted from the PubMed database using the keywords 'N Engl J Med' and Clinical trial. Inclusions were RCTs whereas observational studies, follow-up RCTs, review articles, case series and case reports were excluded. Two authors MK and AM assessed the articles independently to confirm that they were truly RCTs. In addition, a hand search was carried out for each article by the same authors independently. Any discrepancy was resolved by consensus or by referral to senior authors (NG and UT). Data extraction: A standardized data collection form was used to extract the following information from the studies- reporting of safety outcomes (with corresponding P values and 95% confidence intervals) both in the abstract and full-text paper, the type of sponsor (whether investigator-initiated or funded by the pharmaceutical industry), the phase of the RCT, nature of the intervention and therapeutic area. Additionally, data was also collected from regulatory sites of the FDA and EMA and clinical trial registries (clinicaltrials.gov and iscrtn.com) if not found in the published paper. A formal email was sent to the corresponding author where more information was needed or if it was missing. Outcome measures These were a) Proportion of RCTs reporting statistics for safety outcomes in the full-text article and/or in their abstract b) Nature/source of funding {Investigator-initiated or Pharmaceutical industry sponsored studies} c) The phase of the study (whether I, II, III, IV, I/II or II/III) d) Nature of the intervention e) Its therapeutic area f) Presence [or lack there of] of statistical parameters for safety and g) The association of safety information with i) Nature of funding ii) Phase of the study iii) Nature of Intervention and iv) Therapeutic area of the intervention. Statistical analysis Both descriptive and inferential statistics were applied to the data. Categorical data (for example therapeutic area) were expressed as proportions and quantitative data (total RCTs) as median (range). Normality of quantitative data was assessed using the Kolmogorov Smirnov test. Association of safety with respect to nature of funding, therapeutic area, with safety statistics (present or absent] was done at a significance level of P < 0.05 using the Chi-square test and a crude odds ratio (with 95% CI) was calculated. All statistical tests were done using PRISM version 7.
Search findings between 1st January 2013 to 31st December 2017, a total of n = 995 articles were identified in the PubMed search. Of these, n = 239 abstracts were excluded as they did not meet the eligibility criteria allowing for 756 full-text articles for review. Of these 110/756 met the exclusion criteria and thus n = 646/756 (85.4%) RCTs were included for the analysis. [Figure 1]. Of these, in 23/646 RCTs, safety outcomes were not applicable because of the nature of the RCTs, for example-simulation-based trial[8] and thus n = 623/756 (82.4%) RCTs formed the final sample.
Reporting of safety outcomes A total of 275/623 (44.1%) articles reported statistics for safety outcome in term of 'P value' (269/623, 43.2%) and/or 95% CI (6/623, 01%). Safety statistics were reported in 117/275 (42.5%) articles both in the abstract and the full text, while 126/275 (45.8%) articles reported only in the full text, and 32/275 (11.6%) articles reported only in the supplement. Type of RCTs A total of n = 299/623 (48%) RCTs were investigator initiated studies (IIS) and n = 324/623, (52%) were pharmaceutical industry sponsored studies (PISS). Most of the RCTs were in phase 3 (368/623, 57.8%), followed by phase 2 (60/623, 9.6%) and phase 4 (48/623, 7.7%). However, in 121/623 (19.4%) RCTs, the phase of the trial was not mentioned in the article or in the clinical trial registries. RCTs associated with drugs as a mode of intervention (n = 403/623, 64.7%) was highest followed by surgery (n = 63/623, 10.1%), medical devices (n = 23/623,3.7%) and vaccines (n = 22/623,3.5%). The largest numbers of the RCTs was in the therapeutic area of cardiovascular system (n = 172/623, 27.6%) which was followed by oncology (n = 101/623, 16.2%) and infectious disease (n = 94/623, 15.1%) [Table 1].
Subgroup analysis of different phase of trials, interventions and therapeutic areas in respect to the RCTs being an IIS or PISS is shown in [Table 2].
Association of presence or absence of statistics for safety with respect to:
The associations of reporting of safety statistics is shown in [Table 3].
In the audit conducted by us on reporting of safety statistics in 623 RCTs published over a five year period, we found that only 43% of RCTs have reported safety statistics in the form of 'P value'or 95% confidence intervals. Moreover, 11% RCTs reported safety statistics only in the supplement indicating that safety reporting is not a priority. We also found higher reporting of safety statistics in Investigator initiated studies [IIS] relative to studies funded by the pharmaceutical industry. About two-third IIS had better reporting of safety outcomes whereas less than one third PISS reported safety statistics. This indicates the efficacy centric approach of the latter whereas both safety and efficacy is paramount for investigators as they treat patients on day to day basis and have to explain both benefit and risk to the patients to enable them to make an informed choice. Additionally differences in such reporting highlights the lack of a consensus on the use of a statistical tool in reporting of adverse events. Although, the CONSORT statement extension for harm-related data published in 2004 provided guidelines in such reporting, the use of statistics in safety data is not clearly defined. The guideline highlighted common poor reporting practices for harm related data of which one of them is; 'Reporting only the adverse events that reach a P value threshold in the comparison of the randomized arms (for example, P < 0.05)'.[7] The answer to better reporting of safety statistics in phase 4 trials over phase 3 trials may lie in the objective of these trials. Phase 4 trials are more concerned about monitoring drug effectiveness in the general population (widespread use) while collecting information about adverse effects which may result in more reporting of safety 'P value or 95% CI'. Phase 4 studies also have higher sample size compared to the phase 3 studies and hence led to greater reporting of safety signals. We found that cardiovascular trials were more likely to report safety statistics. This may be likely because these trials largely used composite outcome (which included safety) as a primary endpoint. Only one-fifth of the oncology trials reported safety statistics. This is surprising, considering the high prevalence of adverse events associated with oncology drugs. One reason may be inability of the investigators to distinguish AEs caused by drugs and disease. In addition the patients may not report adverse events by themselves unless they are specifically requested to report them because they think it is a necessary part of the therapy. Often, safety takes a backseat to efficacy and P value of safety are criticized and tend to be overinterrupted for secondary outcomes. Safety endpoint as a secondary outcome also faced criticism provided multiplicity issues are adjusted, else there could be over interpretation of the P value or the confidence interval adding potential concern for too many outcomes. There is a responsibility of the authors regardless of whether they are from academia or pharmaceutical industry in being transparent in reporting of safety statistics (transparency in reporting). Responsibility of the journal editor or the peer-reviewers to see efficacy and safety are equally well reported in the manuscript. So, the CONSORT should consider revising their own guidelines and to give specific recommendation for reporting of safety statistics. Our study is limited by being a retrospective audit only from a single journal.
Safety reporting in RCTs continues to take a back seat relative to efficacy reporting and is worse for pharmaceutical industry funded studies. Uniformity in reporting statistics for both efficacy and safety outcomes across all studies will help physicians and patients to make an informed choice about treatments. Financial support and sponsorship A grant in aid was provided by Diamond Jubilee Society Trust, Seth GS Medical College and KEM Hospital. Conflicts of interest There are no conflicts of interest.
[Figure 1]
[Table 1], [Table 2], [Table 3]
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|||||||
