PDF
ePub
Citation
Print
INTRODUCTION
Investigations of stem cells (SC) have been rapidly growing over the past few decades, but they have been shrouded in controversy fueled largely by the lack of quality data to support claims made by the mass media as well as businesses exploiting SC. In particular, clinical applications in neurology are increasing exponentially, with the severe motor and neurological symptoms associated with neurological diseases and the restricted options for treatment or cure turning patients to treatment modalities involving SC.
However, a pressing issue is autologous transplants being offered without sufficient evidence.1 Recent studies have demonstrated numerous instances of malpractice and harm done to patients by those who have taken advantage of global SC tourism and commercialization.2 Particularly in Korea, there is concern that people are readily able to access SC treatments in spite of poor peer-reviewed data.3 It is particularly concerning that as of 2014, four SC-based therapy products have been granted marketing authorization in Korea.4 This illustrates the importance of the public having access to information regarding the scientific and clinical evidence for these treatments. However, the likelihood of selective reporting is higher in a more-popular field,5 and there has been little evidence-based research to support the clinical safety of treating neurological diseases using SC. Because of the issues surrounding the field, many countries have pushed for a global standard and framework in SC research.678 Organizations such as the International Society for Stem Cell Research were established to create a forum for education and developing guidelines for embryonic SC research.7
There has been considerable debate on the regulation of SC in Korea, especially after a particular investigator (Woo Suk Hwang) was found in 2005 to have conducted fraudulent research. This event prompted the Korean government to establish the National Bioethics Committee and place a moratorium on embryonic SC research up to May 2009.9 The Korean Bioethics and Safety Act was also enacted, and has since undergone revisions. SC therapies are currently regulated by the Ministry of Food and Drug Safety (MFDS) in Korea and must be approved in accordance with the Pharmaceutical Affairs Act. Clinical trials of new products must be conducted at designated clinical-trial institutions (including hospitals).4
With the rapid growth of clinical trials involving SC, it is important to know whether the obtained information is prone to publication bias and whether findings are reported regardless of whether they are favorable or unfavorable. Selective reporting can lead to a potentially biased view of trials and results that could hinder the growth of this field in the scientific community and, more importantly, could result in patient harm. Despite events such as the Hwang incident, there are still clinical trials that are not registered and do not publish results.
Registries
Studies have indicated that the registration of randomized clinical trials is positively correlated with improvements in the reporting of results.10 However, studies have shown that some trials are registered after the first subject is enrolled, and that the amount of interventional trials registered before enrollment decreases over time.11 This combined with the failure to provide adequate information and update the status of registered clinical trials can potentially introduce bias that has adverse effects on the transparency, safety, and ultimately trust of the clinical studies.
Concerns about selective outcome reporting and publication bias have led to the implementation of clinical research registration policies and guidelines. The International Committee of Medical Journal Editors requires that authors register their clinical trial when they are wanting to publish results therefrom. The Declaration of Helsinki states that every clinical research study must be registered in a publicly accessible database before the first subject is recruited. These and other efforts are leading to many regulatory and funding entities requiring clinical trials to be registered by law, and research guidelines and public registration systems have now been established in many countries.11
The present study explored two of these databases: clinicaltrials.gov and the Clinical Research Information Service (CRIS). clinicaltrials.gov was created by the US Congress in 1997 to provide information and access to clinical trials. The US Food and Drug Administration Amendments Act then expanded this mandate by requiring sponsors of applicable clinical trials to register and report summary results. The CRIS was developed by the Korea Centers for Disease Control and prevention (KCDC) in 2010 with support from the Ministry of Health and Welfare (MOHW) as one of the primary registries in the World Health Organization (WHO) International Clinical Trials Registry Platform to prevent the ambiguous identification of clinical trials. While registration of clinical research into a public registry system is not legally mandated in Korea, clinical research funded by the MOHW is required to be registered in the CRIS, and the MFDS recommends registration.
This study analyzed the registering of trials and reporting of results by searching clinical trial registries and journal databases, with the aim of identifying the prevalence of selective publishing.
METHODS
Criteria for reviewable studies
The intervention in potentially reviewable studies was the use of SC therapy, but this excluded studies using solely growth factors, solely proteins without SC, or any non-SC-based therapy. The type of SC used in each study was listed.
Analysis and classification were performed on trial registration, whether there were posted or published results for registered trials, and whether publications related to SC were registered.
Search strategy
The following electronic databases were searched: clinicaltrials.gov, the CRIS, and PubMed. Only human clinical trials were included. There were no limitations on trials or articles based on starting date, completion date, or published date.
Study selection and data extraction
The specific strategies applied to each database are presented below.
clinicaltrials.gov
The following terms were entered into the “Other Terms” search bar at clinicaltrials.gov: “stem cell,” “progenitor cell,” “bone marrow,” “mesenchymal stem cell,” and “umbilical cord blood.” The “Study Type” filter under the advanced search option at clinicaltrials.gov was set to “Interventional Studies.” To ensure that Korean studies were identified, the “Locations” filter was set to “Republic of Korea.” Each of the filtered trials was reviewed for neurological relevancy. Studies were excluded when the neurological problem resulted from any form of cancer or cancer treatment, diabetes, medication side effects, or bone disease. To determine whether or not there were published result articles for completed trials, the registration numbers were searched in all National Center for Biotechnology Information (NCBI) databases. Any relevant articles were looked through manually and then matched with trials accordingly.
Clinical research information system
The following terms were searched for on the CRIS website by inputting them in the “Scientific Title” search bar: “stem cell,” “progenitor cell,” “bone marrow,” “mesenchymal stem cell,” and “umbilical cord blood.” The “Study Type” filter under the advanced search option on the CRIS website was set to “Interventional Studies.” Since the CRIS website is run by the Korean government for Korean studies, no additional filter was set for selecting only Korean trials. Each of the filtered trials was reviewed for neurological relevancy. Studies were excluded when the neurological problem resulted from any form of cancer or cancer treatment, diabetes, medication side effects, or bone disease. To determine whether or not there were published results articles for completed trials, the registration numbers of the trials were searched in all NCBI databases. Any relevant articles were looked through manually and then matched with trials accordingly.
PubMed
A list of search terms was created based on the Medical Subject Headings (MeSH) database. The additional word “Korea” was added to the PubMed Search Builder to identify only relevant searches. The following MeSH terms were searched for on PubMed: “stem cell transplantation,” “fetal blood,” “bone marrow,” and “mesenchymal stem cell transplantation.” The results were filtered to include only human clinical trials and results articles. Only neurologically related trials were then selected. Studies were excluded when the neurological problem resulted from any form of cancer or cancer treatment, diabetes, medication side effects, or bone disease. The reference lists in the articles identified in the PubMed search were manually searched for any additional relevant articles.
RESULTS
Our searches of clinicaltrials.gov and the CRIS website identified 17 completed trials, with results articles having been published for 5 of them.1213141516 Our study found 16 publications,12131415161718192021222324252627 of which 1 was a review article,21 1 was a protocol,26 and 8 contained registered trial information.1213141516222326
clinicaltrials.gov (Fig. 1, Table 1)
As at October 28, 2017, 198, 195, 147, 56, and 38 registered trials were identified at clinicaltrials.gov using the search terms “stem cell,” “progenitor cell,” “bone marrow,” “mesenchymal stem cell,” and “umbilical cord blood,” respectively.
Stem cells
Nineteen of the 198 registered trials identified as being related to SC were related to neurological issues: 7 were completed, 7 were recruiting, 3 were active but not recruiting, and 2 were not yet recruiting. Two of the seven completed trials (NCT01363401 and NCT01193660) had posted results on the registry, while the other five had not. Three of the seven completed trials had published results (NCT01363401, NCT01193660, and NCT00911365).131415 However, only one of these three completed trials (NCT01363401) had the publication link on the registry page,13 and the published articles for the other two trials were found via a manual search of PubMed using the trial registration number.1415 However, one more article was found22 from a manual search of the reference list of one of the articles15 with a matching registration trial number (NCT00911365). These two articles1522 described the study as a randomized, double-blind, placebo-controlled trial, whereas it was registered as a randomized, single-blind, parallel-assignment trial on the registry (NCT00911365). One recruiting trial (NCT01716481) was matched to a CRIS trial (KCT0000574), but was listed as not yet recruiting on the CRIS website with conflicting anticipated completion dates: February 2016 on the CRIS website and December 2017 at clinicaltrials.gov. A search of PubMed for the trial number revealed that a protocol paper26 was matched to this trial (NCT01716481). Three of the completed trials without published results had been completed more than 17 months previously (NCT01769872, NCT01624779, and NCT01297218), with two of them completed more than 26 months previously (NCT01624779 and NCT01297218). One trial listed as “active, not recruiting” (NCT02378974) had already passed its anticipated completion date of May 2017.
Umbilical cord blood
Thirteen of the 38 registered trials identified as being related to umbilical cord blood were identified as being related to neurological issues: 8 were completed, 2 were recruiting, 1 was active but not recruiting, and 2 were withdrawn prior to enrollment. Although none of the completed trials had posted results on the registry, one of them (NCT01528436) was matched to an article16 in the search of the MeSH database for “stem cell transplantation.” Three completed trials (NCT01639404, NCT01601158, and NCT01991145) matched CRIS trials (KCT0000483, KCT0000445, and KCT0000950, respectively). However, these three trials were listed as recruiting on the CRIS website. Four completed trials without published results were listed as having been completed more than 17 months previously (NCT01639404, NCT01601158, NCT01884155, and NCT02025972), with two of them having been completed more than 26 months previously (NCT01639404 and NCT01601158).
Clinical research information system (Fig. 2 , Table 2)
As at October 28, 2017, 14, 1, 4, 8, and 5 registered trials were identified on the CRIS website using the search terms “stem cell,” “progenitor cell,” “bone marrow,” “mesenchymal stem cell,” and “umbilical cord blood,” respectively. Applying the search terms “progenitor cell,” “bone marrow,” and “mesenchymal stem cell” yielded no new additional trials after searching the database using the search terms “stem cell” and “umbilical cord blood.”
Stem cells
Three of the 14 registered trials identified as being related to SC were to neurological disorders: 2 were completed (KCT000 1429 and KCT0000879) and 1 was not yet recruiting (KCT 0000574). There were posted results on the registry for the two trials listed as completed. Two of the trials (KCT0001429 and KCT0000574) had a publication posted under “Study Results and Publication.” However, the publication for KCT 0001429 was not found by searching PubMed, and the paper was a review article rather than reporting results. A registration number was also not listed in this article.21 Furthermore, the publication for KCT0000574 was a study protocol.26 One published paper12 was later found during the PubMed search that had a registration number that matched the trial listed as recruiting (KCT0000879). It is noteworthy that the notyet-recruiting trial had passed its projected completion date. This not-yet-recruiting trial also matched a trial at clinicaltrials.gov (NCT01716481) that was listed as recruiting.
Umbilical cord blood
Three of the five registered trials identified as being related to umbilical cord blood were related to neurological disorders, all of which were listed as recruiting (KCT0000950, KCT 0000483, and KCT0000445). All of the trials had extended beyond their anticipated completion dates by more than 26 months. The three trials matched clinicaltrials.gov trials (NCT01991145, NCT01639404, and NCT01601158, respectively). Although these trials were listed as recruiting on the CRIS website, they were listed as completed at clinicaltrials.gov.
PubMed (Table 3)
As at October 28, 2017, 150, 20, 22, and 4 articles were identified using the MeSH terms “stem cell transplantation,” “bone marrow,” “mesenchymal stem cell transplantation,” and “fetal blood,” respectively.
Stem cell transplantation
Eight of the 150 articles identified as being related to SC transplantation were related to neurological issues.1213161718232425 Four articles had registered trial information, of which one12 matched a CRIS trial (KCT0000879) and three131623 matched clinicaltrials.gov trials (NCT01363401, NCT02983708, and NCT01528436). Two of the articles matching clinicaltrials.gov trials (NCT01363401 and NCT01528436) were found during our search of the registry databases. We found an additional article19 related to SC therapy from the references of one article,18 but this additional article did not include trial registry information.
Bone marrow
Only 1 of the 20 articles identified as being related to bone marrow was related to neurological issues.20 This article did not have a registration number listed and was not matched to any registered trial identified in our search.
Mesenchymal stem cell transplantation
Only 1 of the 22 articles identified as being related to mesenchymal SC transplantation was related to neurological issues.22 This article had a registration number listed and was matched to a clinicaltrials.gov trial (NCT00911365); however, the trial did not list that article in its published results.
Fetal blood
Two of the four articles identified as being related to fetal blood were related to neurological issues.1427 One article14 had a registration number listed and was matched to a clinicaltrials.gov trial (NCT01193660); however, the trial did not list that article in its published results.
DISCUSSION
In order to identify reporting or publication bias, we examined whether there are reported results and published articles for registered clinical trials. We found that many registered trials listed as being completed did not have published results available on the registry databases. Many had passed the 12-month time frame recommended by the WHO.28 The reporting of study results, regardless of outcome, is essential to advancing research and ensuring the safety of participants in clinical trials. The timing of registration and results reporting is important for ensuring that timely results are accessible to researchers and also the general public. The WHO guidelines state that the main findings of clinical trials should be submitted for publication in a peer-reviewed journal within 12 months of study completion and be published in an open-access format unless there is a specific reason why open access cannot be used. However, the present study has revealed that many trials do not follow these guidelines—half of the published articles did not have registered trial information available.
Furthermore, discrepancies were identified between the information for one of the trials (NCT00911365) and its linked articles.1522 The registered trial was listed as having a randomized, parallel-assignment, single-blinded design, while the articles linked to the study referred to it as having a randomized, double-blinded, placebo-controlled design. This type of discrepancy is very surprising given that the consistency of information on registries and publications is essential for facilitating the transparency necessary for better insight and understanding new developments.
In other countries, such as the US, certain steps have been taken by enacting the Clinical Trial Registration Policy of the International Committee of Medical Journal Editors, which mandates the registration of clinical trials prior to manuscript submission. However, a 30-month study that assessed the results of this policy on the completeness of registration, changes in data entries, inadequate data, and willingness of researchers to disclose information found that more control procedures are needed.29 Another move toward transparency was the enactment of the US Food and Drug Administration Amendments Act in 2007 that requires sponsors of applicable trials to register and report summary results at clinicaltrials.gov. Although this intervention was designed to increase compliance with industry standards, a recent study showed that compliance with results reporting is generally poor, partly due to the presence of various extensions and exemptions.30 These reports illustrate that considerable improvement is necessary.
In Korea, the KCDC established the CRIS with support from the MOHW in 2010. Although the registration of clinical trials is not mandatory in Korea, the MOHW requires MOHW- funded clinical trials to be registered in the CRIS. The MFDS also recommends the registration of clinical trials after MFDS approval.11 However, despite the MOHW issuing a regulation that requires clinical research funded by them to be registered, there is no official government legislation mandating this. Although the present study briefly examined the relationship between registration and publication, many issues need to be explored further in the field of SC in neurological problems. Analyzing the quality of registered data and publications on SC would be very helpful for determining the usefulness of data obtained in the relevant studies.
This study was subject to some limitations. Because we only analyzed registered trials, it is possible that some relevant trials were overlooked. Based on a study of CRIS reporting, we suspect that many trials regulated by government entities are not registered.11 Regarding the searches and data extraction performed in the present study, although words encompassing broad fields of study were used to maximize study inclusiveness, it is possible that not all relevant studies related to neurological disorders and SC were found. Furthermore, although we used what we deemed were the two most relevant registries for our search, it is possible that studies registered on other databases were missed. PubMed was the only database searched for publications, because it is the most popular engine used by the general public. We also searched the Web of Science after completing this study, but that did not identify any additional relevant publications.
The time needed to prepare publications on studies is an important factor, and so we categorized time frames of 17 and 26 months after the listed trial completion date as delayed and very delayed publication times. Considering that the WHO recommends a maximum postcompletion period of 12 months for submitting a publication, we considered our two time frames to be adequate. However, it is possible that trials conforming with our time frames were excluded due to publication delays.
In conclusion, SC-based therapies for treating neurological illnesses are currently being explored, but most remain unproven in the clinical setting. We have identified widespread underregistration and underreporting of SC clinical trials in Korea, and suspect that this, combined with the implied reporting bias, is not restricted to Korea. Considering the importance of SC research in human diseases, rigorous registration and reporting processes need to be implemented as soon as possible.
XML Download