Truth or Consequences: how to spot a liar and what to do once you catch them

Nothing undermines the credibility of science and scientists more than the retraction a high profile paper. Earlier this year there was a prime example of that when researchers at one of Japan’s most prestigious research institutions, the Riken Center for Developmental Biology in Kobe, had to retract a study that had gathered worldwide attention. The study, about a new method for creating embryonic-like stem cells called stimulus triggered acquisition of pluripotency or STAP, was discredited after it was discovered that the lead author had falsified data.

Publication retractions have increased dramatically in recent years [Credit: PMRetract]

Publication retractions have increased dramatically in recent years [Credit: PMRetract]

The STAP incident drew international coverage and condemnation and raised the question, how common is this and what can be done to combat it? A panel discussion at the World Stem Cell Summit in San Antonio, Texas entitled “Reproducibility and rigor in research: What have we learned from the STAP debacle” tackled the subject head on.

Ivan Oransky, medical journalist and the co-founder of the website Retraction Watch posed the question “Does stem cell research have a retraction problem?” He says:

“The answer to my question is yes. But so does everyone else. All of science has a retraction problem, not just stem cells.”

Oransky says the number of retractions has doubled from 2001 to 2010. One author has retracted 183 times – the record so far – but to break into the top 5 you need to have at least 50 retractions. These come from all over the world from the US to Germany and Japan and most recently Azerbaijan.

Oransky says part of the problem is the system itself. Getting your research results published is critical to advancing a career in science and those kinds of pressures force people to cut corners, take risks or even just falsify data and manipulate images in order to get a paper into a high profile journal. In most cases, journals charge a fee of several hundred to thousands of dollars to publish studies, so they have no incentive to dig too deeply into findings looking for flaws, as it might undermine their own business model.

“Some authors, more than 100, have been caught reviewing their own papers. When the journal they were submitting their paper to asked for the names of recommended reviewers they would submit the names of people who are legitimate reviewers in the field but instead of giving real email addresses they would give fake email addresses, ones they controlled so they could submit their own reviews under someone else’s name.”

What gave them away is that all the potential “reviewers” didn’t first reply and say “yes, I’ll review”, instead they responded by sending back a full review of the paper, raising suspicions and ultimately to detection.

Graham Parker, a researcher at Wayne State University School of Medicine and the editor of Stem Cell and Development says spotting the problem is not always easy:

“As an editor I regard scientific misconduct as fabrication, falsification or plagiarism of data but there are lots of other areas where it’s not always so clear – there are often shades of gray”

He says researchers may make an honest mistake, or include duplicative images and in those cases should be allowed to fix the problems without any stigma attached. But when serious cases of falsification of data are uncovered they can have a big impact by retarding scientific progress and sapping public confidence in the field as a whole.

Jeanne Loring, a stem cell scientist at The Scripps Research Institute and a recipient of funding from CIRM, says the STAP incident was actually a sign of progress in this area. Ten years ago when a Korean researcher named Hwang Woo-Suk claimed to have cloned human embryos it took more than a year before he was found to have falsified the data. But in the STAP case it took a little over a week for other researchers to start raising red flags:

“One of the real heroes in this story is Paul Knoepfler (a CIRM-funded researcher at UC Davis) who takes on difficult issues in his blog. It took Paul just 8 days to post a request for people to crowdsource this study, asking people who were trying to replicate the findings to report their results – and they did, showing they failed over and over again”

Parker said it’s getting easier for editors and others in the field to double check data in studies. For example new software programs allow him to quickly check submitted manuscripts for plagiarism. And he says there is a growing number of people who enjoy looking for problems.

“Nowadays it’s so easy for people to dig very deeply into papers and check up on every aspect of it, from the content to the methodology to the images they use and whether those images were in any way manipulated to create a false impression. Once they find a problem with one paper they’ll dig back through papers in that scientist’s past to see if they can find other problems dating back years that were never found at the time.”

He says that in most cases researchers caught falsifying data or deliberately misleading journals faced few consequences:

“Often the consequences of misconduct are very mild, the equivalent of a slap on the wrist, which does not discourage others from trying to do the same.”

Each panel member says that tougher penalties are needed. For example, in extreme cases a threat of criminal action could be warranted, if the falsified research could lead to serious consequences for patients.

But the panel ended on an encouraging note. Oransky says, for example, that medical journals are now paying more attention and imposing stricter rules and he says there’s even scientific evidence that “doing the right thing might pay off.”

“One study recently showed that if you made an honest error and corrected it publicly not only does the stigma of retraction not apply to you, you don’t get a decrease in your citations—you actually get an increase. So we’d like to think that doing the right thing is a good thing and might actually be a positive thing.”

Policy Matters: Stem Cells and the Public Interest

Guest Author Geoff Lomax is CIRM’s Senior Officer for Medical and Ethical Standards.

In the spirit of Stem Cell Awareness Day, Cell Stem Cell has compiled a “Public Interest” collection of articles covering ethical, legal, and social implications of stem cell research and made it freely available. The collection may be found here.

shutterstock_169882310

The collection covers issues ranging from research involving human embryos to the use of stem cell therapies in patients. For those of you interested in a good primer on the history of stem cell controversies, Herbert Gottweis provides a detailed review of the federal policy debate in the United States. This debate has resulted in inconsistent policy and disrupted research. Gottweis uses this history to support his message that a “comprehensive, and proactive policy approach in this field beyond the quick legal fix” is needed for patients to ultimately benefit from the science.

What I found most interesting about this collection was the focus on stem cell treatments and “tourism.” A majority of the articles address the use of stem cells in patients. This focus is an indicator of how far the field has progressed. Stem cells clinical trials are now a reality and this results in two separated but related considerations. First, is how to make sure prospective patients are well informed should they participate in a clinical trial. Second, how to avoid stem cell “snake oil” where someone is pitching an unproven procedure. These issues are related by their solution that involves empowerment and education of patients and their support networks.

For example, in Stem Cell Tourism and Public Education: The Missing Elements, Master writes:

“It is important for the scientific, medical, ethics, and policy communities to continue to promote accurate patient and public information on stem cell research and tourism and to ensure that it is effectively disseminated to patients by working alongside patient advocacy groups.”

Master’s team found that groups committed to the advancement of good science, including patient advocates and researchers, often lacked basic information about clinical trials and other options for patients. This lack of information may contribute to patients being wooed by those pitching unproven procedures. Thus, the research community should continue to work with patients and advocacy organizations to identity options for treatment.

Another aspect of patient empowerment is what Insoo Huyn refers to as “therapeutic hope” in his piece: Therapeutic Hope, Spiritual Distress, and the Problem of Stem Cell Tourism. Huyn suggests that a supportive system for delivering cell therapies should includes nurturing hope. He writes, “patients might understand when an intervention’s chances of success are extremely remote at best, but may still want to ‘‘give it a shot’’ as long as a beneficial outcome cannot be ruled out as categorically impossible.” Huyn recognizes that well developed early-stage clinical trials are not expected to provide a benefit to patients (they are designed to evaluate safety), but the nature of the therapeutic (often cells) means there may be some real effect.

A third piece by the ISSCR Ethics Taskforce titled Patients Beware: Commercialized Stem Cell Treatments on the Web presents a guide to evaluating therapies. They present five principles that patients, researchers and advocates can rally around to identify credible interventions. The taskforce states:

The guiding principles for the development of the recommended process were that (1) the standards for identifying and reviewing clinics and suppliers should be objective and clear; (2) the inquiry and review process should be publicly transparent and relatively straight- forward for any clinic or practitioner to comply with; (3) conflicts of interest, if any, of the declarant ought to be disclosed to the ISSCR; (4) there should be no actual or apparent conflicts of interest of staff or others involved in the inquiry or review process for any particular matter; and (5) any findings that a clinic fails to meet standards should be communicated in a specific factual way, rather than with broad conclusions of fraudulent practices.

While the Cell Stem Cell Public Interest series covers a range of issues related to stem cells and society, the emphasis on treatments and patients is a reminder of how far the field has come. There is broad consensus that patients, researchers and advocates have roles to play in advancing safe and effective cell therapies.

Geoff Lomax

DISCUSSing iPSC Derivation

Geoff Lomax is CIRM’s Senior Officer for Medical and Ethical Standards. He has been working in the implementation of CIRM’s iPSC Banking Program.

The ability to create high-quality stem cell lines depends, in part, on the generosity of donors. For example, CIRM is sponsoring an induced pluripotent stem cell bank (iPSC bank) that will eventually contain 9,000 stem cell lines. Each of these lines will be generated from tissue donated by 3,000 people suffering from known diseases such as Alzheimer’s disease, autism, hepatitis, blindness, heart disease—and many more. You can learn more about this important initiative here.

shutterstock_182164514

In other countries there are similar initiatives like the one sponsored by CIRM.

We also believe that our donors should have accurate information about how their donated materials will be used, so CIRM has developed variety of tools designed to educate donors. For example, each donor must go through a process called “informed consent” where they are told the details of how iPSC’s are derived and preserved in a bank. We discuss this effort here. In the context of the CIRM bank, new donors are being recruited under ethically and scientifically optimal conditions—where they can be fully informed as to how their cells will be used and how their contribution will spur stem cell research.

There are, however, existing libraries of cell and tissues that have inherent scientific value. For example, they may represent a rare or “orphan” disease. Or, they may be essential for tracking the progress of a patient’s disease over time. These collections have also been developed with the consent of the donor or patient, but, at the time of collection, iPSCs may not have even existed. One question that frequently arises is: can these cells be used for iPSC derivation, research and banking? It is not an abstract concern; CIRM and others often get questions about the adequacy of donor consent for precisely this purpose.

In 2013, CIRM, the NIH and the International Stem Cell Forum (ISCF)/McGill University formed the DISCUSS Project (Deriving Induced Stem Cells Using Stored Specimens) to engage the boarder research community on this issue. Rosario Isasi, a project collaborator from ISCF/McGill University, said that her research tells us that investigators around the world are asking the same questions about use of existing cell lines. To help inform researchers, we started by publishing a report on this very subject. The report included nine points to consider when answering the question of whether existing cell libraries can be used for iPSC research.

We followed this initial effort with a series of meetings and workshops to get reactions to our proposed points to consider. The process culminated with a workshop in March where researchers from around world provided recommendations to the DISCUSS team. Sara Hull, a project collaborator from the NIH, noted that the international perspectives were key to producing a greatly improved product. A major workshop theme was the importance of having an effective management system in place, making sure that the cells are used in a way that is consistent with the donor consent. Participants described a number of specific mechanisms that should be used by the research community to ensure cells are used appropriately. Participants emphasized that having effective systems in place to manage cells and iPSC lines in accordance with donors wishes serves to build trust.

Our workshop report elaborates on specific steps researchers and stem cell banks should take to ensure cell lines are used appropriately. The report also includes a revised set of points to consider based on comments received from meetings and workshops.

The DISCUSS Team looks forward to working with the research community to develop consensus for the responsible use of donated materials in stem cell research.

Geoff Lomax

When Hope Runs up against Reality: Balancing Patient Optimism with Medical Evidence

One of the big concerns among scientists – including many at the International Society for Stem Cell Research (ISSCR) conference in Vancouver, Canada – is that patient expectations about stem cells are often greater than researchers are able to deliver today. That can result in patients in search of a cure heading to overseas clinics that offer unproven therapies.

Megan Munsie – head of the Education, Ethics, Law and Community Awareness Unit at the University of Melbourne in Australia – wanted to find out what happens when patients’ hopes for new treatments come into conflict with scientific views on medical evidence. So she started with a small survey of 16 Australians, patients and patient-caretakers, who had travelled outside Australia for stem cell treatments for a variety of diseases including MS and cerebral palsy.

She says there were a number of interesting findings:

  • They all considered themselves pro-active and well-informed
  • They rejected advice from their own doctor but instead relied on the overseas doctor selling them the treatment for advice
  • They felt they had no choice but to travel overseas because they were running out of time and options in Australia
  • They didn’t consider the health risks, believing that the worst that would happen is that the “treatment” wouldn’t work and they would have spent a lot of money for nothing

Perhaps the most surprising finding was that all of them talked about the “benefits” they gained from going abroad for the treatment, that it gave them a sense of hope even if there was no evidence of medical benefit.

What happens when patients’ hopes for new treatments come into conflict with scientific views on medical evidence?

What happens when patients’ hopes for new treatments come into conflict with scientific views on medical evidence?

This led to a bigger study where Munsie surveyed patients and patient advocates but also stem cell scientists and physicians. Not surprisingly the researchers had a very different view of the subject than the patients.

Researchers/doctors said they felt that patients don’t understand science and don’t appreciate the subtleties of clinical trials

  • They said patients were basing their decisions not on science but desperation
  • They considered overseas providers as dubious, selling hope and taking advantage of a vulnerable patient population

What was interesting, however, is that many doctors said they didn’t try to persuade their patients not to go, instead they chose to respect their autonomy but did at least try to give them the facts so that they could make a decision based on knowledge not ignorance.

When asked why they didn’t tell patients not to go, they said they respected the patients’ need for hope and didn’t want to take that away from them because they had nothing they could offer to replace it.

Munsie says recently some doctors have started offering these kinds of unproven therapies in Australia. She talked to four of them asking how they could justify it. All four said there is a huge unmet medical need and it was better to offer these therapies in Australia than have patients travel to other countries for them. They also said that they felt competent to provide treatment because they had undergone some kind of training or had a license to use equipment needed for the therapy.

Ironically while they all considered themselves legitimate providers of a needed medical therapy – albeit an unproven one – and only interested in the science, they regarded others doing the same as “cowboys” and only interested in the money.

When asked if they would support more regulation of the kinds of therapies they were already offering they said yes, saying that the other doctors who claimed they were “self-regulating” is like “giving the keys to the asylum to the lunatics.”

Munsie says it’s clear that it’s not just patients who could benefit from some guidance on expectations about stem cell therapies.

She says we need to do a better job of managing patient expectations without robbing them of a sense of hope, perhaps by offering them information that is more tailored to their particular needs.

We also need to manage what she called “the unbridled enthusiasm of providers” who are offering speculative treatments as “medical practice”. That might take regulatory change by the government.

She says it’s difficult to strike a balance between hope and scientific evidence, in maintaining a patient’s sense of optimism while acknowledging the reality of the science and the risks posed by unproven treatments.

Kevin McCormack

Arlen Spector: Science Policy Driven by Evidence, not Rhetoric

Geoff Lomax is CIRM’s Senior Officer to the Standards Working Group

In August 2001, I was frantically working to put the final touches on my doctoral dissertation focused on ethical and policy considerations related to the use of genetic testing for disease risk. I always viewed the ethics policy discussion as rather nuanced and academic, so it was quite a surprise to hear President Bush state in his first prime time address that: Research on embryonic stem cells raises profound ethical questions. I knew nothing about the field of stem cell research at the time, but I decided to take notice given that research ethics had come to the forefront of the national policy discussion.

Sen. Arlen Spector

Sen. Arlen Spector

For an aspiring student of research policy, it was essential to follow how the discussion played out. I had experience in a number of state and national legislative campaigns and was keenly aware how rhetoric can trump substance in policy debates.

In 2001 -2002, the most substantive policy deliberations were taking place in the United States Senate. One senator in particular, Arlen Specter, stood out because of his cogent questioning designed to elicit answers to core policy questions. In a 2002 hearing Senator Specter focused on whether under the Bush policy there a sufficient number of stem cell lines available for the required research. It seem quite remarkable that so much attention was being given to whether particular cell lines were sufficiently robust for research on Alzheimer’s and Parkinson’s disease. One of the most striking aspects of these deliberations was Senator Specter’s effort to move past rhetoric and get a complete picture of the policy issues.

This quality in Senator Spector extended to all his work, as the Los Angeles Times noted in their obituary of Spector after his death this week. They wrote, “he habitually asked probing questions of nominees from both parties instead of succumbing to the rhetorical approach favored by his colleagues.”

He brought this probing approach to NIH budget deliberations to document the value of disease treatment and prevention. The result was a doubling of the NIH budget over five years and Francis Collins, director of the National Institutes of Health (NIH), described Specter to The Scientist as “a towering champion for biomedical research and the mission of the NIH.”

Former CIRM Governing Board chairman Robert Klein described Spector as a man of character and conviction. I hope one legacy of Senator Spector’s career is the conviction that issues of science and research policy should be informed by substantive deliberations. Science is fundamentally driven by rules of evidence, and science policy should be no exception.

Geoff Lomax

The First International Vatican Adult Stem Cell Conference: A Gound-Breaking Achievement

John Wagner is Professor and Director of the Division of Pediatric Blood and Marrow Transplantation at the University of Minnesota Medical School. He performed the first umbilical cord blood transplant to treat a child with leukemia and pioneered the use of stem cells in the treatment of the skin disease epidermolysis bullosa. He is also a member of CIRM’s standards and grants working groups.

It was a fascinating week in Rome where I was one of several invited speakers attending the Vatican sponsored: “Adult Stem Cells: Science and the Future of Man and Culture.”

Overall, I was personally ‘moved’ by the uniqueness of this event that mixed science and history. Regardless of anyone’s personal religious beliefs, the Catholic Church does have an undeniable place in the history of western civilization with a legitimate obligation at least for Catholics and Catholic institutions around the world to try to understand the broader impact of science that no longer simply observes nature but now has the capacity to alter it.

My specific role was to highlight some of the ‘lessons learned’ from ‘our’ collective experiences with blood-forming stem cells, the only stem cell therapeutic with proven regenerative medicine potential. Those lessons learned include best practices in ethical conduct in high-risk clinical experimentation in vulnerable populations, the obstacle of the immune response and the need for earlier considerations on issues of access to new technologies and public education on reasonable expectations for phase I studies.

In addition to my contribution, there were a host of other presentations focusing on public confusion on what stem cells are and the public’s understanding of the regenerative capacity of embryonic versus adult stem cells, the impact of politics particularly in the United States, sources and types of adult stem cells, technological advancements in organ and tissue regeneration, regulatory requirements for stem cell therapeutic manufacture from industry’s perspective, and clinical applications in neurological, cardiovascular, autoimmune diseases. These talks were intermixed with moving testimonials from patients who are looking to stem cells as a source of future therapies. Intriguingly, the Conference was also about ‘building bridges’ between the Church and scientists to enhance bioethical-humanistic-cultural considerations of stem cell research and develop strategies for educating the present and future generations of students in the life sciences.

As would be expected, the meeting had a strong focus on the promise of adult stem cells with relative silence on the impact of embryonic stem cells. Statements by others on the ethics of stem cell research were also predictable—“the destruction of the human embryo is never justifiable regardless of any claim to therapeutic benefit“(although it was often stated that any benefit from embryonic stem cells has yet to be demonstrated).

But, there were indeed surprises as well: 1) the Church voiced its desire to open a dialogue with scientists and educators in the life sciences, and 2) the Church announced its unprecedented financial investment into promoting adult stem cells and their role in regenerative medicine.

While we might have wished for more detail on how the Church planned to build such bridges and open dialogue with scientists, the intent was seemingly sincere. In fact the Church had already identified ‘partners’ whose work would itself promote adult stem cell research including neural stem cells derived from recently aborted fetuses, which are already in clinical trials for several conditions. The Church also clarified that its financial investments were principally focused on public education, such as through the Stem For Life Foundation, the philanthropic arm of NeoStem, Inc. which co-sponsored the Conference.

Despite the fact that my views might differ from those of some of the presenters, I was amazed by the uniqueness of this event that placed the science of today into an historical context. Consider the venue and the associated events that included a dinner reception at the Casina Pio IV (the Vatican Academy of Science), a concert at the Basilica of St John Lateran, and an audience with Pope Benedict XVI. At the end of the day, I had the feeling that perhaps this self-proclaimed ‘ground breaking’ achievement was more than self-aggrandizement. Perhaps, the Church indeed strived to promote dialogue and build bridges with those deeply involved in the science of stem cells and its clinical translation into meaningful therapeutics. For sure, the moral status of the embryo may be front and center in terms of the ethical issues, but there are other issues as well that need further discussion regardless of the stem cell source. The creation of such a dialogue on the broader philosophical and humanistic aspects of science in general and stem cell research in particular must be a good thing.

If we agree that the goal of science is the discovery of new knowledge and truth—and en-light-enment—we must also acknowledge that there are risks, such as those realized with the nuclear experiments of the last century. And, recall in Mary Shelley’s Frankenstein: A Modern Prometheus, that the not too distant cousin of light is fire. So, perhaps, it is not unreasonable to listen openly and dialogue with others who may have opposing but reasoned views, with the understanding that you need not completely accept all that others speak. From there new knowledge and a higher level of discourse may take place. If successful, that, in and of itself, would be a ‘ground breaking’ achievement.

John Wagner