Reflections on the recent Cochrane governance crisis

I am a member of Cochrane and these are my thoughts on the current crisis.

About me

My name is Robert Wolff and I joined the German Cochrane Centre (now Cochrane Germany) in 2005: assessed studies for inclusion in CENTRAL, was involved in early stages of translating plain language summaries into German, helped with workshops and with organising the Cochrane colloquium in Freiburg in 2008, and did some EBM-related research.

After leaving the German Cochrane Centre, I continued contributing to Cochrane, for example as:

  • a tutor in workshop on evidence-based medicine and GRADE,
  • a co-author of five Cochrane reviews, and
  • a peer-reviewer of Cochrane reviews as well as colloquium abstracts.

I co-chaired (with Ursula Griebler of Cochrane Austria) the abstract committee for the Cochrane colloquium 2015 in Vienna and (with Cochrane deputy editor-in-chief Karla Soares-Weiser) a plenary session (featuring John Ioannidis, Hilda Bastian and Ben Goldacre) as well as numerous smaller sessions in other colloquia.

Furthermore, I was involved with what later became the Cochrane Cancer Network. I am also an author of PROBAST (a tool to assess the risk of bias and applicability of prediction model studies) and a contributor to the Cochrane Prognosis Methods group.

In short, I am proud to be one of the many contributors that make Cochrane.

Development of the Cochrane Collaboration logo

Cochrane Members for Change

I am also one of the 620 supporters of Cochrane Members for Change, a movement initiated by Jos Verbeek (Coordinating Editor, Cochrane Work), Gerd Antes (former Director, Cochrane Germany), Matteo Bruschettini (Director, Cochrane Sweden), Jani Ruotsalainen (Managing Editor, Cochrane Work), Chris Del Mar (Coordinating Editor, Cochrane Acute Respiratory Infections), Mark Jones (Centre for Research in Evidence-Based Practice (CREBP), Bond University, Australia), Caroline Struthers (UK EQUATOR Centre, University of Oxford), Lotty Hooft (Director, Cochrane Netherlands), Tianjing Li (Co-ordinating Editor, (US Satellite) Eyes and Vision), Gerald Gartlehner (Director, Cochrane Austria), Nicole Skoetz (Senior Editor, Cochrane Cancer), Barbara Nußbaumer-Streit (Associate Director, Cochrane Austria), Nancy Santesso (Deputy Director, Cochrane Canada), Philipp Dahm (Coordinating Editor, Cochrane Urology), Malgorzata Bala (Director, Cochrane Poland), and Jörg Meerpohl (Director, Cochrane Germany).

The initiative identified four major policy issues:

  1. Create a culture of open discussion,
  2. Refocus on the heart of Cochrane,
  3. Increase the involvement of Cochrane members, and
  4. Find a better business model for Cochrane.

My take on the current state of Cochrane, formerly the Cochrane Collaboration

Currently, there appears to be a mismatch between two organisational models, a grassroots science-focussed collaboration and a top-down more business-orientated structure. Generally speaking, a centralised, top-down model will benefit branding and business-related outcomes while a collaborative grassroots approach is more likely to encourage scientific endeavour. In my view, balancing these approaches is the challenge Cochrane is facing.

Focus of Cochrane – Unbalanced scale? Based on: https://commons.wikimedia.org/wiki/File:Scales.png

This is in line with a finding of a review of Cochrane funding by the UK National Institute for Health Research that states: “Centralisation versus decentralisation is a core consideration for Cochrane’s future and should be transparently and swiftly discussed”.

In response to a recent BMJ article by Melanie Newman asking whether “Has Cochrane lost its way?”, Cochrane editor-in-chief David Tovey states that “by all objective measures, Cochrane is thriving”.

I respect David Tovey a lot but wonder about other factors that should be considered, e.g. whether Cochrane currently has the right balance between science and business, between collaboration and top-down. For example, it is good to see a new network initiated in the United States but I noticed that the existing centre “closed down in the spring of 2018, in frustration over management and other centre directors are also contemplating leaving Cochrane”.

There are also concerns about Cochrane’s conflict of interest policy, as discussed by Hilda Bastian and Lisa Bero. Cochrane is currently reviewing the conflict of interest policy.

Expulsion of Peter Gøtzsche

Disagreements surfaced at the recent colloquium in Edinburgh which saw the expulsion of Cochrane Collaboration co-founder Peter Gøtzsche and the resignation of four members of the Cochrane governing board in protest of the decision, namely Gerald Gartlehner, David Hammerstein Mintz, Jörg Meerpohl and Nancy Santesso.

Peter Gøtzsche, former cochranite because of alleged bad behaviour

All involved parties commented on the events:

There was also a fair bit of media attention, e.g.

Furthermore, quite a few comments were published, for example by:

The most comprehensive account of events, including links to all statements and numerous comments as well as details on the legal process investigating the allegations against Peter Gøtzsche can be found on http://www.deadlymedicines.dk/. Note, Peter Gøtzsche maintains this website so some (e.g. selection) bias is possible. However, according to Peter, his only defence against attacks on his scientific and personal reputation is full transparency so I consider this to be the most reliable account of events.

Peter’s website deadlymedicines.dk

In contrast, the statement by the remaining members of the governing board admittedly did not include all facts: “So, here are the facts as we are able to report them. We may be able to tell you more later, we may not. Time will tell.”

During the AGM in Edinburgh, Cochrane governing board co-chair Martin Burton described the process leading to the expulsion of Peter Gøtzsche as “independent, transparent and fair”. In the same AGM, former Cochrane steering group (now governing board) co-chair Jeremy Grimshaw called for calm and stated that “people can contest this” process.

During the meeting, I had my doubts about the process being “independent, transparent and fair” as I was not convinced by the sparse information communicated by the remaining Cochrane governing board. In an article in STAT, Cochrane governing board co-chair Marguerite Koster later confirmed that the “bad behaviour” leading to expulsion of Peter Gøtzsche was in part the use of “wrong letterheads”. This seems very odd to me. If that was the case, why not simply clarify any ambiguity? Also, if there is a conflict between Cochrane CEO Mark Wilson and Peter Gøtzsche, why not seek mediation as suggested by the UK Charity Commission?

There were serious doubts about expelling Peter Gøtzsche from Cochrane, as shown by the tight vote of the governing board (6 for, 5 against, 1 abstaining, Peter not voting). The members who resigned described the expulsion of Peter Gøtzsche as “disproportionate.

I would have liked further transparency, e.g. a clear definition of the ambiguous and unhelpful term “bad behaviour” which apparently some people took as indication for allegations of sexual harassment or similar which had to be clarified later (Spoiler: there was none). Losing one member can happen, losing four indicates more serious problems. I would have applauded the remaining members of the Cochrane governing board to step down and seeking re-confirmation, ideally explaining the reasoning behind their decision to expel Peter Gøtzsche.

Personally, I find it difficult to trust the remaining members as I value the opinion of the four members who stepped down (Gerald Gartlehner, David Hammerstein Mintz, Jörg Meerpohl and Nancy Santesso) who saw the legal counsel’s report and surely resigned for a strong reason.

“Death of a whistleblower and Cochrane’s moral collapse”

In the latest twist of this saddening story, Peter Gøtzsche has now published a book with a detailed account of events. As stated before, this should be taken with a pinch of salt. However, based on an audio recording, the book describes the decision-making behind the expulsion of Peter Gøtzsche (chapter 3) and other internal processes.

The cover of Peter’s latest book

In many respects, I don’t want these details in the public domain. However, I hope this might be the starting point of an open debate on the future direction of Cochrane: Business? Science? Business AND science? Top-down or collaboration? Both?

The book also highlights some potential issues leading to the decision to expel Peter Gøtzsche as well as potential problems regarding the centralised, top-down approach which might be good for business but bad for the collaborative spirit. Given expelling Peter Gøtzsche has wider implications (see above), it is disheartening to see Martin Burton being quoted commenting on the expulsion vote with “It’s a majority of one which I think is personally, I think is disappointing, very disappointed with that indeed … I feel unhappy in many ways”.

Assuming the audio recording is correctly reflected by the book, it also highlights some major problems with the first two of 10 Cochrane key principles, i.e. collaboration and building on enthusiasm of individuals.

Final thoughts

Overall, I am deeply saddened by these events. Based on the best available evidence, it seems like the decision to expel Peter Gøtzsche has been at the very least “disproportionate” and it might have been possible to avoid this as well as the subsequent tensions.

I agree that Peter has a unique style and is quite outspoken. However, I strongly believe that having dissenting voices raising important issues can only benefit a scientific organisation and hence the output of Cochrane.

Furthermore, I am concerned about the direction Cochrane is taking. Obviously, strengthening the business-side is important. However, I feel that it is crucial to balance this with the spirit which the Cochrane Collaboration (!) was founded on. Current tensions show the bad sides of centralism which is very worrying, e.g. the Cochrane central executive team holding back emails, as indicated by two Cochrane centres (Austria and Sweden).

I would urge everyone with an interest in Cochrane to read up on these aspects and the events surrounding the dismissal of Peter Gøtzsche. A lot (!) of material can be found on his website http://www.deadlymedicines.dk/.

Other people discussed these and other aspects related to current state of affairs of Cochrane as well, including

I wrote this text as everybody should have access to the available information so that they can meaningfully contribute to the discussion of the future direction of Cochrane.

Robert Wolff

This text was based on a series of 46 tweets on https://twitter.com/wolffrf. It has been lightly edited for this blog.



Four major policy issues

Most Cochrane members will have felt at best uneasy about the recent events in Cochrane. We also felt uncomfortable and wondered what caused our feelings. The biggest problem we felt was the lack of transparency. Who had been doing what? Who makes the decisions in Cochrane? What can I do as a member of a Cochrane Centre or a Review Group? Many questions were also triggered by the statement of the Governing Board that there will be ‘zero tolerance for bad behaviour’. We fear this will easily lead to ‘zero tolerance for different opinions’.

We are all members of Cochrane because we believe that evidence-based health care is an important asset for public health. Evidence that can be trusted is what we all want to create. We believe that the lack of transparency and trust which we experience at the moment is detrimental to Cochrane and we have to do something about it.

There are four major policy issues that we would like to improve.

  1. Create a culture of open discussion

First of all, we need a culture of open discussion. At the moment, many people don’t dare to speak out because they fear that it will be used against them. We ask the Governing Board and the Cochrane Central Executive Team (CET) to create a forum where interactive discussions can be held. Modern organizations commonly have a freely accessible online discussion platform that can be used by all members.

  1. Refocus on the heart of Cochrane

Authors, members of methods and review groups put enormous effort in producing Cochrane Reviews without payment. The revenues of these reviews finance a large Cochrane CET. In the past, many good investments have been made such as the Game Changer project. We would like to see an increase in support to the heart of Cochrane by for example financially supporting review production, methods development and stakeholder interaction worldwide. Cochrane CET and Cochrane members in various groups around the world should work together to achieve this.

Senior Lady Giving an idea to her Colleagues

  1. Increase the involvement of Cochrane members

We as Cochrane members are very motivated to do our reviews, train our authors, develop methods, assist with difficult analyses or other important tasks for getting our high-quality reviews published. However, our involvement in the governance of the organization is minimal. At the Annual General Meeting only 240 members voted out of a potential 11 250 voters (2%). To support the democratic process in general and to increase ownership in the implementation of new policies in Cochrane, voter participation rates should be increased. The members are the basis of the organization according to the constitution of Cochrane. A good start would be to have a lively debate among members during the coming Governing Board elections and to organize general meetings more effectively.

  1. Find a better business model for Cochrane

Cochrane reviews are sold to subscribers. The European Union has decided that science funded by public money should be published Open Access, which means that it is freely available for the end-user. Cochrane does not have such an Open Access model. Even though content is freely available to a large part of the world, important new reviews are behind a paywall. Also, content that is older than 12 months is free but this is largely not up-to-date. This is different from our promise to be the most reliable, up-to-date and accessible source of evidence. We conclude that in the long run the current business model is untenable and we must find better solutions.

Support

Signed by the following Cochrane Members:

  1. Jos Verbeek, Coordinating Editor, Cochrane Work
  2. Gerd Antes, former Director, Cochrane Germany
  3. Matteo Bruschettini, Director, Cochrane Sweden
  4. Jani Ruotsalainen, Managing Editor, Cochrane Work
  5. Chris Del Mar, Coordinating Editor, Cochrane Acute Respiratory Infections
  6. Mark Jones, Centre for Research in Evidence-Based Practice (CREBP), Bond University, Australia
  7. Caroline Struthers, UK EQUATOR Centre, University of Oxford
  8. Lotty Hooft, Director, Cochrane Netherlands
  9. Tianjing Li, Co-ordinating Editor, (US Satellite) Eyes and Vision
  10. Gerald Gartlehner, Director, Cochrane Austria
  11. Nicole Skoetz, Senior Editor, Cochrane Cancer
  12. Barbara Nußbaumer-Streit, Associate Director, Cochrane Austria
  13. Nancy Santesso, Deputy Director, Cochrane Canada
  14. Philipp Dahm, Coordinating Editor, Cochrane Urology
  15. Malgorzata Bala, Director, Cochrane Poland
  16. Joerg Meerpohl, Director, Cochrane Germany

We asked people to sign this petition if they wanted to join us. By the start of voting, when we closed the petition, 620 people had signed it. We think this is great and totally new in Cochrane. Thank you to everyone who signed!

 

“All effective treatment must be free” — Archibald Cochrane