Minutes of first Cochrane Members for Change open meeting in Krakow, April 3rd, 2019

Participants: 11 in person/ 10 online

View of Krakow from St.Mary’s church

To begin with, Jos Verbeek and other comrades from Cochrane Members for Change explained the origin of the movement. The main theme of the discussions following from this focused on the question “Is this group needed?”

  • We began by asking if there are other groups in Cochrane that already fulfill the function of Cochrane Members for Change. That is, is there a grassroots membership platform for bringing things to the attention of other groups (e.g. Council and Governing Board)?
  • Some Cochrane members attending the meeting expressed the feeling that it is not OK to openly express criticism against Cochrane. There is fear that some doors (in academia) will be closed for such members and this is problematic especially in low-and middle-income countries.
  • One goal of this forum could be to alleviate fear. People are removing Cochrane logos from teaching materials so as not to appear breaching the spokesperson policy.
  • Since the events in Edinburgh there have been some improvements in the Governing Board’s dealings towards members (tone, calls, involvement), e.g. center directors’ meetings.
  • But there were problematic events such as censoring of emails sent to mailing lists.
  • Also, not all members feel represented in existing structures – but is everyone aware of all structures and how to interact?
  • This group provides a no-threshold forum for everyone to raise voice/concerns.
Attendees at the Cochrane Governance Meetings in Krakow

How are we going to continue?

  • Goal: transparency should be improved (openness, reduction of fear to communicate, ways of communications, decisions)

Is the group officially recognized? Implicitly yes. How much can we expect to be listened to? Key question: how much weight will the board give to the effort of this group? As we have seen, the Governing Board has already started implementing suggestions the group has made, so we can assume it is listening.

Using petitions to emphasize the group’s initiatives is a possible mechanism but it is a rather cumbersome instrument. There used to be a mechanism in Cochrane to raise and rate initiatives according to perceived importance, but this was limited to software development only and even that was discontinued.

  • The group was surprised by the paucity of comments raised at the General Meeting of the Krakow mid-year meeting. This could be because people are experiencing fatigue, or they expect that their concerns would not be addressed anyway.
  • Process – lack of understanding in Cochrane – if issues are raised then they need to be discussed. Many discussions are not conducted openly and without real consequences. For example, with the Code of conduct it is not clear whether it is intended to defend people or to defend Cochrane as an organisation but the overall feeling was that its development process consisted of good discussions and a democratic process. Nonetheless, we should check if feedback will be incorporated.

In the General Meeting in Krakow when asking what the Governing Board had learned from the crisis and how they acted, the answer felt like it was intended to extinguish any further questions.

Cochrane leaders in Krakow

Where does fear come from? Perhaps from asymmetry in hierarchy in the organization and an unwillingness to admit mistakes. It would be good if the Governing Board would show a more collaborative approach and tone, or admit that some mistakes have happened.

For the success/usefulness of this group it depends how discussions of our group are incorporated in the final products, meaning policies, processes, and so on.

There was a common feeling that a lot of issues are not receiving enough attention by the leadership. Some people felt that the discussion on the code of conduct was unwieldy because it is highly inefficient to develop a document in a group consisting of hundreds of people.

There was some disagreement on whether we, as Cochrane, are too big to have direct democracy – others think this could easily be facilitated by voting on documents etc. Cochrane certainly has the means to organize the necessary mechanisms to achieve this.

What is the mechanism to provide feedback on activities and to whom? Although many felt that this was senseless – no one mentioned this afterwards – why? Where should we put criticism?

One participant raised the comment: Is Cochrane as a whole perceived to have a conflict of interest with drug companies such that the latter would have undue influence on reviews like the one on HPV vaccines. How can we react or move forwards here? Is this group a way to be aware of this problem? If you complain about the conduct of how a Cochrane Review is worded, produced or whatever, there is genuine fear of punishment by Cochrane leadership and consequently academia more broadly. There was agreement that Cochrane could still improve openness of discussion to erase concerns. Suspicions are raised in papers, from external people – but there is no evidence that Cochrane is favouring industry. It was felt that conflict of interest with regards to the drug industry is not an issue and Cochrane strives to set high standards here.

Attendees at the Cochrane Governance Meetings in Krakow

Is there an entity within Cochrane with a control mechanism to implement and enforce if editorial groups do not address their own CoI? – Yes CEU and EiC.

Conspiracy theories have been nurtured by the non-transparent process – and this has not been acknowledged by the Governing Board.  Lack of acknowledgement has created this atmosphere.

This group needs to focus on discrete actions. One suggestion: reach acknowledgement that there have been mistakes, Cochrane’s image has been seriously harmed. Main issue also for others that this was not acknowledged. What we could do? Bring it up again? Acknowledgement that communication should not take so much time. Suggestion: Ways to raise questions anonymously – maybe such a system could help.

Overarching themes of the discussion:

  • Need for a culture change to remove fear of speaking up and increase transparency.
  • More accessibility of Council and Governing Board needed.
  • More debate and feedback from community needed.

To do/wish list:

  • One page directing concerns: Increase ease and transparency of knowing who to go to when you have an issue.
  • Mechanism for raising concerns anonymously: Given people are fearful to speak up in a way that their name/face is attached to the comment/question there should be a way/mechanism to raise concerns anonymously and/or to escalate concerns if one feels ignored when they have used established channels.
  • Accessibility of leadership: More accessibility to Council and Governing Board since people are fearful of putting their hand up and asking questions at large meetings.
  • Frequency and mode of updates from leadership: There is a need to have more information updates on ongoing projects that are not hidden in PDF reports or are news items after all the work and decisions are made. This would allow people to provide feedback or raise concerns earlier in the process, rather than at the end of a project.
  • Improve governing communication: Culture change to one of openness and transparency and willingness to collaborate and work with the Cochrane community i.e. the Governing Board may not always know best. Requires a change in tone as communication has been through PDF agendas/minutes and lawyer vented statements (presenting one narrative, not facts), need to be more open, more frequent, more digestible. Also need to admit to mistakes or at least say what the lessons learnt were. Acknowledge that things aren’t 100% fantastic all the time, that there are struggles and there has been a tarnish on our reputation.
  • Pauses for feedback: There is a desire for more pauses to get feedback from community on major issues (hiring editor-in-chief, budget spending) and transparency to how that feedback will be used. Could be that more explaining of the thinking behind things and process is needed in the updates (e.g. when new editor-in-chief is hired, explain process of getting there).
  • More clarity on Spokesperson policy: There should be an update and training for spokesperson policy. There should be a page of tips, a video, clear contact person if you aren’t sure of the policy or what to check before posting or saying something.

The remaining questions for the CMFC group would be:

  • How does the group want to be officially recognized?
  • What direct supports can be offered centrally? (would we like an open Slack group which has built-in conference meeting capabilities?)
  • What are some practical recommendations for the colloquium? (e.g., anonymous questions for general meeting, discussion sessions with governing board and council, room for a CMFC meeting)

Final conclusion: There are enough reasons to continue with Cochrane Members for Change. We will communicate this to the Council and the Governing Board and put forward the concerns with the discussion in Cochrane as described above. We will meet again via video in May, 2019. Click this link (activated when the time is decided) to join the meeting at the allotted time.

Minutes by: Barbara Nussbaumer-Streit, Holger Schünemann, Muriah Umoquit and Jani Ruotsalainen


Reflections on the recent Cochrane governance crisis

I am a member of Cochrane and these are my thoughts on the current crisis.

About me

My name is Robert Wolff and I joined the German Cochrane Centre (now Cochrane Germany) in 2005: assessed studies for inclusion in CENTRAL, was involved in early stages of translating plain language summaries into German, helped with workshops and with organising the Cochrane colloquium in Freiburg in 2008, and did some EBM-related research.

After leaving the German Cochrane Centre, I continued contributing to Cochrane, for example as:

  • a tutor in workshop on evidence-based medicine and GRADE,
  • a co-author of five Cochrane reviews, and
  • a peer-reviewer of Cochrane reviews as well as colloquium abstracts.

I co-chaired (with Ursula Griebler of Cochrane Austria) the abstract committee for the Cochrane colloquium 2015 in Vienna and (with Cochrane deputy editor-in-chief Karla Soares-Weiser) a plenary session (featuring John Ioannidis, Hilda Bastian and Ben Goldacre) as well as numerous smaller sessions in other colloquia.

Furthermore, I was involved with what later became the Cochrane Cancer Network. I am also an author of PROBAST (a tool to assess the risk of bias and applicability of prediction model studies) and a contributor to the Cochrane Prognosis Methods group.

In short, I am proud to be one of the many contributors that make Cochrane.

Development of the Cochrane Collaboration logo

Cochrane Members for Change

I am also one of the 620 supporters of Cochrane Members for Change, a movement initiated by Jos Verbeek (Coordinating Editor, Cochrane Work), Gerd Antes (former Director, Cochrane Germany), Matteo Bruschettini (Director, Cochrane Sweden), Jani Ruotsalainen (Managing Editor, Cochrane Work), Chris Del Mar (Coordinating Editor, Cochrane Acute Respiratory Infections), Mark Jones (Centre for Research in Evidence-Based Practice (CREBP), Bond University, Australia), Caroline Struthers (UK EQUATOR Centre, University of Oxford), Lotty Hooft (Director, Cochrane Netherlands), Tianjing Li (Co-ordinating Editor, (US Satellite) Eyes and Vision), Gerald Gartlehner (Director, Cochrane Austria), Nicole Skoetz (Senior Editor, Cochrane Cancer), Barbara Nußbaumer-Streit (Associate Director, Cochrane Austria), Nancy Santesso (Deputy Director, Cochrane Canada), Philipp Dahm (Coordinating Editor, Cochrane Urology), Malgorzata Bala (Director, Cochrane Poland), and Jörg Meerpohl (Director, Cochrane Germany).

The initiative identified four major policy issues:

  1. Create a culture of open discussion,
  2. Refocus on the heart of Cochrane,
  3. Increase the involvement of Cochrane members, and
  4. Find a better business model for Cochrane.

My take on the current state of Cochrane, formerly the Cochrane Collaboration

Currently, there appears to be a mismatch between two organisational models, a grassroots science-focussed collaboration and a top-down more business-orientated structure. Generally speaking, a centralised, top-down model will benefit branding and business-related outcomes while a collaborative grassroots approach is more likely to encourage scientific endeavour. In my view, balancing these approaches is the challenge Cochrane is facing.

Focus of Cochrane – Unbalanced scale? Based on: https://commons.wikimedia.org/wiki/File:Scales.png

This is in line with a finding of a review of Cochrane funding by the UK National Institute for Health Research that states: “Centralisation versus decentralisation is a core consideration for Cochrane’s future and should be transparently and swiftly discussed”.

In response to a recent BMJ article by Melanie Newman asking whether “Has Cochrane lost its way?”, Cochrane editor-in-chief David Tovey states that “by all objective measures, Cochrane is thriving”.

I respect David Tovey a lot but wonder about other factors that should be considered, e.g. whether Cochrane currently has the right balance between science and business, between collaboration and top-down. For example, it is good to see a new network initiated in the United States but I noticed that the existing centre “closed down in the spring of 2018, in frustration over management and other centre directors are also contemplating leaving Cochrane”.

There are also concerns about Cochrane’s conflict of interest policy, as discussed by Hilda Bastian and Lisa Bero. Cochrane is currently reviewing the conflict of interest policy.

Expulsion of Peter Gøtzsche

Disagreements surfaced at the recent colloquium in Edinburgh which saw the expulsion of Cochrane Collaboration co-founder Peter Gøtzsche and the resignation of four members of the Cochrane governing board in protest of the decision, namely Gerald Gartlehner, David Hammerstein Mintz, Jörg Meerpohl and Nancy Santesso.

Peter Gøtzsche, former cochranite because of alleged bad behaviour

All involved parties commented on the events:

There was also a fair bit of media attention, e.g.

Furthermore, quite a few comments were published, for example by:

The most comprehensive account of events, including links to all statements and numerous comments as well as details on the legal process investigating the allegations against Peter Gøtzsche can be found on http://www.deadlymedicines.dk/. Note, Peter Gøtzsche maintains this website so some (e.g. selection) bias is possible. However, according to Peter, his only defence against attacks on his scientific and personal reputation is full transparency so I consider this to be the most reliable account of events.

Peter’s website deadlymedicines.dk

In contrast, the statement by the remaining members of the governing board admittedly did not include all facts: “So, here are the facts as we are able to report them. We may be able to tell you more later, we may not. Time will tell.”

During the AGM in Edinburgh, Cochrane governing board co-chair Martin Burton described the process leading to the expulsion of Peter Gøtzsche as “independent, transparent and fair”. In the same AGM, former Cochrane steering group (now governing board) co-chair Jeremy Grimshaw called for calm and stated that “people can contest this” process.

During the meeting, I had my doubts about the process being “independent, transparent and fair” as I was not convinced by the sparse information communicated by the remaining Cochrane governing board. In an article in STAT, Cochrane governing board co-chair Marguerite Koster later confirmed that the “bad behaviour” leading to expulsion of Peter Gøtzsche was in part the use of “wrong letterheads”. This seems very odd to me. If that was the case, why not simply clarify any ambiguity? Also, if there is a conflict between Cochrane CEO Mark Wilson and Peter Gøtzsche, why not seek mediation as suggested by the UK Charity Commission?

There were serious doubts about expelling Peter Gøtzsche from Cochrane, as shown by the tight vote of the governing board (6 for, 5 against, 1 abstaining, Peter not voting). The members who resigned described the expulsion of Peter Gøtzsche as “disproportionate.

I would have liked further transparency, e.g. a clear definition of the ambiguous and unhelpful term “bad behaviour” which apparently some people took as indication for allegations of sexual harassment or similar which had to be clarified later (Spoiler: there was none). Losing one member can happen, losing four indicates more serious problems. I would have applauded the remaining members of the Cochrane governing board to step down and seeking re-confirmation, ideally explaining the reasoning behind their decision to expel Peter Gøtzsche.

Personally, I find it difficult to trust the remaining members as I value the opinion of the four members who stepped down (Gerald Gartlehner, David Hammerstein Mintz, Jörg Meerpohl and Nancy Santesso) who saw the legal counsel’s report and surely resigned for a strong reason.

“Death of a whistleblower and Cochrane’s moral collapse”

In the latest twist of this saddening story, Peter Gøtzsche has now published a book with a detailed account of events. As stated before, this should be taken with a pinch of salt. However, based on an audio recording, the book describes the decision-making behind the expulsion of Peter Gøtzsche (chapter 3) and other internal processes.

The cover of Peter’s latest book

In many respects, I don’t want these details in the public domain. However, I hope this might be the starting point of an open debate on the future direction of Cochrane: Business? Science? Business AND science? Top-down or collaboration? Both?

The book also highlights some potential issues leading to the decision to expel Peter Gøtzsche as well as potential problems regarding the centralised, top-down approach which might be good for business but bad for the collaborative spirit. Given expelling Peter Gøtzsche has wider implications (see above), it is disheartening to see Martin Burton being quoted commenting on the expulsion vote with “It’s a majority of one which I think is personally, I think is disappointing, very disappointed with that indeed … I feel unhappy in many ways”.

Assuming the audio recording is correctly reflected by the book, it also highlights some major problems with the first two of 10 Cochrane key principles, i.e. collaboration and building on enthusiasm of individuals.

Final thoughts

Overall, I am deeply saddened by these events. Based on the best available evidence, it seems like the decision to expel Peter Gøtzsche has been at the very least “disproportionate” and it might have been possible to avoid this as well as the subsequent tensions.

I agree that Peter has a unique style and is quite outspoken. However, I strongly believe that having dissenting voices raising important issues can only benefit a scientific organisation and hence the output of Cochrane.

Furthermore, I am concerned about the direction Cochrane is taking. Obviously, strengthening the business-side is important. However, I feel that it is crucial to balance this with the spirit which the Cochrane Collaboration (!) was founded on. Current tensions show the bad sides of centralism which is very worrying, e.g. the Cochrane central executive team holding back emails, as indicated by two Cochrane centres (Austria and Sweden).

I would urge everyone with an interest in Cochrane to read up on these aspects and the events surrounding the dismissal of Peter Gøtzsche. A lot (!) of material can be found on his website http://www.deadlymedicines.dk/.

Other people discussed these and other aspects related to current state of affairs of Cochrane as well, including

I wrote this text as everybody should have access to the available information so that they can meaningfully contribute to the discussion of the future direction of Cochrane.

Robert Wolff

This text was based on a series of 46 tweets on https://twitter.com/wolffrf. It has been lightly edited for this blog.



Four major policy issues

Most Cochrane members will have felt at best uneasy about the recent events in Cochrane. We also felt uncomfortable and wondered what caused our feelings. The biggest problem we felt was the lack of transparency. Who had been doing what? Who makes the decisions in Cochrane? What can I do as a member of a Cochrane Centre or a Review Group? Many questions were also triggered by the statement of the Governing Board that there will be ‘zero tolerance for bad behaviour’. We fear this will easily lead to ‘zero tolerance for different opinions’.

We are all members of Cochrane because we believe that evidence-based health care is an important asset for public health. Evidence that can be trusted is what we all want to create. We believe that the lack of transparency and trust which we experience at the moment is detrimental to Cochrane and we have to do something about it.

There are four major policy issues that we would like to improve.

  1. Create a culture of open discussion

First of all, we need a culture of open discussion. At the moment, many people don’t dare to speak out because they fear that it will be used against them. We ask the Governing Board and the Cochrane Central Executive Team (CET) to create a forum where interactive discussions can be held. Modern organizations commonly have a freely accessible online discussion platform that can be used by all members.

  1. Refocus on the heart of Cochrane

Authors, members of methods and review groups put enormous effort in producing Cochrane Reviews without payment. The revenues of these reviews finance a large Cochrane CET. In the past, many good investments have been made such as the Game Changer project. We would like to see an increase in support to the heart of Cochrane by for example financially supporting review production, methods development and stakeholder interaction worldwide. Cochrane CET and Cochrane members in various groups around the world should work together to achieve this.

Senior Lady Giving an idea to her Colleagues

  1. Increase the involvement of Cochrane members

We as Cochrane members are very motivated to do our reviews, train our authors, develop methods, assist with difficult analyses or other important tasks for getting our high-quality reviews published. However, our involvement in the governance of the organization is minimal. At the Annual General Meeting only 240 members voted out of a potential 11 250 voters (2%). To support the democratic process in general and to increase ownership in the implementation of new policies in Cochrane, voter participation rates should be increased. The members are the basis of the organization according to the constitution of Cochrane. A good start would be to have a lively debate among members during the coming Governing Board elections and to organize general meetings more effectively.

  1. Find a better business model for Cochrane

Cochrane reviews are sold to subscribers. The European Union has decided that science funded by public money should be published Open Access, which means that it is freely available for the end-user. Cochrane does not have such an Open Access model. Even though content is freely available to a large part of the world, important new reviews are behind a paywall. Also, content that is older than 12 months is free but this is largely not up-to-date. This is different from our promise to be the most reliable, up-to-date and accessible source of evidence. We conclude that in the long run the current business model is untenable and we must find better solutions.

Support

Signed by the following Cochrane Members:

  1. Jos Verbeek, Coordinating Editor, Cochrane Work
  2. Gerd Antes, former Director, Cochrane Germany
  3. Matteo Bruschettini, Director, Cochrane Sweden
  4. Jani Ruotsalainen, Managing Editor, Cochrane Work
  5. Chris Del Mar, Coordinating Editor, Cochrane Acute Respiratory Infections
  6. Mark Jones, Centre for Research in Evidence-Based Practice (CREBP), Bond University, Australia
  7. Caroline Struthers, UK EQUATOR Centre, University of Oxford
  8. Lotty Hooft, Director, Cochrane Netherlands
  9. Tianjing Li, Co-ordinating Editor, (US Satellite) Eyes and Vision
  10. Gerald Gartlehner, Director, Cochrane Austria
  11. Nicole Skoetz, Senior Editor, Cochrane Cancer
  12. Barbara Nußbaumer-Streit, Associate Director, Cochrane Austria
  13. Nancy Santesso, Deputy Director, Cochrane Canada
  14. Philipp Dahm, Coordinating Editor, Cochrane Urology
  15. Malgorzata Bala, Director, Cochrane Poland
  16. Joerg Meerpohl, Director, Cochrane Germany

We asked people to sign this petition if they wanted to join us. By the start of voting, when we closed the petition, 620 people had signed it. We think this is great and totally new in Cochrane. Thank you to everyone who signed!

 

“All effective treatment must be free” — Archibald Cochrane