Doug Altman (Chair), Gerd Antes, Kaveh Barjesteh, Luc Bijnens, Michael Bracken, Gina Bravo

Minutes of the Meeting of the Statistical Methods Group
3rd Cochrane Colloquium, Oslo, 6th October 1995
Draft minutes prepared by Jon Deeks and Doug Altman
Present

Doug Altman (chair), Gerd Antes, Kaveh Barjesteh, Luc Bijnens, Michael Bracken, Gina Bravo, Jon Deeks, Stephen Evans, Paul Glasziou, Steve Goodman, Joseph Lau, Alain Li Wan Po, Dianne O'Connell, Arne Ohlsson, Ken Schulz, Richard Sylvester, George Wells, Fred Wolf.
Meeting Summary

This meeting addressed several major issues and attempted to identify and define some of the roles of the statistical methods working group (SMWG) within the work of the Cochrane Collaboration:

- Communication within SMWG will primarily be through the e-mail list, CCSTAT. SMWG will develop a close link with the software group.

- SMWG will ensure that Update Software receives the appropriate statistical input to implement the meta-analytical methods agreed at the 1st SMWG meeting.

- All statistical methods implemented in future versions of the software should be published, documented and approved by the SMWG. The Steering Group will be asked to ratify this point. Statistical queries which arise whilst programming the methods should be debated within SMWG.

- Members of SMWG are willing to produce help files for the software and the Cochrane Handbook.

- Cochrane Review Groups should attempt to recruit a statistician to provide technical advice. Groups without statisticians should contact the appropriate Cochrane Centre for advice.

- Both Cochrane centres and statisticians within review groups should use the CCSTAT list to post problematic statistical queries.

- The SMWG is aware of a research agenda of statistical issues. Members who are interested in working on these issues should form informal working groups within the SMWG, and ensure that all members of the SMWG are aware of their work and findings by communicating on CCSTAT.

- Members of the SMWG will continue to run statistical sessions at future colloquiums and will send information concerning suitable training events to Cochrane Centres.

- The SMWG was concerned about future fragmentation of statistical methods into other methods working groups. This concern will be expressed to the steering group.

Detailed Minutes

(0) History of the Statistical Methods Working Group

The group was created retrospectively following a meeting of 12 statisticians and interested individuals at the UK Cochrane Centre in Oxford in the summer of 1993. Other statisticians involved in the work of the collaboration have been added since then. The Oslo meeting was effectively
the second meeting of the SMWG.

(1) Communication within the Statistical Methods Working Group

An electronic discussion list (CCSTAT) which has been recently implemented by Doug Altman was welcomed by the group. All people attending the meeting who were not on this electronic discussion list will be added. The list should be used for discussing all issues of importance for the group,
whether software, training, research or administration. Standard list etiquette will be followed and all members were encouraged to use informative titles in their messages (see further details appended to these minutes).

(2) Input of the statistical methods group into the development of Revman and CDSR.
Revman is the software that is used by reviewers within the Cochrane Collaboration for the management of references and statistical analysis. The Cochrane Database of Systematic Reviews (CDSR) is the end product of the collaboration. It contains the text, data, tables and graphical displays of the systematic reviews. Revman is in fact a component of CDSR, and is used to analyse and present the results of each review. This is important, as extensions to the options for analysis available to the reviewers in Revman will automatically become available to the user of CDSR.
It was noted with some concern that the current version of Revman only implemented the Peto method of combining log odds ratios and the weighted mean difference for continuous data. The meeting supported the proposal that the methods for meta-analysis agreed at the first meeting of the SMWG in Oxford in 1993 should be implemented as quickly as possible. This covered implementing fixed effect and random effect models for odds ratios, relative risk and absolute risk reductions. [At an advanced statistics workshop later at the colloquium it was decided that the standardised mean
difference method should be implemented for the analysis of continuous data]. It was agreed that the relevant sections of the minutes of the 1993 Oxford meeting will be circulated on the email list. It was felt essential that the SMWG approved of all methods implemented in the software, and wherever possible the formulae should be those which are widely accepted and published. It was requested that the Steering Group was asked to ratify this decision. All the formulae implemented should be documented in the handbook.
The delay in implementation related to unclear responsibilities and a lack of communication between the software development group and the SMWG. It was noted with some concern that neither the software nor Revman development or advisory groups included a member of the SMWG. Jon Deeks volunteered to work with Update Software in implementing the standard methods of
meta-analysis agreed at the Oxford meeting, and to keep the methods group informed of progress through the electronic discussion list. It was agreed that any queries that arose in the implementation of the Oxford proposal would be debated within the group.
Many members of the group volunteered to test the prototype software when it was available. They identified that the software written by Joseph Lau and the algorithms produced by Simon Thompson and Rebecca Hardy should be used for comparison.
The group also recognised that it is important to provide on-line help facilities for those using Revman and CDSR. Many of the SMWG volunteered to write and review sections of the help files. In addition relevant sections of the handbook may need to be updated to fall in line with the new software developments. The editors of Section 6 of the Handbook will be made aware of the willingness of members of the SMWG in assisting with this.

There was some discussion of the need for developments in the software necessary to analyse and display the results of individual patient data analyses. This particular topic was being tackled by the individual patient data methods working group.
It was noted that the Revman software will remain somewhat restrictive in the perceivable future.

(3) Statistical Support for Cochrane Review Groups
It was noted that the Cochrane Collaboration's Annual Report (circulated at the colloquium) stated that the methods working groups should be "providing methodological support and advice for the Cochrane Review Groups". During the colloquium many members of the SMWG had been approached by reviewers with specific problems, which was partially fulfilling this role. However,
it was thought that a better working arrangement would be to suggest that each CRG recruited a statistician to its editorial group who would advise on queries. Some members of the statistical methods group were already working in this manner. Groups which failed to recruit a statistician should contact the appropriate Cochrane Centre, who if necessary would pass the query to the SMWG. All statisticians in CRGs should be on the SMWGs e-mail list.
There was concern over the amount of work that this would be likely to generate. It was thought that it should not be expected that a statistician should review all review protocols.
There was some consideration of the type of problems that are likely to occur. It was expected that many problems would be quite straightforward. However some problems may require help from experts within the SMWG, and it was thought that the e-mail list should provide a suitable method of
communicating and discussing such queries. Such questions may not have known solutions and may be added to the research agenda of the SMWG, or possibly other groups.

(4) Developing a research agenda
Doug Altman presented a list of areas where further methodological and empirical research is needed. It included:
Cross-over trials
Continuous data
Survival data
Heterogeneity
Mixed endpoints
Mixed designs
Graphical presentation
Numerical presentation
Statistical communication
Analysis of non-RCT data
Trial quality
Individual vs summary analyses
Group (cluster) randomised trials
Some of these topics are also the remit of other methods working groups.
Many individuals at the meeting indicated that they had an interest in developing one or more of these areas. Rather than forming new subgroups for each of these topics in was hoped that the organised electronic discussion list should allow interested individuals to collaborate in informal 'focus' groups within the SMWG. If these discussions are kept open to the whole group in would enable many interested people to listen to the discussions which would be of benefit.

(5) Providing training in statistical methods
It was noted that several members of the group were involved in running statistical workshops at the colloquium. It was thought that it would be useful to publicise other suitable courses, possibly through the Cochrane centres. In the first instance information about training events should be
posted on the electronic discussion list. However it was not thought reasonable that the group should be required to provide training to members of CRGs.

(6) Future of the SMWG and its place in the Cochrane Collaboration
The group so far had taken a relatively low-key role in the collaboration as it had been unclear what its remit and responsibilities were. It was felt that in the future the SMWG should take a more proactive role and ensure that it becomes involved where it felt that statistical input would be of
benefit.

The SMWG felt that it had a very wide but unclear remit, and that many of the other methods working groups were tackling problems of a statistical nature. Indeed many members of the group were also participants of other groups. There was particular concern that additional groups may be
registered to consider problems of a statistical nature, such as graphical displays. It was considered unclear to what the role of the statistical methods group would become if all future interesting and clear problems were taken up by other groups. One suggestion was to rename the group 'Statistical Methods and Graphics Working Group', to clarify the group's view that graphical display should not be considered in isolation. It was agreed that the this concern over excessive fragmentation should be expressed to the steering group, and that members of the statistical methods group should be attending other methods working group meetings to promote communication and efficiency where it seemed appropriate.