TDI Expert Meeting 25-26 September 2006

TDI Expert Meeting 25-26 September 2006

Data quality and data coverage within the Treatment Demand Indicator

Drug use among very young people in treatment for drug use

______

6thAnnual Expert Meeting on TDI

Final minutes

25-26 September 2006

EMCDDA - Lisbon

Tables of contents

Summary

PART I

State of Progress of the Treatment Demand Indicator

1.Welcome and introduction by the EMCDDA Director (W. Götz)

2. Overview of the meeting and state of progress of the indicator(L. Montanari - EMCDDA)

3. Data quality on 2004 TDI tables (S. Sleiman - EMCDDA)

PART II

Methodological Issues

1. Data coverage assessment (E. Iversen – Norway; R.Holmberg – Sweden)...

2. Outcome following treatment demand (M.Donmall – United Kingdom; T. Koos – Hungary)

3. Improvement of treatment demand data quality (J. Long – Ireland; X. Poos – EMCDDA)

PART III

Data analysis

4. Drug use among very young people: a German experience (J. M. Fegert - Germany)

5. Treatment demand and drug related deaths (P. Schifano - Italy)

6. European research on co-morbidity: the ISADORA project (K. B. Moeller- Denmark; T. Greacen - France)

PART IV

Updates on current TDI projects

7. Gender analysis (E. Maffli - Switzerland)

8. Prevalence data (T. Ouwehand – The Netherlands)

9. Specific methodological issues

10. Countries information system overview (K. Christaki - EMCDDA)

11. The “Fonte” project: web applications and central depository (U. Solberg – EMCDDA)

12. The UNODC project on ASI. Relation with the TDI (V. Raes – Belgium)

General conclusions

Summary

The 6th TDI expert meeting 2006 took place at the EMCDDA in Lisbon on the 25-26 September 2006.

It was a fruitful meeting and many issues were discussed from methodology to analysis to problematic areas and ideas for future developments.

The meeting was organised in four sessions:

  • Introduction on the state of art of the indicator and the quality of the collected data
  • Work-shops:three workshops were organised on data quality, outcome following treatment demand, data coverage
  • Data analysis;concerningthree topics: very young people in treatment, treatment evaluation comparing treatment data and drug related deaths, co-morbidity
  • Up-date on current and on-going projects (gender analysis, prevalence project, FONTE system, UNODC project on treatment).

In the first session the meeting opened with an overview of the state of progress of the indicator, showing improvement in data quality and coverage. The data quality working group underlined the importance of feedback between EMCDDA, countries and national data providers. Effective experiences should be shared among European experts (e.g. guidelines for data collection, procedures for data quality control)

In the second session three parallel work-shops were organised.

In the work-shop on data coverageparticipants agreed that coverage is an important issue that can affect data analysis and interpretation. Assessing coverage of treatment centres and clients is therefore necessary in order to have a clear picture of treatment system and clients characteristics.

The second work-shop concerned the evaluation of treatment outcome, which is a topic of growing importance in the drug treatment field. The experts agreed on the importance of monitoringtreatment outcome and the possibility of its inclusion within the TDI project, taking into account the methodological limitations and the political implications. Data on client situation and treatment process were also agreed to by the participants as common basic information to be collected for the evaluation of treatment outcome.

The third sessionof the meeting focused on analysis of treatment data on three topics:

  • Very young people using drugs
  • Relation between type of treatment and drug deaths
  • Co-morbidity

In the first presentation,the focus was on very young people using drugs, their specific problems and the risk factors for drug use. The importance of effective referral to treatment of young people was also underlined.

From the follow up study on treatment exits using treatment data and drug related deaths data, the relevance of having good data on treatment and of combining different information sources was shown. Results of the study concluded that having no treatment or short treatment might have negative effects for the clients outcome.

Experts from ISADORA project – a European project on co-morbidity – presented the first project results. Data collected in 7 European cities on drug users with mental health problems and psychiatric patients with drug use showed that co-morbidity is an important issue for social and health services, which often are not dealing appropriately on that issue. Often services are set around the needs of the organisation instead of the needs of the patients. Results research should be disseminated and further analysis on the topic is needed. The EMCDDA will include co-morbidity as a subject of the next year´s work programme.

The fourth part of the meeting continued with the up-date on the on-going projects and a discussion on specific methodological issues.

The working group on gender has done a first analysis of data, showing interesting results. The work will be concluded in 2007 with the publication of a scientific article on gender analysis of treatment data.

The prevalence project started in 2006 with a working group that produced guidelines for a first pilot data collection. Data have been collected in September 2006 from 5 countries and the first results were analysed. 5 more countries volunteered to participate but did not send data. Results showed interest in continuing the work as a pilot project and presenting them to the National Focal Points meeting.

In the last part of this meeting session specific methodological issues were discussed, with respect to categorization of drugs by drug groups and classification of clients by treatment setting. There is still a need for further discussion of classification of drug types and client by treatment setting. The two issues will be discussed in-depth in the next expert meeting in September 2007. One specific point concerned the data collection on secondary drugs, which currently seems to record only the frequency each substance has been reported and not the number of clients, as additionally proposed by the Greek experts. The proposal will be sent to the countries for in the 2007 Guidelines.

The last part of the meeting concerned the presentation of the new FONTE system for data collection and the UNODC project on treatment and revision of the ASI –Addiction Severity Index.

From 2007 a new system for data collection will be introduced at the EMCDDA and an on-line tool will be given to countries. The system will allow an on-line data entry, which will make the data provision easier and more user friendly for the countries. It will also allow a more precise control on data quality.

Finally the UNODC project on treatment Treatnet was presented. It involves several regions of the world with the aim to improve treatment capacity, accessibility and effectiveness. One of the module of the project concerns the revision of the instrument to evaluate the client situation during and at the end of treatment. ASI has information which is also included in the TDI and a high compatibility with the TDI is recommended in order to allow future comparisons.

PART I

State of Progress of the Treatment Demand Indicator

1.Welcome and introduction by the EMCDDA Director(W. Götz)

The meeting was introduced by the director of the EMCDDA Wolfgang Götz, who welcomed the participants and stressed the importance of the TDI indicator as it is the oldest and one of the most advanced centre’s indicators. The role of the EMCDDA is to co-ordinate Member States activities related to TDI,to collect, analyse and disseminate treatment demand data. Quality and coverage of the indicator have improved overtime as well as the level of analysis. However there is a need for more targeted information and close collaboration in terms of implementation of the EU action plan which is a responsibility of both the MS and the centre.

The directorexpressed his appreciation for the large participation at the meeting and particularly welcomed the external hosts: the Russian delegation who participated for the first time, the experts from UNODC, CICAD and the other external technical experts. In the context of TDI,particular attention has always been given to international relations.The joint EMCDDA-UNODC toolkit for collection and analysis of treatment demand data and indicators, which will be publishedsoon, is an example of a good concrete collaboration between EMCDDA and external partners.The EMCDDA supports this initiative and hopes for more future collaborations with international organisations and other regions on specific topics.

2. Overview of the meeting and state of progress of the indicator(L. Montanari - EMCDDA)

(see presentation: Annex 3 - State of Progress of TDI 2006.ppt)

A general picture on the state of progress of the indicator in 25 countries was presentedincluding a comparison between 2004, 2005 and 2006 treatment demand data. The situation of treatment demand data in 2006 has improved; in particular more staff are working on the TDI at national level and the data coveragehas extended compared to the previous two years. However,by providing more consolidated data the potential of analysis could be improved even further.There are still problematic areas of the indicator, which include:

  • the coverage: variable in some countries and often limited, especially when the system in based on voluntary participation;
  • the lack of financial funds;
  • the absence of control on double counting;
  • the lack of motivation of professionals in collecting data;
  • the resistance to adapt specific registers to national and European standards.

An overview of the meeting was presented; the meeting was organised in 4 sessionsas follow:

  1. State of progress of the treatment demand indicator and data quality issues
  2. Methodological issues (work-shops): data coverage, outcome following treatment demand, the improvement of treatment demand data quality
  3. Data analysis, with a focus on drug use amongst very young people;combined analysis of drug related deaths and treatment demand data for an evaluation study; co-morbidity among drug patients (presentation of the main results of the ISADORA project, a European research project on co-morbidity)
  4. Follow up of projects under implementation within the TDI project: gender analysis, TDI prevalence, new drugs and classification in the TDI; tables on secondary drugs; other issues
  5. Future developments

The TDI annual expert meeting was followed by a meeting dedicated to the issue of drug use with an international perspective amongst very young people.

(In annex 1 and 2 of the minutes the agenda and the list of participants are available)

3. Data quality on 2004 TDI tables (S. Sleiman - EMCDDA)

(See presentation: Annex 4 - Data Quality 2004 Tables.ppt)

Quality assurance in data reporting is a tool for all partners involved (EMCDDA, NFPs and their partners). Quality control is defined as a set of procedures to assess whether data meets quality standards, including in-depth evaluation of national reports and tables, quality audit not done by the EMCDDA and quality enhancement implemented through ReitoxAcademies and expert meetings.

The objectives of quality assurance are:

•To improve quality of data collection instruments

•To improve quality of data sets

•To reduce workload in validation phase

•To improve credibility of EMCDDA and NFPs

In practice,controlling the quality of the standard tablesmeans checking whether theycomply with the guidelines set by the EMCDDA (deadline, methodology instructions and references) and whether the data included are reliable (calculation, consistency with previous data).

EMCDDA deals with a vast amount of information. In 2005, 759 data tables and 3111 pages of National Reports were received, an ever-growing number considering that in 2002 there were 493 data tables and in 2004, 723 data tables.

Regarding TDI tables in 2006, the great majority of countries respected the deadlines (more than for other indicators); only 2countries uploaded the tables with delay. The level of implementation of the TDI indicator was good for two thirds of the countries. Clarity of TDI information in National Reports wasalso good.However,7 countries do not implement the indicator at all andapproximately 1/3 of them,, do not have a mechanism for control of double-counting and the rest have a control only at centre level. Of all queries sent by the EMCDDAduring the validation period in 2006, 46% were related to missing values, errors of calculation, not understandable data, wrong data reported and/or reported in the wrong place.37% of all queries were related to clarifications of provided data, unclear abbreviations anddefinitions, missing references and methodological information. Other problems encountered include missing values, inconsistencies and problems with the interface. Most of these errors could be avoidedby cross-checking the figures included in the tables before sending them, as well as by checking the internal consistency of the data (TDI/ST03/ST04).It would also be useful to send comments and bibliographic references when available. Communication between the national providers and the EMCDDA is essential during all parts of the process. The EMCDDA encourages and fosters the development of quality control mechanisms in the NFPs. In the near future FONTE, the new information system currently developed at the EMCDDA, will allow for a more systematised quality control.

Discussion

NO, NL: commented on the amount of TDI personnel stated in the progress reports. The numbers given do not correspond to personnel working full time on the indicator but only partly, apartfromthe fact that most of thesepersonnel areworking on the national system. In terms of personnel the resources are limited and this could be one of the main problems.Budgetary issues are also a problematic issue in some countries.

DE: also added that it is often difficult to draw the line between national systems and TDI in the NFPs.

PG: focus should be given to the increasing coverage of the indicator and in finding ways to include GPs in the system.IE is the only country with a big GP coverage (72%) while in the UK there exists an arrangement of shared treatment care between GPs and outpatient treatment centres, so in a sense GPs are included in the system.

LM: With regards to data quality stressed that it would be interesting to know the reason of repeated problems like missing information and inconsistency between tables (ST3 and ST4 and TDI) and whether TDI experts participate in the production of the national report.

NL: noted that the quality report doesnot reach the experts when they are placed outside the NFP, which is the case of the NL, nor do the expertssee the feedback sent to the EMCDDA. There seems to be a communication problem between the experts and the NFPs.

BE: agreed by stating that data collection is happening in the field and then the NFP is gathering the information,therefore it is not simple to find out what is happening in the data collection phase. Better communication should be established between different levels, apart from implementing the protocol.

DE: raised the issue of difference in deadlines between indicators; when writing the national report for instance, TDI data are not all ready and what is written in the report is not reflecting the latest situation; it is not always realistic to have all the information from different parties (ministries, field,etc) at the requested time.

NL: agreed and added that even though they are using a sophisticated system, in some cases they have to leave data out, like in a case where one organisation has trouble delivering data on time, so they start weighing down the quality of data.

IE: suggested to introduce some space in the table dedicated to comments, so that the person who fills in the table would be able to comment on the table itself.

PG: stressed the need to understand the contextual information like number of units reporting. He proposed to have a workshop in the future on aspects of the reportingas there is an increasing need to develop a quality report.

Conclusions

Data quality of TDI has improved over time: comparing 2004 and 2006 more countries report TDI data and are in line with the TDI Protocol. Respect of deadlines and completeness of the data have also improved. The organisation for data collection has also a better structure, with more staff and resources dedicated to the TDI work in several countries.

Still, problems remain, related on one hand to methodological aspects - limitations in data coverage, lack of internal consistency of the data, missing information, calculation errors, lack of contextual information - and on the other hand to data collection organisation -lack of communication between experts and NFP, lack of resources.Solutions for improvement:

  • the experts will pay more attention to the internal consistency and try to avoid calculation errors.
  • a space for comments will be added to the tables in order to add contextual information which can explain the specific distribution and coverage issues
  • a working group on coverage will be organised in order to better understand the factors to consider for a correct data analysis (see also work-shop on coverage)

PART II

Methodological Issues

Three parallel work-shops were organised in order to discuss in small groups specific subjects and find operational solutions. The chosen topics were in line with the TDI work plan for 2006 and the conclusions of the 2005 expert meeting.Each work-shop was introduced with a presentation; a country expert chaired the discussion and reported back to the whole group.

The work-shops focused on the following subjects:

  • Data coverage assessment (chairman,Erik Iversen)
  • Outcome following treatment demand (chairman,Michael Donmall)
  • Improvement of treatment demand data quality (chairman,Jean Long)

1. Data coverage assessment (E. Iversen – Norway; R.Holmberg – Sweden)

(see presentation: Annex 5 - Data coverage.PPT)