COMARE Statement - Further Statement on the Incidence of Childhood Cancer in Wales

Further Statement on the Incidence of Childhood Cancer in Wales

Background

In March 1999 the Welsh Office asked COMARE to examine two unpublished studies. The first by Busby et al¹ of "Green Audit" concluded that there was a significant excess of childhood leukaemia in North Wales associated with residential proximity to the coast. The second study, carried out by Steward et al² of the Welsh Cancer Intelligence and Surveillance Unit (WCISU), did not support this conclusion. We were asked to advise as to whether we considered that there was a real raised incidence of childhood leukaemia near the coast of North Wales and whether further study was required.

To do this we initially organised a comparison of the figures quoted by both the Green Audit and WCISU with the database held by the Childhood Cancer Research Group (CCRG) in Oxford. This group maintains the National Registry of Childhood Tumours (NRCT), data for which are supplied from a variety of sources including cancer registries but also directly from medical cancer specialists as well as from death certificates. As a consequence this provides an independent check on much of the data on childhood cancer held by cancer registries in Great Britain and is clinically validated. After carrying out the independent check on the number of cases of childhood leukaemia in these Welsh counties it was immediately apparent that the data held by Green Audit, on which the analysis by Busby et al was based, were incorrect. These data were received from the Welsh Cancer Registry (WCR) in 1995. A further data set was received from WCR in 1996 but was not used in the analysis by Busby et al.

In June 1999 we issued a statement to the Welsh Office. In that statement we noted that Dr Busby and his colleagues appeared to have used erroneous data in their study. On the basis of the Steward et al data, COMARE also stated that we found no evidence to support the contention that there is an increased incidence of childhood leukaemia or other childhood cancers amongst the Welsh population living close to the Irish Sea.

Further investigations

At that time we were unable to resolve the confusion in the data used by Green Audit. We were unable to establish that WCR had kept a copy, either on computer or on paper, of exactly what was released to Green Audit. Following the general re-organisation of cancer registries in England and Wales, the responsibility for maintaining such databases, in Wales, had been passed to WCISU.

COMARE said it would like to ascertain exactly how the erroneous data presented by Green Audit arose. To this end the COMARE Chairman wrote to Dr Busby to ask him to release to this Committee copies of the electronic files which he received from the Welsh Cancer Registry, so that an attempt could be made to establish the nature and cause of the error. Following a considerable correspondence we have received only a limited amount of data from Dr Busby. Though far from satisfactory, these limited data have allowed us to make further data comparisons.

The data sent to us by Dr Busby were printouts of WCR data for Gwynedd for 1984 and 1988. We focussed, therefore, on data for Gwynedd for the period 1984 to 1988, for which there were summarised WCR data published in 1994. We compared these data with those held by WCISU, the Office for National Statistics (ONS) formerly the Office of Population Censuses and Surveys (OPCS), and NRCT. The ONS database is entirely dependent upon input from cancer registries. The ONS informs cancer registries of death certificates that have any mention of cancer and it relies on the registries using this information in combination with other sources of data to create a set of checked records, which are then sent to the ONS.

The NRCT was found to have the most complete database. It receives information from a variety of sources in addition to the cancer registries. It receives totally independent data directly from the UK Children's Cancer Study Group, which started in 1977 and has evolved so that its members now see around 85% of all cases of childhood cancer in the UK. The NRCT also receives data on all cases enrolled into clinical trials. Furthermore, the NRCT receives details of any death certificate recording childhood cancer. Because of these independent checks this database is considered to be as complete as is practically possible.

A comparison of these databases revealed a number of discrepancies, which are discussed in detail in the accompanying technical annex. The only major discrepancy between the various data-sets held by WCR, WCISU, ONS and NRCT arose through some adult cases being classified as children on the WCR, WCISU and ONS data files but this has little effect on the leukaemia data. The data on childhood leukaemia currently held by WCISU are similar to those held by the NRCT and there are only relatively minor discrepancies with data (a) published by WCR, (b) from WCR tapes "frozen" in 1994 and 1996, or (c) sent to Green Audit in 1996. As detailed in the annex, however, the 1995 WCR data analysed by Green Audit contain an erroneous category of 'All leukaemias'. This group, which was presumably intended to give a total count of leukaemias in each area, does not in fact agree with the data on the same file for the individual leukaemia sub-types and is not found in any other data-set. If this group is excluded from the 1995 WCR data analysed by Green Audit, the total for childhood leukaemias is essentially in agreement with those in the other databases. In summary, it is clear that the inclusion of the erroneous `All leukaemias' group has led to the incorrect analysis of Busby et al.

The situation regarding other childhood malignancies is more complex and is also explained in the technical annex.

The Welsh Cancer Registry

Our enquiries have also allowed us to determine, to some extent, the workings of the Welsh Cancer Registry before its replacement by WCISU in 1997. The picture has emerged of an under-resourced group trying to build a satisfactory registration database on a shared mainframe computer. Cross checking of data appeared to take a very long time and amendments might have gone unrecorded in appropriate cross-referenced material. We have been informed that OPCS became aware in 1987 of considerable inconsistencies in some of the data it had obtained from WCR. Attempting to cross check these data with WCR proved extremely difficult. Further discrepancies were found to have occurred during the 1980s. We understand that in the early 1990s, the existence of unacceptable resource pressures on the Registry and consequent low morale were recognised. A new operating system was introduced in the mid 1990s but this resulted in further delays in entering information because of unavoidable testing and software checking. It is possible to speculate that unchecked tabulations could have been sent to Green Audit in 1995 in view of the history of under-resourcing and consequent problems in the operation of the Registry. These problems have now been all been resolved with the formation of WCISU.

Improvements to cancer registration generally

Even at the time of publication of our Third Report in 1989, we were aware that false conclusions could be drawn from the use of the cancer registration data then available. In that report we recorded the fact that OPCS researchers and our own enquiries had highlighted a number of problems with the National Cancer Registration Scheme. In recommendation 3 of our Third report we said that urgent consideration should be given to the validity of cancer registration data, which was at that time of variable quality. In addition we recommended specific improvements relating to registration data in England and Wales to ensure that the database was complete and accurate. Our current investigations have also revealed that despite the considerable improvements that have taken place in the last ten years many cancer registries have been unable to carry out the annual 2% data audit required in their contracts with the funding authorities.

Conclusions

In examining possible errors in the data analysed by Busby et al we have attempted to distinguish between the errors of cancer registration generally, some problems known to have occurred at WCR and the very specific tabulation error that exists in the data file on which the analyses by Busby et al are based. That data file contains about twice the number of cases of leukaemia as those recorded by WCR and WCISU for the relevant period and because these are concentrated in certain geographical areas the report by Busby et al includes an even greater excess of cases. Neither the general problems of cancer registry data nor the specific problems experienced by WCR can account for the excess reported by Busby et al. We note that Busby et al chose not to use the dataset sent to them by WCR in 1996 which appears to be free of the erroneous childhood leukaemia entries.

In summary, there are recognised errors in cancer registration arising from failure to ascertain cases or remove duplicates, incorrect diagnoses and incorrect location of cases. However, none of these errors can explain the findings of Busby et al. At the WCR, during the relevant period, a large number of adults were wrongly classified as children because of errors in birth dates. However, because leukaemia accounts for about 3% of adult cancer compared with about 33% of childhood cancer this error had only a small effect on the total number of childhood leukaemia cases registered. Furthermore, the various other childhood leukaemia datasets described in this statement agree reasonably well, while that used in the Busby et al analyses is totally different. Hence we conclude that the data used by Busby et al are incorrect.

Therefore, we reiterate our original conclusion that we have found no evidence to support the contention that there was an increased incidence of childhood leukaemia or other childhood cancers close to the North Wales coast. This conclusion is supported by a more complete analysis by Steward et al³.

We hope that Dr Busby will now make all the data he received from WCR available to us for further checking and that he will withdraw the conclusions of his earlier analyses, which have given rise to considerable public concern in Wales.

We have previously expressed concerns about Cancer Registry practices generally and we are still concerned that, despite our previous recommendations, some cancer registries may still lack sufficient resources to carry out their duties efficiently. It would have been of benefit to us if the information concerning registry resources in Wales had been made available to us at an earlier date.

Recommendations

Although we recognise the considerable improvements that have taken place in the working of the cancer registration system in England and Wales in the last ten years, we are concerned that some individual registries may still be under-resourced. We are also concerned that data compiled under conditions less favourable than today may contain errors that could lead to incorrect conclusions being drawn from studies employing those data. The committee has encountered considerable difficulty in establishing the cause of the discrepancy between data held by WCISU and Green Audit. We have already noted Dr Busby's reluctance to release his data to us but a further cause of this difficulty was the lack of an adequate record (whether computerised or hard copy) by WCR of the data tables sent to Dr Busby or of their specification.

We recommend that cancer registries should include as part of each individual case record the date of its creation and most recent update. They should also keep a record of cases included in the registry and subsequently deleted, giving dates and reasons for each deletion. Registries should make and keep a "frozen copy" of their complete data file at regular intervals of 3 or 6 months and keep, either on paper or in electronic form, the specification and output for any listing or tabulation of the data sent to outside users.

The use of cancer registry data for clinical, epidemiological and administrative purposes is likely to increase. Reports of investigations into possible environmental hazards are often based on these data and these may be expected to come under detailed scrutiny. Cancer registries must, therefore, keep a record of any data released to outside organisations. For these and other purposes it is essential that cancer registry data should be of high quality. The Committee also recommends that adequate resources should be made available to the cancer registration system in order to ensure that the registries can provide such data. Furthermore, where specialist registries exist (eg, for childhood cancer), data from the national registration system should be checked against those held by such registries before analyses are released, or data provided for analysis by others. We also recommend that steps be taken to ensure that registries are able to fulfil their contractual obligations with regard to annual data audits.

References

1. Busby C, Kocjian B, Mannion E and Scott Cato M (1998). Proximity to the Irish Sea and Leukaemia incidence at ages 0-4 in Wales from 1974-1989. Green Audit Occasional Papers 98/4; August 1998.

2. Steward J A, Adams Jones D, Beer H and John G (1999). Results of a preliminary study to test the Irish Sea proximity hypothesis of Busby et al. Welsh Cancer Intelligence and Surveillance Unit, Cardiff. March 1999.

3. Steward J A, John G (2001). An ecological investigation of the incidence of cancer in Welsh children for the period 1985-1994 in relation to residence near the coastline. J R Statist Soc A (2001) 164, Part 1, 29-43.

ANNEX

Comparison of childhood cancer data held by the Welsh Cancer Intelligence and Surveillance Unit (WCISU) (previously the Welsh Cancer Registry (WCR)), the Office for National Statistics (ONS) and the National Registry of Childhood Tumours (NRCT) held by the Childhood Cancer Research Group (CCRG).

1 Introduction

There are differences between the datasets held by WCISU, ONS, and NRCT. The dataset analysed by Busby et al differs from all of these. In this Annex we summarise the extent of, and reasons for, these discrepancies and, in particular, give the evidence for the belief that the data analysed by Busby et al contained a major error. We have attempted to determine whether the known levels and sources of error in cancer registry data can explain this error or whether there is some entirely different explanation. For this purpose we have examined published tables, unpublished data from ONS, WCISU and NRCT and tables provided by Dr Busby based on the data he received from WCR in 1995. For WCR, WCISU, ONS and NRCT data it has been possible to make detailed comparisons of the different datasets, subject only to constraints imposed by the confidentiality of the data. Unfortunately Dr Busby has not made his complete dataset available to us. The data provided by WCR to Dr Busby consisted of a detailed tabulation of all cases on their file in 1995, and again in 1996, covering the period 1974-89 and subdivided by year of registration, 5-year age-group, sex, diagnostic classification, and small area. Dr Busby has provided two tabulations based on the 1995 version of these data.

2 The source of error in the study by Busby et al

Even with the limited data sent to us by Dr Busby the source of his error is clear: the tables contain six categories relating to leukaemia - the standard ICD codes, 204, 205, 206, 207, 208 for leukaemia and a category labelled 'All leukaemias'. It is clear from an examination of the tables held by Dr Busby that this 'All leukaemias' group is simply an error:

Table A, a small extract from tables for 1988 provided by Dr Busby shows that for two sample areas the data for 'All leukaemias' are inconsistent with the data for the individual leukaemia sub-groups. Moreover the former do not appear in the 'All malignancies' category which appears to be a correct total for all the other diagnostic groups including the leukaemia sub-groups.

This shows that the WCISU data agree reasonably well with the aggregated data for leukaemia groups 204-208 that we believe to be a correct extract from the WCR files as they existed in 1995. For 'All Wales', the Busby 'All leukaemias' data agrees approximately with the WCISU and Busby 204-208 data (though it should agree precisely with the sum of 204-208), but within individual areas (county districts), the numbers are totally different. All the cases have been allocated to about half the areas - with zero entries for the remaining half.

The fact that the 'All leukaemias' category does, in total, have the right proportionate age-distribution for leukaemias leads us to speculate that these cases are the result of a computer programming error. The nature of the error is such that it seems more probable that it was an error at WCR in the original extraction of the data from the main computer files or in compiling sub-totals to form the 'All leukaemias' group. But this is speculation on our part.

As an independent check we have also been able to examine tapes of WCR data "frozen" in 1994 and 1996. Table 1 shows that in these two years the totals for leukaemia ICD sub-groups 204-208 for the period 1974-1989 agree well and are comparable with data currently held by WCISU, ONS, and NRCT. They also agree with the total of sub-groups 204-208 sent to Green Audit in 1995 (ie, excluding the aberrant tabulation of `All leukaemias'). There is, therefore, no evidence for any significant change in the data for childhood leukaemia held on file by WCR during the period in which the disputed data-set was sent to Green Audit.

3 Other possible sources of discrepancies

Differences in tabulations of cancer registry data can arise both from errors in the data and, more easily overlooked, from differences in definition. It was obviously essential that we should consider whether such differences could have contributed to the different conclusions about the incidence rates in Wales reached by COMARE and WCISU on the one hand and Green Audit on the other. We have concluded that although there are discrepancies between the ONS/WCR/WCISU/NRCT datasets they cannot account for the Green Audit findings.

3.1 Programming errors. The only programming error we know of is that which led to the creation of the 'All leukaemias' category appearing in tables provided by Dr Busby. We are convinced that this is responsible for his findings. No other explanation is plausible.

3.2 Failure to register cases. There is no reason to suppose that this is a major problem. See Table 2 which shows that of the 609 cases notified directly to the NRCT in 1974-89 only 17, ie, less than 3%, were not also recorded by cancer registries.

3.3 Duplicates. Review of the old WCR data showed that the file included 91 duplicate entries.

3.5 Date/age errors. This appears to be the major source of error in the WCR / ONS Wales data and in published tables based on these. Work is in hand to correct these errors.

3.6 Address. Cases are sometimes registered from a temporary address. There is no reason to suppose that this is a major problem here, but see Para 3.8.

3.7 Definition of cases to be included: Neoplasms/Registrable neoplasms/Malignant neoplasms. English and Welsh cancer registries register all malignant neoplasms ICD8/9, 140-208 and some non-malignant, ie, some of 210-239; non-malignant brain tumours are registrable. NRCT tables always include these. Some published cancer registry data include only ICD 140-208. It is possible that some, relatively minor, discrepancies between WCR / WCISU analyses and those by Busby et al arose from different case definitions in the programs used to define the tabulations. (These can arise from differing decisions concerning whether or not to include some possibly non-malignant conditions, and how to handle records with incomplete data.)

3.8 Area. Cases should be allocated to registry region according to domicile at diagnosis even if they are initially registered by another registry, such cases being re-allocated by inter-regional transfers. NRCT allocates cases to Wales on the basis of the census enumeration district in which the address postcode is located. These do not always agree, either because the inter-registry transfers have not taken place or because the different ways of defining a "Welsh" case (ie, by postcode or enumeration district) may misallocate addresses very close to the English / Welsh border.

Table 1 compares total numbers of childhood cancer cases aged 0-14 on various files, for Wales for the time periods 1984-88 and 1974-89.

We have compared NRCT data with cases recorded as age 0-14 on the ONS files i.e. the old WCR data. Some cases, particularly in 1984, 1987, and 1988 have incorrect dates of birth and are wrongly classified as children. It appears that in the process of linking WCR cases to ONS records a second, incorrect, identification number was allocated to some cases, which were consequently linked to the wrong record and hence allocated an incorrect date of birth. It is important to note that, because this resulted in adult cases being incorrectly classified as children, and because only 3% of adult cases of cancer are leukaemias (as against 33% of childhood cases), this error has little effect on the childhood leukaemia data that we are mainly concerned with.

Although the numbers of cases recorded by NRCT and ONS/WCR/WCISU are roughly comparable, it should be recognised that the discrepancies are greater than is apparent. For instance, in Table 1, of the 1028 childhood cases recorded by WCISU more than 40 are probably adults since they have diagnoses that are extremely rare in children (cf Para 3.5 above). Some differences in the cases selected for analysis files may be attributable to differences between registries in the data held for some cases.

Table 2 shows the number of Welsh registrations notified to the NRCT through the cancer registration system and through clinicians or clinical trials. It can be seen that nearly all cases ascertained through these independent sources were in fact also registered through the cancer registration system. For leukaemias the majority, 84% of cases, were notified through both sources. This suggests that there is a high level of ascertainment, at least for childhood leukaemia. In addition nearly all cases with other diagnoses that were recorded by clinicians or clinical trials were also recorded though the cancer registration system.

Table 1 Comparison of total numbers of cases of childhood cancer aged 0-14 in Wales* from various sources 1984-88, 1974-89

* 'Wales' in this table is defined by inclusion in WCR or WCISU by region of cancer registration for ONS, but by postcode/ED of domicile for NRCT.

** WCR is calculated from published data for annual average registrations multiplied by 5.

+ Counts obtained using 'frozen data files', ie, data preserved as they existed in 1994 and 1996 respectively on WCR files.

++ Total of leukaemia subgroups ICD 204, 205, 206, 207, 208 provided by WCR to Green Audit 1995 (ignoring the 'All leukaemia' category which is clearly erroneous).

Table 2 Independent registration of the 1051 NRCT Welsh cases 1974-89 (`Other sources of registration' are UKCCSG clinicians or clinical trials registries)