Kirsty G Hope, Tony D Merritt, David N Durrheim, Peter D Massey, Julie K Kohlhagen, Kerry W Todd, Catherine A D'Este

Abstract

Communicable disease monitoring and response activities must be based upon local public health surveillance systems, even during infectious disease emergence, natural disasters, and during bioterrorism events. The NSW Department of Health has developed an emergency department surveillance system intended to monitor important public health conditions during mass gatherings and to identify outbreaks of importance. An evaluation of this system conducted in the Hunter New England region of New South Wales emphasised its usefulness when it was focused on a limited number of syndromes of public health importance and during mass gatherings and public health disaster responses. Commun Dis Intell 2010;34(3):310–318.

Top of page

Introduction

During the past 2 decades, biological and other terrorism incidents, have resulted in health authorities globally investing considerable resources into implementing syndromic surveillance that identifies outbreaks of public health importance earlier than traditional surveillance systems.^1–3 However there is currently limited evidence that syndromic surveillance systems add value to local public health disease surveillance.^4–7

The real-time evaluation of an emergency department (ED) syndromic surveillance system in Taiwan identified peaks in seasonal community-wide illness, such as influenza and gastroenteritis.³ In addition, a syndromic surveillance system developed for the military in French Guiana identified a dengue fever outbreak 3 weeks earlier than traditional surveillance, permitting early implementation of control measures.⁸ However, the ability to detect localised disease clusters remains an elusive goal for many syndromic surveillance systems.^4,5–9

Syndromic surveillance

Syndromic surveillance generally relies on recognising clinical features that are discernible before a diagnosis is confirmed. Data sources used include ED primary diagnoses, ambulance dispatches and sales of over-the-counter medication.¹⁰ Unlike traditional surveillance systems, which may rely on voluntary passive reports from health practitioners, syndromic surveillance systems rely on continual data acquisition generated by routine activities that are generally electronically transmitted.² While many syndromic surveillance systems are focused on detecting bioterrorism events, it is important that these are integrated into routine health surveillance to ensure staff become familiar with them and know how to interpret signals.

New South Wales real-time public health emergency department surveillance system

In 2003, the Centre for Epidemiology and Research developed a real-time public health emergency department surveillance system in response to biosecurity threats associated with the Rugby World Cup.¹¹ Currently, over 50 EDs provide data to the system; eight are situated within the Hunter New England (HNE) area in northern New South Wales. HNE currently has 38 EDs, ranging from 1 large tertiary referral hospital to small general practitioner-serviced multi-purpose centres. The 8 hospitals include the tertiary referral hospital and one of the 2 rural referral hospitals in the region.¹²

The NSW Department of Health (NSW Health) receives 4 hourly electronic file transmission from participating EDs (using HL7 messaging) containing data items including the primary ED diagnoses (ICD-10 codes), which are mapped to one of 39 syndrome categories.¹¹

Statistical control charts are used to automatically detect increases in syndrome activity (signals) based on the previous 51 weeks' data. The measures used to detect increased activity include:

1. when a specific day's ED presentations exceed the average number of presentations for that day for the previous 51 weeks, by more than 5 standard deviations;
2. when a specific week's ED presentations exceed the average number of presentations for the previous 51 weeks, by more than 5 standard deviations and
3. when a 15-fold increase occurs in the modified cumulative sum. This is called the Index of increase.

The cumulative sum is calculated by cumulatively summing the difference between the previous day's count and the count 7 days prior. The accumulated cumulative sum is divided by the mean syndrome count for the available baseline up to a maximum 365 days. The mean-standardised cumulative sum is then divided by the standard deviation of its differenced values, again using whatever baseline is available up to a maximum of 365 previous days.^11,13

Regional public health services

New South Wales public health units are legally mandated to monitor and respond to infectious disease and other public health threats.¹⁴ This is facilitated by notification of specific diseases by laboratories, doctors and institutions.¹⁵ To enhance regional surveillance, the ED syndromic surveillance system would need to identify these specific diseases earlier or more completely than the traditional notification systems, or identify other disease threats not covered by existing surveillance.

The regional objectives identified by Hunter New England Population Health (HNEPH) prior to the evaluation were:

1. routine monitoring to identify cases or clusters of public health importance requiring a prompt response, without duplicating existing surveillance systems; and
2. during emergency situations or mass gathering events, to actively monitor ED presentations for situational awareness.

While the system has met state level surveillance objectives,^11,13,16 the current evaluation aimed to determine the utility of the New South Wales ED syndromic surveillance in a regional public health service.

Top of page

Methods

The US Centers for Disease Control and Prevention 'framework for evaluating public health surveillance system for early detection of outbreaks' was used to guide the evaluation process.¹⁷ Some system attributes, including algorithms, information technology platform and syndrome groupings were not assessed during this evaluation. Outbreak detection was assessed prospectively for all syndromes and retrospectively for a subset of syndromes classified as syndromes of public health importance.

Prospective signals produced by the ED surveillance system were investigated to determine if they represented an outbreak; while retrospectively, signals were compared against known outbreaks reported through existing surveillance systems. System experience including the usefulness, flexibility and acceptability of the system was assessed through a stakeholder survey during the prospective evaluation component.

Prospective evaluation

Each weekday (excluding public holidays) between 1 January and 31 December 2008 the ED surveillance reports for participating HNE hospitals were accessed and any syndrome signal noted. The date of signal; syndrome; type of signal (daily, weekly, index of increase); hospital; threshold level; standard deviation of the signal (i.e. signal strength) and whether it was the first signal for that hospital and syndrome in that week, were systematically documented. In addition, the investigation description, whether an outbreak was confirmed and the resulting outbreak response, were also recorded.

An investigation hierarchy was used with each progression involving a more in depth public health response. The initial investigation was labelled a 'subgroup' investigation which involved reviewing age, sex, admission and triage categories of patients that had elicited the signal. If a subgroup investigation showed clustering by age, sex, location or admission status, an 'internal' investigation was conducted. An internal investigation involved a public health unit (PHU) staff member reviewing the ED patient database or pathology database to ascertain further information on demographic characteristics, clinical assessment and pathology results of patients. If warranted an 'external' investigation was then conducted, involving PHU staff contacting ED directors or treating physicians to ascertain further patient-specific clinical or epidemiological information. The time taken was also recorded.

Syndromes of public health interest

A reference group consisting of surveillance, public health and epidemiology staff reviewed the list of ED syndromes and selected a syndrome subset, classified as syndromes of public health interest, based on: their severity or public health consequence; potential for large outbreaks; limitations of traditional surveillance; and local disease epidemiology. The syndromes selected were: 'meningitis', 'pneumonia', 'influenza', 'gastrointestinal' and 'poisoning'. Signals and investigations were however, recorded for all syndromes.

Retrospective evaluation

Standard ED surveillance reports for 2005–2007 were generated for the syndromes; 'gastrointestinal', 'influenza', 'pneumonia' and 'meningitis', and all signals identified. Reports for selected syndromes were also generated by a restricted age grouping (5–65 years) and 'admission to hospital' status and all signals identified. The signals were compared to known outbreaks recorded in 3 existing surveillance systems; notifiable diseases, OzFoodNet foodborne outbreaks and the institutional gastrointestinal and respiratory outbreak databases.

Stakeholder survey

A questionnaire was sent to local stakeholders including ED directors, PHU and laboratory staff, and senior managers who received NSW Health emails reporting syndrome signals. Participants were asked how often they read these emails, whether they had taken any action in response to an email, the outcome of any action, and their preference for future email alerts. This self-administered questionnaire was sent and returned electronically, with prompting by a senior PHU staff member after a month if a response had not been forthcoming.

Top of page

Results

Prospective evaluation

All syndromes

During 2008, a total of 958 signals occurred across the 8 EDs; 237 daily signals, 467 weekly signals and 254 index of increase signals (Table 1). Elevated counts in 1 syndrome could result in signals over multiple days and across multiple categories (daily, weekly, index of increase). When repeat signals were ignored, there were 382 initial signals by syndromes.

Table 1: Summary of signals produced by the New South Wales emergency department syndromic surveillance system for Hunter New England Area Health Service participating emergency departments, 1 January to 31 December 2008

Syndromes	Alerts in standard reports					Investigation
	Total	Initial	Daily	Weekly	Index	Sub-group†	Internal‡	External§
All syndromes	958	412	237	467	254	366	25	6
Syndromes public health interest	143	76	45	85	11	100	22	6
Abdominal pain	2	2	0	2	0	1	0	0
Convulsions (not clearly epilepsy)	7	7	6	1	0	7	0	0
Collapse/ syncope/coma/ delirium/ dizziness	0	0	0	0	0	0	0	0
Neuromuscular/ vision problems	19	16	14	5	0	16	2	0
Cough	24	17	13	7	4	13	0	0
Headache/ migraine	2	2	2	0	0	2	0	0
Malaise/fatigue	30	19	7	21	2	18	0	0
Meningitis/ encephalitis*	18	15	6	5	5	17	10	4
Dehydration*	11	11	10	1	0	11	0	0
Gastrointestinal	6	5	3	3	0	6	3	1
All cardiovascular/ chest pain	1	1	1	0	0	1	0	0
Chest pain	0	0	0	0	0	0	0	0
Cardiac arrest	9	7	6	3	0	8	0	0
Cardiac dysrhythmias	3	3	2	1	0	3	0	0
All respiratory diagnoses	175	29	2	53	120	27	0	0
Asthma	16	6	3	7	6	6	0	0
Influenza	78	29	14	58	6	42	2	0
Pneumonia	23	13	10	13	0	20	4	1
Other/unspecified respiratory infections	106	31	11	58	37	29	1	0
Respiratory failure/distress	3	3	2	1	0	0	0	0
Bronchiolitis	51	23	11	30	10	13	0	0
All injury diagnoses	7	6	1	4	2	0	0	0
Joint injury	17	6	2	9	6
Head injury	5	3	2	3	0	2	0	0
Burns	6	6	3	2	1	6	0	0
Bite or sting (insect/spider/ snake)	158	53	47	92	19	15	0	0
Open wounds	9	1	0	7	2	0	0	0
Hypothermia*	9	3	1	7	1	5	0	0
Heat stroke*	12	6	0	7	5	5	0	0
Illicit drugs diagnoses	8	8	7	0	1	4	0	0
Alcohol acute effects	15	13	11	4	0	8	0	0
Unspecified infection	26	15	6	14	6	9	0	0
Skin problems	24	9	3	17	4	17	0	0
Poisoning (not illicit drug or alcohol)	18	13	12	6	0	15	3	0
Mental health diagnosis	3	2	2	1	0	2	0	0
Admitted to critical care unit	0	0	0	0	0	0	0	0
Death in emergency department	10	10	9	0	1	5	0	0
Triage one	11	8	4	7	0	6	0	0
Ambulance arrival	0	0	0	0	0	0	0	0
All unplanned visits	38	2	4	18	16	2	0	0

* Syndromes added or altered in August 2008.

† When a signal occurred the emergency department syndromic surveillance system, then produces a breakdown page by subgroup, using the same algorithms.

‡ Entailed a public health unit staff member accessing the emergency department patient database, the pathology database or NetEpi.

§ When a public health unit staff member contacted a person external to the public health unit such as an emergency department director or treating physician

Top of page

Overall, 366 (38%) signals were investigated to the sub-group level with an internal investigation necessary for 25 (3%) and an external investigation on 6 occasions (0.6%). The remaining signals were not investigated. No HNE outbreaks were detected by the ED surveillance system during 2008, while existing surveillance systems detected 17 gastroenteritis outbreaks and 9 meningococcal cases were notified during this period.

The 6 external investigations included four for meningitis and one each for gastrointestinal and pneumonia syndromes. Of the 4 meningitis syndrome signals that led to an external investigation, 1 viral meningitis cluster was identified by the ED surveillance system with additional information concurrently received through existing surveillance systems. One suspected bacterial meningitis case was identified, which had not been notified through traditional surveillance but was subsequently proven not to be meningococcal disease. None of the 9 meningococcal disease notifications were identified by the ED surveillance system but four had a purely septicaemic presentation and would likely not be identified as a meningitis syndrome.

The single gastrointestinal and pneumonia signals were investigated externally with no outbreak detected. However, the pneumonia signal external investigation resulted in a better understanding of ED coding practices with only two of the 6 presentations subsequently confirmed as pneumonia cases.

Of the 3 poisoning signals investigated internally, two were chemical exposures requiring attendance by the Fire Department's Hazardous Materials Response Team but both had already been notified through regular emergency communication channels to the PHU. The signals however demonstrate the ED surveillance system's ability to identify acute events requiring ED presentation due to chemical exposures.

The initial signal review took a median time of 15 minutes (range 2–20 minutes). Where further investigation was required, the time required ranged from a 10 minute telephone consultation with a physician to 4 hours checking clinical details and laboratory results.

Fourteen per cent (143 of 958) of signals related to a syndrome of public health interest. All 2008 external investigations related to one of these 5 syndromes.

Retrospective evaluation

The retrospective analysis identified 1 pneumonia signal, which was associated with a confirmed outbreak, and an increase in influenza and gastroenteritis signals during the winter and spring months respectively, but no other outbreaks were identified.

The pneumonia outbreak was first reported on 10 August 2006 by a hospital paediatrician who noted an unusual pneumonia cluster in young, previously healthy males.¹⁸ The ED surveillance system signalled in the 'pneumonia' syndrome 3 days later. However, by stratifying the pneumonia syndrome by age group, a signal appeared in the 5–16 year age group 4 days earlier than the paediatrician's notification and this signal was maintained for 14 days. This signal would not have been identified by current ED standard reports.¹³ Thirty-five of the 69 pneumonia signals during 2008 occurred in this age group.

Influenza signals were increased (158) during June to September in 2007 compared with 2006 (78) and 2008 (78). The increases were predominantly in 2 non-metropolitan EDs but no specific outbreak was identified and existing surveillance systems indicated 2007 was generally a more severe influenza season than 2006 and 2008.¹⁹ The influenza syndrome ICD-10 codes were used infrequently by all HNE EDs. In HNE during 2008, counts at individual EDs ranged from nine to 57 with a median of 13.

Most gastroenteritis syndrome signals between 2005 and 2007 occurred in 2 hospitals, both being referral centres for children. The majority of alerts occurred during September and October each year, but this trend was absent in 2008 (Table 2). There was no clear association between known outbreaks and ED surveillance signals (Table 2). However, information captured in the OzFoodNet outbreak investigation database indicated that very few known outbreak cases presented to hospital.

Table 2: Comparison of gastrointestinal outbreak notifications in institutions, OzFoodNet notifications and emergency department syndromic surveillance signals, 2006 to 2008

	2008			2007			2006
	Institutions	OzFoodNet	ED signal	Institutions	OzFoodNet	ED signal	Institutions	OzFoodNet	ED signal
Jan	4	3	1	5	1	0	2	1	3
Feb	3	1	0	4	2	0	3	1	0
Mar	5	1	0	6	0	0	6	1	0
Apr	4	1	0	6	1	0	7	1	0
May	10	0	0	6	0	0	11	0	1
Jun	5	0	0	5	0	0	12	4	4
July	7	0	0	9	2	0	19	0	0
Aug	17	1	3	18	0	2	13	0	18
Sept	12	0	0	9	1	1	4	2	56
Oct	5	0	0	20	1	69	10	2	26
Nov	7	2	0	14	0	12	6	0	0
Dec	4	1	3	10	1	2	6	2	0

ED Emergency department.

Top of page

Stakeholder survey

Six of the 7 ED directors returned completed questionnaires. Only one of the 6 ED directors surveyed always read the alert emails, others responded that they only did so intermittently depending on workload. None of the ED directors reported taking any action from the alerts unless the PHU contacted them.

Six of the 10 public health staff surveyed returned completed questionnaires. All six reported utilising the surveillance data during mass events and disaster responses, 67% indicated they had used the surveillance data to inform public health action, while all indicated they still wanted to receive the alert emails, with five only wanting to know about a restricted number of syndromes.

Additional applications

Acute events surveillance

Severe storms and extensive flooding occurred in the Hunter Valley during the June 2007 long weekend (Friday 8 to Monday 11 June) resulting in the region being declared a natural disaster area.²⁰ No gastroenteritis outbreaks were identified by the ED surveillance system during the recovery phase. The system detected increased presentations of respiratory syndromes, which were within seasonally expected levels when compared with data for the previous 5 years. The ED data informed response planning, while providing reassurance that there were no large infectious disease outbreaks threatening the health of the disaster affected population.²¹ The surveillance system acquired data on all presentations to EDs, allowing consideration of additional syndromes or conditions not included in the standard surveillance reports. This occurred during the 2007 Hunter storms when information was provided on hypothermia presentations to local EDs. Hypothermia was subsequently added to the standard reports in 2008.

Mass gatherings surveillance

The Tamworth country music festival is a mass gathering occurring for 2 weeks each January, ending on the Australia Day long weekend, with a doubling of the Tamworth population from 40,000 to over 80,000 people. Temporary camping facilities accommodate the influx of people, and many transitory food vendors cater for the crowd. Enhanced ED surveillance through the New South Wales ED syndromic surveillance system has been used during the festival since 2007 with data reported daily to the local disaster emergency management team. Signals have prompted public health investigations, for example clusters of otitis externa and respiratory illness in 2007, informed workforce planning and assisted in prioritising public health activity and media messages.

Top of page

Discussion

The New South Wales real-time emergency department surveillance system was evaluated from the perspective of a regional PHU for its capacity to identify cases or clusters of public health importance requiring a prompt response, without duplicating existing surveillance systems; and for enhanced surveillance during emergency situations or mass gathering events for situational awareness.

The ED surveillance system is potentially a useful tool to assist with situational awareness, particularly during natural disasters and mass gathering events. Recent experience in HNE has demonstrated the value of ED syndromic surveillance in both these circumstances. The recent H1N1 pandemic also established the value of the ED syndromic surveillance system in monitoring statewide demand on ED services during a prolonged public health emergency.²² The system is flexible, allowing for adding or adapting syndromes in response to changing situations, and accommodates tight reporting time frames. ED surveillance data have informed health messages for the media and guided response planning. Previous reviews of the use of syndromic surveillance systems during acute events support their efficacy.^23–25

This study identified potential utility for this surveillance system to detect public health threats requiring prompt intervention for a few specific syndromes. During 2008, only six of 958 signals across 8 HNE EDs required further public health intervention. The prospective evaluation of the 39 syndromes provided empirical support for focusing on only 5 syndromes (gastrointestinal, meningitis, pneumonia, influenza and poisoning). If daily monitoring had been restricted to these syndromes then there would have been substantially fewer signals (143 in table versus 958) to investigate. System acceptability and representativeness could be improved by including the second rural referral hospital in HNE and by investigating the specific surveillance needs of ED staff.

Outbreaks are relatively rare events and their severity determines whether there are ED presentations. There is no gold standard of outbreak identification to which to compare syndromic surveillance, as existing surveillance systems themselves do not detect all outbreaks even with a delay. This complicates the determination of the sensitivity and specificity of the ED syndromic surveillance system. During the prospective evaluation, 1 viral meningitis cluster was detected by the ED surveillance system, concurrently with existing surveillance systems. In addition, the system did not identify any outbreaks during the Hunter 2007 storms or during mass gathering events in the region, consistent with results from existing surveillance.

The single pneumonia outbreak indicated that although the syndrome may not signal earlier than reporting by an astute clinician, a narrower age-band excluding the 'noise' generated by the very young and old, may be a more sensitive measure of unusual respiratory outbreak activity. For example, during the second wave of an influenza pandemic, when different age groups may be hospitalised compared with those usually affected by seasonal influenza.

Past evaluations of emergency department syndromic surveillance systems have found that they can detect community wide outbreaks, such as seasonal influenza and gastroenteritis, however their effectiveness in identifying smaller clusters of interest to local PHUs has not been established.^4–7 When known gastrointestinal outbreaks investigated by OzFoodNet and institutional outbreaks reported to the HNE PHU were compared to the gastrointestinal syndrome signals, no known outbreaks were identified by the ED surveillance system. The OzFoodNet database did indicate that a restricted number of people had presented to EDs, however the high level of 'background noise' and fixed threshold level reduced the likelihood of a signal being generated. Similarly, none of the notified cases of meningococcal disease in 2008 resulted in a signal.

While detection of the influenza and gastroenteritis season commencement has value at a state level for providing information on severity and spread and to inform media releases, such seasonal trends are of less value at the local level where outbreak detection and investigation are the priority.

While local objectives were developed for this evaluation, it is important that specific operational objectives are established for the ED surveillance system to guide reporting and investigation of signals. As each syndrome represents a separate disease or condition it may be necessary for each syndrome to have its own surveillance objectives describing local public health relevance (Box). Clear regional objectives may also assist in engaging EDs in the reporting process, as the benefit of investing time may become more evident. ED surveillance objectives should complement existing surveillance systems rather than duplicate efforts.

Box: Examples of surveillance objectives Objectives for pneumonia syndrome Identify unusual clusters of pneumonia that require a rapid response, without duplicating existing surveillance systems (e.g. Legionella) Identify bioterrorism event Objectives for meningitis/encephalitis syndrome Identify unusual clusters of viral meningitis or encephalitis, to ensure appropriate testing Objectives for monitoring poisoning syndrome Identify chemical exposures requiring response Identify foodborne poisoning events requiring response Objectives for gastrointestinal syndrome Identify unusual clusters of severe gastrointestinal disease that require an acute and timely response, without duplicating existing surveillance systems Objectives during acute events/disaster response Identify clusters or increasing trends in presentations of public health importance that require a rapid response

Top of page

Limitations

During the evaluation, the ED standard reports were generally only monitored on weekdays; reports from Fridays and Saturdays were not reviewed until Monday, except during the declared disaster and mass event monitoring. This could have lead to delays in event detection. The retrospective reproduction of the ED standard reports may differ slightly to reports that were produced in real time. This is due to higher data completeness in the retrospective data compared to the real-time data.

During this evaluation it was not possible to adequately measure the 'cost-effectiveness' of the ED syndromic surveillance system. While PHU staff time required to follow-up on signals was recorded, information on the cost of setting up and maintaining the system was not available to the researchers. Therefore it is not possible to determine whether the costs of providing situational awareness and community reassurance are justified, nor can it prospectively be determined if the system will repay its running costs by averting a major disaster.

The stakeholder survey conducted as part of this evaluation only consisted of a small number of participants; which limits the ability to generalise the findings. When applying these results to other regions it is important to consider that ED patient management systems and coding practices may vary across EDs and regions. Therefore the performance of specific syndromes may differ between hospitals and regions.

Conclusion

ED syndromic surveillance may inform local public health action or serve as a surveillance safety net for traditional surveillance when focused on pneumonia, meningitis/encephalitis, poisoning and possibly gastrointestinal syndromes. It appears to have specific local utility during mass gathering or disaster response surveillance. Clear objectives for each syndrome are needed, emphasising the difference between local and state surveillance objectives and variability between syndromes. A handbook of response options may prove valuable in guiding the response to specific syndromic surveillance signals.

Top of page

Acknowledgements

The authors acknowledge the Hunter New England Public Health Unit staff for all their input and support during this evaluation and the NSW Health Centre for Epidemiology and Research especially David Muscatello and Wai Zheng for their ongoing support and important work in developing and maintaining the syndromic surveillance system. The authors also acknowledge the funding support provided by the NSW Department of Health through the Hunter Medical Research Institute.

Author details

Kirsty G Hope^1,2

Tony D Merritt^1,2

David N Durrheim^1,2

Peter D Massey^1,2

Julie K Kohlhagen¹

Kerry W Todd¹

Catherine A D'Este²

1. Hunter New England Population Health Unit, Wallsend, New South Wales

2. University of Newcastle, Newcastle, New South Wales

Corresponding author: Ms Kirsty Hope, Hunter New England Population Health, Locked Bag 10, WALLSEND NSW 2287. Telephone: +61 2 4924 6496. Facsimile: +612 4924 6408. Email: kirsty.hope@hnehealth.nsw.gov.au

Top of page

References

1. Reis B, Mandl K. Syndromic surveillance: the effects of syndrome grouping on model accuracy and outbreak detection. Ann Emerg Med 2004;44(3):235–241.

2. Mandl K, Overhage J, Wagner M, Lober W, Sebastiani P, Mastashari F, et al. Implementing syndromic surveillance; a practical guide informed by the early experience. J Am Med Inform Assoc 2004;11(2):141–150.

3. Wu T, Shih F, Yen M, Wu J, Lu S, Chang K, et al. Establishing a nationwide emergency department-based syndromic surveillance system for better public health responses in Taiwan. BMC Public Health 2008;8:18.

4. Hope K, Durrheim D, D'Espaignet ET, Dalton C. Syndromic surveillance: is it a useful tool for local outbreak detection? J Epidemiol Community Health 2006;60(5):374–375.

5. Bravata DM, McDonald KM, Smith WM, Rydzak C, Szeto H, Buckeridge DL, et al. Systematic review: surveillance systems for early detection of bioterrorism-related diseases. Ann Intern Med 2004;140(11):910–922.

6. McLeod M, Mason K, White P, Read D. The 2005 Wellington influenza outbreak: syndromic surveillance of Wellington Hospital Emergency Department activity may have provided early warning. Aust N Z J Public Health 2009;33(3):289–294.

7. Wallace DJ, Arquilla B, Heffernan R, Kramer M, Anderson T, Bernstein D, et al. A test of syndromic surveillance using a severe acute respiratory syndrome model. Am J Emerg Med 2009;27(4):419–423.

8. Meyard J, Chaudet H, Texier G, Ardillon V, Ravachol F, Deparis X, et al. Value of syndromic surveillance within the armed forces for early warning during a dengue fever outbreak in French Guiana in 2006. BMC Med Informatics Decision Making 2008;8:29–38.

9. Buckeridge D. Outbreak detection through automated surveillance: a review of the determinants of detection. J Biomedical Informatics 2007;40:370–379.

10. Centers for Disease Control and Prevention. Syndromic Surveillance: reports from a national conference, 2004. MMWR Morb Mortal Wkly Rep 2005;54(Suppl):1–180.

11. Muscatello D, Churches T, Kaldor J, Zheng W, Chiu C, Correll P, et al. An automated, broad-based, near real-time public health surveillance system using presentations to hospital Emergency Departments in New South Wales, Australia. BMC Public Health 2005;5:141–152.

13. Hunter New England Area Health Service. HNE services and facilities. Accessed on 10 February 2010. Available from: http://www.hnehealth.nsw.gov.au/services_and_facilities

14. Hope K, Durrheim D, Muscatello D, Merritt T, Zheng W, Massey P, et al.Identifying pneumonia outbreaks of public health importance: can emergency department data assist in earlier identification? Aust N Z J Public Health 2008;32(4):361–363.

15. New South Wales Government. NSW Public Health Act 1991. Accessed on 10 February 2010. Available from: http://www.austlii.edu.au/au/legis/nsw/consol_act/pha1991126/

16. NSW Department of Health. Notification of Infectious Diseases under the Public Health Act 1991. 2006. Policy directive PD2006_014. Accessed on 10 February 2010. Available from: http://www.health.nsw.gov.au/policies/pd/2006/pdf/PD2006_014.pdf

17. Zheng W, Aitken R, Muscatello DJ, Churches T. Potential for early warning of viral influenza activity in the community by monitoring clinical diagnoses of influenza in hospital emergency departments. BMC Public Health 2007;7:250.

18. Buehler J, Hopkins R, Overhage J, Sosin D, Tong V, CDC Working Group. Framework for evaluating public health surveillance systems for early detection of outbreaks: recommendations from the CDC working group. MMWR Recomm Rep 2004;53(RR–5):1–11.

19. Cashman P, Massey P, Durrheim D, Islam F, Merritt T, Eastwood K. Pneumonia cluster in a boarding school – implications for influenza control. Commun Dis Intell 2007;31(3):296–298.

20. Dalton C, Durrheim D, Fejsa J, Francis L, Carlson S, d'Espaignet ET, et al. Flutracking: A weekly Australian community online survey of influenza-like illness in 2006, 2007 and 2008, Commun Dis Intell 2009;33(3):316–322.

21. Cretikos MA, Merritt TD, Main K, Eastwood K, Winn L, Moran L, et al. Mitigating the health impacts of a natural disaster— the June 2007 long-weekend storm in the Hunter region of New South Wales. Med J Aust 2007;187(11–12):670–673.

22. Hope K, Merritt T, Eastwood K, Main K, Durrheim D, Muscatello D, et al. The public health value of emergency department syndromic surveillance following a natural disaster, Commun Dis Intell 2008;32(1):92–94.

23. Churches T, Conaty SJ, Gilmour RE, Muscatello DJ. Reflections on public health surveillance of pandemic (H1N1) 2009 influenza in NSW. N S W Public Health Bull 2010;21(1–2):19–25.

24. Toprani A, Ratard R, Straif-Bourgeois S, Sokol T, Averhoff F, Brady J, et al. Surveillance in hurricane evacuation centers—Louisiana, September–October 2005. MMWR Morb Mortal Wkly Rep 2006;55(2):32–35.

25. Steiner-Sichel L, Greenko J, Heffernan R, Layton M, Weiss D. Field investigations of emergency department syndromic surveillance signals—New York City. MMWR Morb Mortal Wkly Rep 2004; 3(Suppl):184–189.

26. Carrico R, Goss L. Syndromic surveillance: hospital emergency department participation during the Kentucky Derby Festival. Disaster Manag Response 2005;3(3):73–79.