David J Muscatello, Patricia M Morton, Ingrid Evans, Robin Gilmour

Abstract

Influenza is a serious disease that seasonally causes varying but substantial morbidity and mortality. Therefore, strong, rapid influenza surveillance systems are a priority. Surveillance of the population mortality burden of influenza is difficult because few deaths have laboratory confirmation of infection. Serfling developed a statistical time series model to estimate excess deaths due to influenza. Based on this approach we trialled weekly monitoring of excess influenza mortality. Weekly, certified death information was loaded into a database and aggregated to provide a time series of the proportion of all deaths that mention pneumonia or influenza on the death certificate. A robust regression model was fitted to the time series up to the end of the previous calendar year and used to forecast the current year's mortality. True and false alarm rates were used to assess the sensitivity and specificity of alternative thresholds signifying excess mortality. Between 1 January 2002 and 9 November 2007, there were 279,968 deaths registered in New South Wales, of which 77% were among people aged 65 years or more. Over this period 33,213 (12%) deaths were classified as pneumonia and influenza. A threshold of 1.2 standard deviations highlighted excess mortality when influenza was circulating while providing an acceptable false alarm rate at other times of the year. Prospective and reasonably rapid monitoring of excess mortality due to influenza in an Australian setting is feasible. The modelling approach allows health departments to make a more objective assessment of the severity of seasonal influenza and the effectiveness of mitigation strategies. Commun Dis Intell 2008;32:435–442.

Keywords: influenza, surveillance, Serfling, excess mortality, robust regression, false alarm rate, sensitivity, specificity, seasonal

Top of page

Introduction

Influenza, an acute viral disease of the upper respiratory tract, is a major threat to public health worldwide because of its capacity for distinct mutation (antigenic shift) that can result in rapid spread through populations and widespread morbidity and mortality.^1–3 Even in the absence of such pandemics, seasonal influenza epidemics cause substantial burden of morbidity and mortality annually.^4–9 More frequent, minor mutations (antigenic drift), cause substantial variability in the impact of seasonal epidemics.^2,3 The capricious nature of influenza demands constant vigilance to ensure seasonal vaccines are appropriate to current strains, and supplies of these and antiviral medications are sufficient for potential need. For these reasons, strong influenza surveillance systems are a priority for health departments.¹⁰

Assessing the population burden of influenza is difficult. Symptoms are non-specific and few clinical influenza diagnoses are laboratory confirmed. Hospitalisations and deaths from influenza are often due to secondary complications such as pneumonia that occur well after the initial influenza virus infection.¹¹ Therefore, influenza may not be listed on death certificates for many influenza related deaths because it is not recognised as the underlying cause of the condition.

Because of these difficulties, statistical models were developed to estimate the burden of mortality caused by influenza. As early as 1932, Collins determined that excess mortality during winter months in the United States of America (USA) was a consequence of epidemic influenza and therefore could be used as an indicator for the recognition of influenza outbreaks.¹² In 1963, Serfling described a regression model, based on the seasonal pattern of pneumonia and influenza (P&I) deaths, to infer excess deaths due to influenza. Since then, Serfling's model has been applied in a number of temperate countries (USA,^13,14 France,¹⁴ Australia,¹⁴ Italy¹⁵) to demonstrate that excess mortality occurring during winter months is associated with pandemic and seasonal epidemics of influenza.

The US Centers for Disease Control and Prevention (CDC) took the approach further and established the 122 Cities Surveillance System to provide timely, prospective information of excess mortality due to influenza. The system collects and reports weekly counts by age of all deaths registered, usually within a week of the date of death, in 122 cities around the country. Deaths due to P&I are also counted, so that the weekly proportion of all deaths due to pneumonia and influenza can be monitored. The system covers approximately one third of all deaths in the USA and allows epidemiologists to determine an early quantitative estimate of the severity of an influenza epidemic.^16,17

In Australia, prospective seasonal surveillance of influenza is largely based on: targeted laboratory surveillance, specific subtype and strain identification of circulating influenza viruses by the World Health Organization Collaborating Centre for Research on Influenza in Victoria; and general practice or emergency department consultation rates for influenza-like illness (ILI).¹⁸ The New South Wales Influenza Surveillance Program runs from May to September each year and has primarily monitored the proportion of influenza positive specimens from all respiratory specimens from the major public health laboratories, the proportion of emergency department visits diagnosed with influenza, and outbreaks of influenza reported to Public Health Units by residential care facilities.¹⁹

This paper describes the introduction of prospective monitoring of excess P&I mortality due to influenza in New South Wales and demonstrates a sound and repeatable statistical approach to the selection of a threshold for signalling excess mortality due to influenza.

Top of page

Methods

Data source

Under the New South Wales Public Health Act 1991, the Registrar of Births, Deaths, and Marriages is required to make death registry information available for inspection by the NSW Department of Health.²⁰ Prior to 2007, incremental, password-protected updates in the form of a data file were transmitted monthly to the Department of Health's Population Health Division. Each update was saved on a secure, password-protected server²¹ using SAS statistical software²² and was only accessible by authorised departmental officers. In early 2007 the Registry of Births, Deaths and Marriages commenced providing weekly updates of the information.

The data file contains an electronic copy of the Medical Certificate of Cause of Death (death certificate) for each death certified in New South Wales. The certificate includes a section describing the disease or condition directly leading to death, and any antecedent causes, co-morbid conditions, or other significant contributing conditions. Deaths are required to be certified within 7 days. Deaths referred to a Coroner are not immediately certified and are therefore not available until completion of the coronial inquest.

For the period the data were provided weekly, we assessed the timeliness of the mortality data by calculating the median interval between the date of each death and the date it was provided to the Department of Health.

Top of page

Identification of pneumonia and influenza deaths

The SAS program that loads the data includes a module that scans the cause of death text for the text strings 'PNEUMONIA' or 'INFLUENZA', including some common misspellings (available on request). The words 'HAEMOPHILUS INFLUENZAE' and 'ASPIRATION' (for aspiration pneumonia) are excluded. The resulting dataset includes the date of death and a flag indicating whether either of the 2 conditions was mentioned. The data are then aggregated into weekly counts by date of death to provide a time series of the proportion of all deaths that mention pneumonia or influenza on the death certificate.

To validate the automatic classification of P&I deaths using the SAS program above, we created 3 samples of death certificates from the complete death certificate dataset from 1 January 2002 to 9 November 2007: 299 deaths that were automatically categorised as influenza or pneumonia, 120 deaths categorised as influenza and 299 deaths categorised as neither influenza or pneumonia. There were insufficient deaths mentioning influenza to obtain a sample of 299. The 'uniform' function in SAS software was used to select the random samples. Author PM manually compared the cause of death text with the categorised value. Any inconsistency was defined as a misclassification.

Statistical analysis

The time series of the proportion of P&I deaths is strongly seasonal with a winter peak. Serfling's method involved fitting a seasonally cyclical linear regression model to the time series of weekly proportions of P&I deaths. The details of the model specification are described below. Because the influenza epidemics cause excess deaths, or outliers, in the observed data, the model fitting could be overly influenced by the epidemic behaviour it is trying to detect. For this reason, Serfling manually excluded past epidemics from the model fitting. An upper threshold was chosen to define the upper limit of the expected proportion of P&I deaths in the absence of an influenza epidemic. The threshold was the predicted proportion in a given week plus a constant multiple of the standard error of the time series of differences between each value predicted by the model and the actual observed values (the 'model residuals'). Serfling chose the constant multiple to be 1.64 standard errors, and considered 2 consecutive weeks above the threshold to indicate epidemic behaviour.²³

During the 2007 influenza season we implemented a simplification of Serfling's method to demonstrate the feasibility of weekly, prospective excess P&I mortality monitoring. Each week, we fit a cyclic linear regression model to the full time series including data for the previous 5 calendar years and the current year-to-date. The threshold indicating an excess proportion of deaths due to pneumonia or influenza was defined as the upper 95% confidence limit of the expected mean proportion of P&I deaths predicted by the model, as output by PROC REG in SAS statistical software.²³ Graphs of the raw time series and the upper threshold using a model fit to the latest available data were incorporated into weekly seasonal influenza surveillance reports.

Top of page

This paper concentrates on a more sound variation of the above approach similar to that adopted by the US CDC.²⁴ The approach involves fitting the same cyclic linear regression model, but using a 'robust' estimation procedure for fitting the model. Robust regression down-weights the influence of extreme observations (outliers) in the model-fitting procedure.²⁵ The model is fit to the 5-year time series up to the end of the previous calendar year and the model is used to forecast expected behaviour for the current year. Forecasting the current year's time series from data up to the end of the previous year ensures the threshold is consistent from week to week in the current year.

The cyclic regression model includes: a linear time term, t, with values 1, 2, 3,... for each week of the time series, and the square of the time term, t², to accommodate long-term linear and curvilinear changes in the background proportion of P&I deaths arising from factors such as population growth or improved disease prevention or treatment. Also included are annual seasonal harmonic variables to describe the cyclical seasonal background pattern. The harmonic variables are functions of the week number, t, and the periodicity in the same units – in this case, yearly (52.18 weeks). The 2 harmonic variables in this case are: sine(2π t/52.18) and cosine(2π t/52.18).

The final model was:

Expected(proportion) = A + Bt + Ct² +D sine(2πt/52.18) + E cosine(2πt/52.18)

where A, B, C, D, and E

To evaluate the approach, we fitted the model to the 5-year time series from 1 January 2002 to 31 December 2006 using PROC ROBUSTREG in SAS Software with the simplest, default 'M estimation' method.²⁶ We then used the PROC SCORE procedure to forecast values for the 2007 year.²³

Top of page

Threshold identification

To identify a threshold at which to signal excess mortality, an estimate of the expected variability of the weekly proportion of P&I deaths is required. If an observed proportion exceeds the expected range of variability, then excess mortality can be signalled. Because PROC ROBUSTREG only offers limited statistical output from the model-fitting procedure, one of us (DM) wrote a SAS program (available on request) to calculate the 'standard error of prediction' of the model. The formula is equivalent to that of the 'STDI' parameter available from other regression procedures in SAS.²⁶

The standard error of prediction is a more logical choice for assigning the threshold of excess mortality than the root mean square error (standard error of the model residuals) because it incorporates not only the variance of the residuals but also the variance of the model parameter estimates.²⁷ This provides an estimate of the expected variability of the observed values in the absence of influenza epidemics.

To determine the best threshold of excess mortality due to influenza, we varied the constant factor by which the standard error of prediction was multiplied from 0.1 to 5 standard errors in increments of 0.1. We then calculated the true positive and false positive signalling rates (threshold exceedences) of each threshold. A true positive signal was a weekly exceedence occurring during an influenza season and a false positive was a weekly exceedence occurring outside the influenza season in each year. We defined influenza seasons to be periods where 4-week moving average counts of laboratory-notified influenza cases from the New South Wales notifiable diseases database²¹ were above 30. This was the lowest level that clearly discriminated seasonal from unseasonal activity in the study period. The true and false positive rates of surveillance signals are equivalent to the sensitivity (the true positive rate) and specificity (=1-false positive rate) of a laboratory test. Thus a graph describing the relationship between these 2 quantities at different thresholds ('receiver operating characteristics' (ROC) curve) can be plotted to assist in choosing a threshold that provides the surveillance system with the most useful balance between sensitivity and specificity. The complete time series available at the time of the study, 1 January 2002 to 9 November 2007, was used in assessing the true and false positive rates with the weekly threshold determined from the combined fitted model and 2007 forecast described above.

Ethics approval was not required for this study as it used data collected and used in accordance with New South Wales legislation for the purpose of health protection. Identifying variables and codes that could be used for re-identifying individuals were excluded from the study data.

Top of page

Results

Timeliness of the data

For the period when data were supplied weekly to the Department of Health, May to November 2007, the median interval between the date of death and the date of registration was 10 days, and the median interval between registration and receiving the information at the Health Department was an additional 5 days.

Descriptive analysis of mortality statistics

Between 1 January 2002 and 9 November 2007 there were 268,048 deaths registered in New South Wales, of which 33,220 (12%) were classified as P&I deaths. The age distribution of P&I deaths was older than that of all deaths combined, with 79% of P&I deaths in persons aged 75 years or more, compared with 64% in all deaths combined (Table 1). Influenza was mentioned only rarely in the cause of death text. Of the P&I deaths, 61 (0.2%) mentioned influenza and not pneumonia in the cause of death text, 59 (0.2%) mentioned both pneumonia and influenza, while the remaining 33,100 (99.6%) mentioned pneumonia only.

Table 1. Age distribution of all deaths and pneumonia and influenza deaths, 1 January 2002 to 1 November 2007

Age group (years)	All deaths		Pneumonia and influenza deaths
	n	%	n	%
0-1	2,475	0.9	31	0.1
2-4	296	0.1	16	0.1
5-14	555	0.2	45	0.1
15-24	2,281	0.9	70	0.2
25-34	3,604	1.34	86	0.3
35-44	6,236	2.3	241	0.7
45-54	12,802	4.8	588	1.8
55-64	23,985	9.0	1,614	4.9
65-74	43,872	16.4	4,143	12.5
75-84	86,062	32.1	11,350	34.2
85 or more	85,800	32.0	15,029	45.2
Total	268,048*	100.0	33,220*	100.0

* Age-specific deaths do not sum to the total, due to 80 deaths (including 7 pneumonia and influenza deaths) with missing age information.

Top of page

Applying the 4-week moving average criterion to influenza notifications from laboratories, influenza seasons occurred in the periods: 14 June 2002 to 6 September 2002, 25 July 2003 to 12 September 2003, 20 August 2004 to 22 October 2004, 1 July 2005 to 23 September 2005, 28 July 2006 to 8 September 2006, and 22 June 2007 to  4 September 2007. During these periods combined, a somewhat lower proportion of P&I deaths were in persons aged 45–54 years or 65–74 years, while a greater proportion were aged 85 years or more (Table 2).

Table 2. Contribution of each age group to pneumonia and influenza deaths during influenza and non-influenza seasons, 1 January 2002 to 1 November 2007

Age group (years)	Non-influenza season		Influenza season
	% (95% confidence interval) (n=23,774)		% (95% confidence interval) (n=9,446)
0-1	0.08	(0.05-0.12)	0.13	(0.07-0.22)
2-4	0.04	(0.02-0.08)	0.06	(0.02-0.14)
5-14	0.12	(0.08-0.17)	0.18	(0.10-0.29)
15-24	0.21	(0.16-0.28)	0.20	(0.12-0.31)
25-34	0.27	(0.21-0.35)	0.22	(0.14-0.34)
35-44	0.72	(0.61-0.83)	0.75	(0.59-0.95)
45-54	1.90	(1.73-2.08)	1.45	(1.22-1.71)*
55-64	4.97	(4.70-5.25)	4.58	(4.17-5.03)
65-74	12.90	(12.5-13.4)	11.30	(10.7-12.0)*
75-84	34.40	(33.8-35.0)	33.60	(32.7-34.6)
85 or more	44.40	(43.8-45.0)	47.40	(46.4-48.5)*

* The proportion differs significantly between the influenza and non-influenza seasons.

Validation of automated classification of pneumonia and influenza deaths

From the 3 random samples: (1) deaths categorised as flu or pneumonia; (2) those categorised as influenza; and (3) those classified as neither influenza or pneumonia, there were 1 (0.33%), 0 (0.00%), and 1 (0.33%) misclassifications, respectively. These were a result of spelling errors in the cause of death text resulting in the wrong category being assigned. This indicates that misclassification was negligible.

Top of page

Routine surveillance reporting, 2007

During most of the 2007 influenza surveillance reporting season, data were provided weekly by the New South Wales Registry of Births, Deaths, and Marriages and a graph similar to Figure 1 was included in the weekly surveillance report.¹⁹ Clear influenza season behaviour could be observed in 2003 and 2004. However, using a threshold of the upper 95% confidence interval of the predicted mean resulted in many weekly exceedances outside of the influenza seasons (false positives). This made the graphs difficult to interpret.

Figure 1. Example of simple mortality surveillance implemented during the 2007 influenza season, deaths due to pneumonia and influenza per 1,000 death

Improved method of modelling and threshold setting

Figure 2 shows the true and false positive rate of threshold exceedences in response to changing the threshold in multiples of the number of standard errors of prediction. Three thresholds are indicated, 0.9, 1.2 and 1.5 standard errors that provide a false positive rate of 0.16, 0.10 and 0.04 threshold exceedences per week, respectively. These equate to approximately 8, 5 or 2 weeks per year in which false alarm (non-influenza season) threshold exceedences occur, respectively. The corresponding true positive rates of 0.45, 0.23 and 0.16, respectively, are low and indicate many weeks during influenza seasons in which threshold exceedences do not occur.

Figure 2. Relationship between the epidemic threshold and the true and false positive rates, indicating the rate of threshold exceedences per week during influenza and non-influenza seasons, respectively

Figure 3 shows the result of the robust regression modelling, along with the 3 thresholds highlighted above. The excessive number of non-influenza weeks in which the threshold is exceeded when 0.9 standard errors is used, is evident. This threshold would offer limited discriminatory power between influenza and non-influenza seasons. A threshold of 1.5 standard deviations excludes all but a few peaks during influenza seasons. A threshold of 1.2 standard deviations appears to be the best compromise, and clearly highlights a high proportion of deaths occurring in at least 1 week of each of the 2002, 2003, 2004, 2005 and 2007 seasons.

Figure 3. Time series of raw weekly pneumonia and influenza mortality proportions, the fitted robust regression model, and 3 epidemic threshold curves. Weekly counts of influenza notifications from laboratories and the boundaries of the seasonal influenza periods

Top of page

Discussion

We found that prospective, largely automated monitoring of excess influenza mortality is possible in the Australian setting. Further, we demonstrated a sound statistical approach based on Serfling's methods for defining a surveillance threshold. A threshold of 1.2 standard errors of prediction provides an acceptable false positive rate that limits false alarms to 5 weeks per year while being sensitive enough to highlight some excess mortality in the years we studied. Epidemic sensitivity appears to be quite low, however, which could reflect either high levels of vaccine coverage in the population at risk, or relatively low virulence of circulating strains during the years studied. Another explanation for some relatively short-lived peaks in excess deaths could reflect different strains with varying virulence circulating at different times within seasons.^28,29

An important question is whether to analyse the time series by date of death or date of death registration. In prospective surveillance the earliest available date closest to the exposure that caused infection is most representative of when the disease is circulating. In our case, the date of death is the earliest available date. While analysing by date of registration gives a date closer to the date of surveillance reporting, there is a risk of shifting the perceived onset of the epidemic forward in time. We found the interval between date of death and date of registration to be a median 10 days, and from date of death to date of receipt at the Health Department to be 15 days. With more real-time registry data extraction we could reduce the interval from registration to reporting, but this would still be more than 1 week from the date of death. In these circumstances, excluding the last data point from the time series prior to analysis and reporting is sensible.

We found that the majority of P&I deaths occurred in those aged 85 years or over and the contribution of this age group to deaths increased slightly but statistically significantly during influenza seasons. This finding is consistent with that of Thomson, et al with 23 years of USA mortality data.8 Future work could evaluate whether applying these methods to age-specific data would improve detection of excess mortality due to influenza.

Clearly, mortality is the most extreme outcome of disease, and prospectively monitoring excess mortality due to influenza provides an objective perspective on the virulence of circulating strains or seasonal vaccine effectiveness. It could well prove valuable in assessing the impact of a pandemic.

Top of page

Limitations

Unlike in the USA,³⁰ New South Wales influenza mortality data does not show clear peaks indicating influenza epidemics. This may partly be due to the smaller population and thus greater statistical variability; in 2007, the New South Wales population was around 7 million while the USA population was over 300 million. Another explanation could be higher influenza vaccination coverage for persons aged over 65 years in New South Wales (75% in 2006)³¹ compared with the USA (64% in 2006).³² Given that the majority of deaths are in the elderly, this could have some influence.

The best threshold we estimated is based on a limited time period that included a mix of observed and forecast data. In practice, it would be best to determine the threshold by calculating the true and false positive rates in a real, prospective scenario over many years. Also, some of the exceedences that we designated as false alarms may have been caused by respiratory syncytial virus, which, while estimated to have one-third the mortality risk of influenza, still does contribute to seasonal excess mortality.⁸ Another factor that could be considered in future work is the delay between infection with influenza and death, which could be several weeks. While the moving average we used for the laboratory time series may have limited this problem, lagging the mortality time series relative to the laboratory time series could have improved the true alarm rate. In addition, we used the default settings in the PROC ROBUSTREG procedure. The procedure does allow the degree of weighting of outliers to be controlled, and provides alternative model estimation procedures. These options could be evaluated in future work.

The completeness and accuracy of the death registration information received by the NSW Department of Health needs to be evaluated. The laboratory notification data used to define influenza seasons also has some limitations. In recent years, increasing use of rapid influenza diagnostic tests may have inflated influenza notifications. Also, in late 2004 to early 2005 there was a known problem of false positive influenza notifications, evident in Figure 3.

Conclusions

Prospective and reasonably rapid monitoring of excess mortality due to influenza in an Australian setting is feasible and can provide valuable information on the impact of influenza on the population. Appropriate statistical methods for automatically identifying excess mortality are available and can be applied. The additional information can help health departments make a more objective assessment of the severity of seasonal and possibly pandemic influenza as well as the effectiveness of mitigation strategies.

Top of page

Acknowledgements

We are grateful to William Thompson and colleagues at the National Centre for Immunization and Respiratory Diseases, US Centers for Disease Control and Prevention, who provided helpful advice on the development of this study. We are also grateful to the New South Wales Registry of Births, Deaths and Marriages for consistently providing the death information, and to Tim Churches for establishing the death certificate data collection in HOIST. We also thank the peer reviewers for their instructive and helpful comments on the work.

Author details

Mr David J Muscatello, Senior Epidemiologist, Population Health Information Branch

Ms Patricia M Morton, New South Wales Public Health Training Program

Dr Ingrid Evans, New South Wales Public Health Training Program

Ms Robin Gilmour, Senior Surveillance Officer, Respiratory Diseases, Communicable Diseases Branch

NSW Department of Health, North Sydney, New South Wales

Corresponding author: Mr David Muscatello, Centre for Epidemiology and Research, NSW Department of Health, Locked Bag 961, NORTH SYDNEY NSW 2059. Telephone: +61 2 9391 9408. Facsimile: +61 2 9391 9232. Email: dmusc AT doh.health.nsw.gov.au

Top of page

References

1. Nicholson KG, Wood JM, Zambon M. Influenza. Lancet 2003;362:1733–1745.

2. Hampson AW, Mackenzie JS. The influenza viruses. Med J Aust 2006;185 Suppl:S39–S43.

3. Firestone SM, Barr IG, Roche PW, Walker JC. Annual report of the National Influenza Surveillance Scheme, 2005. Commun Dis Intell 2006;30:189–200.

4. Reichert T, Simonsen L, Sharma A, Pardo S, Fedson D, Miller M. Influenza and the winter increase in mortality in the United States, 1959–1999. Am J Epidemiol 2004;160:492–502.

5. Pitman RJ, Melegaro A, Gelb D, Siddiqui MR, Gay NJ, Edmunds WJ. Assessing the burden of influenza and other respiratory infections in England and Wales. J Infect 2007;54:530–538.

6. Poehling KA, Edwards KM, Weinberg GA, Szilagyi P, Staat MA, Iwane MK, et al. The underrecognized burden of influenza in young children. N Engl J Med 2006;355:31–40.

7. Newall AT, Wood JG, Macintyre CR. Influenza-related hospitalisation and death in Australians aged 50 years and older. Vaccine 2008;26:2135–2141.

8. Thompson WW, Shay DK, Weintraub E, Brammer L, Cox N, Anderson LJ, et al. Mortality associated with influenza and respiratory syncytial virus in the United States. JAMA 2003;289:179–186.

9. Thompson WW, Shay DK, Weintraub E, Brammer L, Bridges CB, Cox NJ, et al. Influenza-associated hospitalizations in the United States. JAMA 2004;292:1333–1340.

10. Grijalva CG, Weinberg GA, Bennett NM, Staat MA, Craig AS, Dupont WD, et al. Estimating the undetected burden of influenza hospitalisations in children. Epidemiol Infect 2006;13:951–958.

11. Noble GR. Epidemiological and clinical aspects of influenza. In: Beare AS ed. Basic and Applied Influenza Research. Boca Raton, Florida: CRC Press; 1982:11–50.

12. Collins SD. Excess mortality from causes other than influenza and pneumonia during influenza epidemics. Public Health Rep 1932;47:2159–2180.

13. Reichert T, Simonsen L, Sharma A, Pardo S, Fedson D, Miller M. Influenza and the winter increase in mortality in the United States, 1959–1999. Am J Epidemiol 2004;160:492–502.

14. Viboud C, Boelle P, Pakdaman K, Carrat F, Valleron A, Flahault A. Influenza epidemics in the United States, France, and Australia, 1972–1997. Emerg Infect Dis 2004;10:32–39.

15. Rizzo C, Bella A, Viboud C, Simonsen L, Miller M, Rota M, et al. Trends for influenza-related deaths during pandemic and epidemic seasons, Italy, 1969–2001. Emerg Infect Dis 2007;5:694–699.

Top of page

16. Centers for Disease Control and Prevention. 122 Cities Mortality Reporting System, Manual of Procedures. Atlanta (Georgia, United States): US Department of Health and Human Services; 2004.

17. Simonsen L, Clarke MJ, Stroup DF, Williamson GD, Arden NH, Cox NJ. A method for timely assessment of influenza-associated mortality in the United States. Epidemiology 1997;8:90–395.

18. O'Brien K, Barr IG. Annual Report of the National Influenza Surveillance Scheme, 2006. Commun Dis Intell 2007;31:167–179.

19. NSW Health, Communicable Diseases Branch. New South Wales Influenza Surveillance Report. NSW Health  Available from: http://www.health.nsw.gov.au/PublicHealth/Infectious/ Accessed on 11 September 2008.

20. New South Wales Government. New South Wales Public Health Act, 1991: Part 7, Division 1, Section 64. Available from: http://www.legislation.nsw.gov.au/viewtop/inforce/act+10+1991+pt.7-div.1-sec.64+0+N/

21. Centre for Epidemiology and Research. Health Outcomes and Information Statistical Toolkit (HOIST). NSW Department of Health [computer database].

22. SAS version 9.1.3. Cary (USA): SAS Institute, 2002–2003 [computer program].

23. Serfling RE. Methods for current statistical analysis of excess pneumonia-influenza deaths. Public Health Rep 1963;78:494–506.

24. Centers for Disease Control and Prevention. Update: Influenza activity – United States, 30 September – 1 December 2007. MMWR Morb Mortal Wkly Rep 2007;56:1287–1291.

25. The ROBUSTREG procedure. SAS/STAT(R) 9.2 User's Guide. Cary (USA): SAS Institute, 2008. Available from: http://support.sas.com/documentation/cdl/en/statug/59654/HTML/default/rreg_toc.htm

26. Model Fit and Diagnostic Statistics. SAS/STAT(R) 9.2 User's Guide. Cary (USA): SAS Institute, 2008. Available from: http://support.sas.com/documentation/cdl/en/statug/59654/HTML/default/statug_reg_sect039.htm

27. Hill C, Griffiths W, Judge G. Undergraduate Econometrics. New York: John Wiley and Sons; 1977: 174–175.

28. Wright PF, Thompson J, Karzon DT. Differing virulence of h2N1 and h4N2 influenza strains. Am J Epidem 1980;112:814–819.

29. Olson DR, Heffernan RT, Paladini M, Konty K, Weiss D, et al. Monitoring the impact of influenza by age: Emergency Department fever and respiratory complaint surveillance in New York City. PLoS Med 4(8):e247 doi:10.1371/journal.pmed.0040247 Available from: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1939858

Top of page

30. Centers for Disease Control and Prevention, National Center for Immunization and Respiratory Diseases. Flu View: A weekly influenza report prepared by the influenza division. Centers for Disease Control and Prevention [updated 18 January 2008]. Available from: http://www.cdc.gov/flu/weekly/ Accessed on 21 January 2008.

31. New South Wales Population Health Survey 2006. Centre for Epidemiology and Research, NSW Department of Health. Available from: http://www.health.nsw.gov.au/PublicHealth/Surveys/ Accessed on 11 September 2008.

32. Centers for Disease Control and Prevention. Influenza: Self-Reported Vaccination Coverage Trends 1989–2006  Available from: http://www.cdc.gov/flu/professionals/vaccination/pdf/vaccinetrend.pdf Accessed on 21 January, 2008.