Skip to main content
  • Brief Research Report
  • Open access
  • Published:

Emergency medical services key performance measurement in Asian cities

Abstract

Background

One of the key principles in the recommended standards is that emergency medical service (EMS) providers should continuously monitor the quality and safety of their services. This requires service providers to implement performance monitoring using appropriate and relevant measures including key performance indicators. In Asia, EMS systems are at different developmental phases and maturity. This will create difficultly in benchmarking or assessing the quality of EMS performance across the region. An attempt was made to compare the EMS performance index based on the structure, process, and outcome analysis.

Findings

The data was collected from the Pan-Asian Resuscitation Outcome Study (PAROS) data among few Asian cities, namely, Tokyo, Osaka, Singapore, Bangkok, Kuala Lumpur, Taipei, and Seoul. The parameters of inclusions were broadly divided into structure, process, and outcome measurements. The data was collected by the site investigators from each city and keyed into the electronic web-based data form which is secured strictly by username and passwords. Generally, there seems to be a more uniformity for EMS performance parameters among the more developed EMS systems. The major problem with the EMS agencies in the cities of developing countries like Bangkok and Kuala Lumpur is inadequate or unavailable data pertaining to EMS performance.

Conclusions

There is non-uniformity in the EMS performance measurement across the Asian cities. This creates difficulty for EMS performance index comparison and benchmarking. Hopefully, in the future, collaborative efforts such as the PAROS networking group will further enhance the standardization in EMS performance reporting across the region.

Findings

Background

In today’s health care environment, the demand for objective comparative information about the performance of health care organizations and providers has created a need for data-driven evaluation processes [1]. As a part of the health care environment, pre-hospital emergency medical services (EMS) systems are no different in their need for objective comparative system information to assist government officials at all levels to establish relevant policy, select appropriate system design, and monitor system quality and effectiveness. Assessing EMS performance may be a simple task if it is carried out within a local health care system, but things can get complicated if the comparison is made between states or countries [2,3]. Different states or countries may deliver EMS differently. In Asia, EMS systems are at different developmental phases and maturity. For example, Singapore, Japan, and Korea have mature and systematic EMS whereas in other developing countries, the system is still at its infancy [4,5]. This will create difficultly benchmarking or assessing the quality of EMS performance across the region. Despite limited data, we attempted to compare basic EMS performance based on reports by the Pan-Asian Resuscitation Outcome Study (PAROS) group [6]. Hopefully, this will be the first step in understanding the quality of EMS in Asia and the stepping stone for benchmarking in the future.

EMS performance measurement

Measuring quality in EMS systems is challenging [7]. Measuring quality in EMS is important since EMS is the practice of medicine in the pre-hospital setting. The goal of EMS Performance Measurement (EMSPM) is to apply knowledge, data, and experience to evaluate and improve EMS service delivery, personnel performance, and clinical care. The need for increased coordination in patient care and higher quality care at lower costs has made it essential for EMS agencies to have in-place quality control or quality improvement programs that rely on key performance indicators to continuously monitor the system’s overall performance and the effectiveness of the different pre-hospital interventions [8,9]. The Institute of Medicine (IOM), in a report entitled “Emergency Medical Services at the Crossroads” and published in 2006, recommended the development of “evidence based performance indicators that can be nationally standardized so that statewide and national comparisons can be made” [10]. The development and implementation of these indicators would enhance accountability in EMS and provide EMS agencies with data to measure their system’s overall performance and to develop sound strategic quality improvement planning.

Performance indicators

Performance indicators are measurement tools that should be “specific, measurable, action oriented, relevant and timely” [11]. An indicator is a metric that reflects on the performance of a system or process. As the indicator value rises or falls, it suggests that the system or process is operating better or worse - like a performance thermometer. The common measurable indicators include structure, process, and outcome. (Table 1) However, the National Highway and Traffic Safety Act (NHTSA) has recommended a more comprehensive measurement which includes system design and structure, human resources (culture, training, safety, credentialing, etc.), clinical care and outcome, response, finance, quality management, and community demographics [12]. Problems may arise from a complex measurement system especially when the system is used for multiple agencies or international usage. Different agencies or countries may have different definitions for a specific measure such as response time, or the data may not be easily available within that particular community (such as pre-hospital defibrillation) [13,14]. This will create standardization of EMSPM across the region really difficult. Finding the most common available denominators is perhaps the best method to carry out the inter-agency EMSPM [15].

Table 1 Performance indicators

The structure of EMS agencies differs significantly between the Asian countries. The focus of emergency health care in most of the Asian countries relies heavily on the hospital system, with little emphasis on pre-hospital care other than rapid transport. The quality of care available at all levels appears to be directly related to the economic status of that country and the efforts of government and health authorities to identify and support the development of pre-hospital services. Countries like Singapore, South Korea, Japan, and Taiwan have a much more mature EMS setup compared to other countries in the Asian region [16,17]. They have robust data system collection partly for the purpose of quality improvement. Unfortunately, there are no standardized quality assurance and monitoring system within this region. Overall, comparing the EMSPM is difficult partly due to lack of data and non-standardization of the delivery system. However, we attempted to collect basic common data within the PAROS study group in 2013.

Methods

The data collected comprises of structure, process, and outcome information of the EMS system within each of the EMS agencies. A survey form was created and sent via email to all the representatives of the EMS agencies or the medical directors of the PAROS group. Some of the survey forms were handed to the participating members by hand during the annual PAROS meetings held in various cities within Asia. Some of the data was obtained from the previous publications within the PAROS group that is adopted from cardiac arrest registry to enhance survival (CARES) study based in the USA [18,19]. Twenty-four variables were assessed and compared, all of which were derived by consensus across the PAROS countries. The components were agreed upon after series of meeting and followed the Utstein template. The most common available denominators of measurement were taken into analysis and comparison. The other variables that were not available in all the participated countries would be excluded from analysis. The overall comparison of EMSPM is carried out based on the countries and cities where the EMS agencies originate from.

Results

The final response reported in this study is from seven cities (Seoul, Tokyo, Osaka, Taipei, Singapore, Bangkok, and Kuala Lumpur) and six countries (Singapore, South Korea, Taiwan, Japan, Thailand, and Malaysia). General demography of each of the cities is as shown in Table 2. In all, we managed to gather measurement indicators for 13 structural components, nine process components, and two outcome indicators. Korea, Singapore, Taiwan, and Japan provided the data from their existing EMS registries whereas countries like Malaysia and Thailand based their data mainly from hospitals within the big cities like Kuala Lumpur and Bangkok. Some of the data is national and others are specific only for the cities (Tables 3, 4, and 5).

Table 2 General demography of the cities in the survey
Table 3 Structure comparisons of EMS systems in Asia
Table 4 Process comparison of EMS systems in Asia
Table 5 Outcome comparison of EMS systems in Asia

Discussion

Generally, there seems to be a more uniformity for EMS performance parameters among the more developed EMS systems. The major problem with the EMS agencies in the cities of developing countries like Bangkok and Kuala Lumpur is inadequate or unavailable of data pertaining to EMS performance. This is partly due to lack of uniform and standardized data collection. EMS agencies in Seoul, Tokyo, Osaka, Taipei, and Singapore utilize established electronic patient care record systems for all EMS cases [20]. This allows a more consistent and sustainable data collection for multiple purposes, including for performance analysis. Unfortunately, the EMS agencies in Bangkok and Kuala Lumpur do not use similar electronic patient care record systems. Most patient care records are manual, and data collection is not standardized and non-uniform. Interestingly, the obvious difference in the structure of EMS between the developed and developing countries in Asia lies with the EMS providers and their location. The cities in the more developed countries utilize the fire department and their personnel as the providers whereas the cities in the developing countries prefer hospital-based EMS provision. Similarly, the training hours for the EMS providers for each city differ significantly, partly due to different training legislation and requirements. The ratio of the number of ambulance to the number of providers varies from 2.16 in Bangkok to 0.05 in Taipei. Such data is unavailable for Kuala Lumpur. Mean response time for out-of-hospital cardiac arrest (OHCA) is within 10 min for all the cities in the developed countries. The response time is much longer in Kuala Lumpur and unavailable for Bangkok. Singapore reported the highest pre-hospital defibrillation rate compared to other cities but unfortunately, the survival to discharge is fairly low. No such data is available for Bangkok and Kuala Lumpur.

There are many challenges in reporting the core measures of EMS performance. Variability in data storage and core measure definitions is very common. Not all countries use a systematic database registry system, especially in the developing countries. Not all data systems utilized similar data dictionaries. This resulted in disparity in the way measurements were derived. Some EMS agencies followed the measure exactly, while others had interpreted the measure more loosely. Many local EMS systems use paper pre-hospital patient care records while others use the electronic format. Abstracting information from paper forms is difficult, time-consuming, and not necessarily accurate [21]. In contrast, some software systems have a high degree of technological sophistication that forces users to complete forms before closing the record [22]. Moreover, without training in the specific core measures, users may not have understood the criticality of completing each data point. One of the clear challenges identified was the difficulty and inability by EMS agencies to obtain universal hospital outcome data on every ambulance transport. This was evidenced by the low response rate for specific cardiac arrest outcome measures. These measures relied upon the hospital to report survival to emergency department discharge and survival to hospital discharge. Another significant limitation of reporting EMS information is related to the nature of the “tiered” EMS system that is present in cities like Kuala Lumpur. Because there are EMS first responders and separate ambulance transport units that arrive at a later time, often, two records are initiated for each patient. This inability to aggregate first responder data with transport provider data could lead to a conclusion that care was not provided, when in fact, it may have been provided to the patient by a different provider.

Recommendation

A few efforts should be considered in order to reduce the limitations in obtaining universal and uniform EMS performance measures in the Asian countries. The establishment of the PAROS in the Asian region in 2010 has significantly changed the landscape of networking among EMS experts in the region. The PAROS is a collaborative research group formed by dedicated pre-hospital emergency care (PEC) providers conducting PEC research in the Asia-Pacific region. The initial step involved the data collection pertaining to the OHCA within the Asian region. Before the study was conducted, basic data on the EMS components were obtained from the involved countries and cities, namely, the general structure, process, and outcome of the EMS system. The regional networking has created standardized definitions across the PAROS network by adopting a consensual common taxonomy and data collection methodology. This will allow consideration in an attempt to create uniform EMS measures in the region and to identify common available core measures from each country or cities. The uniform core measures will create a more meaningful EMS practice comparison and benchmarking in the region. This is a very important step to a very long journey in creating a sustainable unique, low-cost, self-funded model of a collaborative research and education network in EMS. No doubt, there will be a lot more hurdles to be faced but with perseverance and commitment this effort will succeed.

Conclusion

EMS is an integral part of every community’s total health care delivery system. Consistent evaluation of clinical and response performance indicators are crucial components in ensuring that first response services are operating at peak efficiency. To achieve this, continuous quality improvement (QI) practices must become an essential and seamless part of normal EMS routines. Unfortunately at the present time, there is an obvious significant difference in the EMS practice of reporting in the Asian region. This creates difficulty for EMS performance index comparison and benchmarking. Data systems that are robust and agile, with the ability to report clinical indicators and performance measures, are a key tool in quality improvement activities. Hopefully, in the future, collaborative efforts such as the PAROS networking group will further enhance the standardization in EMS performance reporting across the region.

Abbreviations

EMS:

emergency medical services

PAROS:

Pan-Asian Resuscitation Outcome Study

EMSPM:

Emergency Medical Services Performance Measurements

IOM:

Institute of Medicine

NHTSA:

National Highway and Traffic Safety Act

OHCA:

out-of-hospital cardiac arrest

PEC:

pre-hospital emergency care

References

  1. Silber JH, Rosenbaum PR, Ross RN, Ludwig JM, Wang W, Niknam BA, et al. A hospital-specific template for benchmarking its cost and quality. Health Serv Res. 2014;49(5):1475–97.

    Article  PubMed  Google Scholar 

  2. Ma MH, Chiang WC, Ko PC, Huang JC, Lin CH, Wang HC, et al. Outcomes from out-of-hospital cardiac arrest in Metropolitan Taipei: does an advanced life support service make a difference? Resuscitation. 2007;74(3):461–9. Epub 2007 Apr 25.

    Article  PubMed  Google Scholar 

  3. Vaillancourt C, Stiell IG. Canadian Cardiovascular Outcomes Research Team. Cardiac arrest care and emergency medical services in Canada. Can J Cardiol. 2004;20(11):1081–90.

    PubMed  Google Scholar 

  4. Ong ME, Cho J, Ma MH, Tanaka H, Nishiuchi T, Al Sakaf O, et al. Comparison of emergency medical services systems in the pan-Asian resuscitation outcomes study countries: report from a literature review and survey. Emerg Med Australas. 2013;25(1):55–63.

    Article  PubMed  Google Scholar 

  5. Nielsen K, Mock C, Joshipura M, Rubiano AM, Zakariah A, Rivara F. Assessment of the status of prehospital care in 13 low- and middle-income countries. Prehosp Emerg Care. 2012;16(3):381–9.

    Article  PubMed Central  PubMed  Google Scholar 

  6. Shin SD, Ong ME, Tanaka H, Ma MH, Nishiuchi T, Alsakaf O, et al. Comparison of emergency medical services systems across pan-Asian countries: a Web-based survey. Prehosp Emerg Care. 2012;16(4):477–96.

    Article  PubMed  Google Scholar 

  7. Harrawood D, Easton K. EMS measuring stick. How to develop an EMS clinical performance scorecard. JEMS. 2000;25(7):38–45.

    CAS  PubMed  Google Scholar 

  8. MacFarlane C, Benn CA. Evaluation of emergency medical services systems: a classification to assist in determination of indicators. Emerg Med J. 2003;20(2):188–91.

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  9. Roudsari BS, Nathens AB, Cameron P, Civil I, Gruen RL, Koepsell TD, et al. International comparison of prehospital trauma care systems. Injury. 2007;38(9):993–1000. Epub 2007 Jul 20.

    Article  PubMed  Google Scholar 

  10. Berger E. Crossroads: report calls for EMS standards of care. Ann Emerg Med. 2006;48(2):142–3.

    Article  PubMed  Google Scholar 

  11. Pittet V, Burnand B, Yersin B, Carron PN. Trends of pre-hospital emergency medical services activity over 10 years: a population-based registry analysis. BMC Health Serv Res. 2014;14(1):380. doi:10.1186/1472-6963-14-380.

    Article  PubMed Central  PubMed  Google Scholar 

  12. Mears GD, Pratt D, Glickman SW, Brice JH, Glickman LT, Cabañas JG, et al. The North Carolina EMS Data System: a comprehensive integrated emergency medical services quality improvement program. Prehosp Emerg Care. 2010;14(1):85–94.

    Article  PubMed  Google Scholar 

  13. Cady G. 200 city survey. JEMS 2001 annual report on EMS operational and clinical trends in large, urban areas. JEMS. 2002;27(2):46–65. 68–70.

    PubMed  Google Scholar 

  14. Sayre MR, White LJ, Brown LH, McHenry SD. National EMS Agenda Writing Team. National EMS Research Agenda. Prehosp Emerg Care. 2002;6(3 Suppl):S1–43.

    CAS  PubMed  Google Scholar 

  15. Maio RF, Garrison HG, Spaite DW, Desmond JS, Gregor MA, Stiell IG, et al. Emergency Medical Services Outcomes Project (EMSOP) IV: pain measurement in out-of-hospital outcomes research. Ann Emerg Med. 2002;40(2):172–9.

    Article  PubMed  Google Scholar 

  16. Joshipura M, Hyder AA, Rehmani R. Emergency care in South Asia: challenges and opportunities. J Coll Physicians Surg Pak. 2004;14(12):731–5.

    PubMed  Google Scholar 

  17. Razzak JA, Kellermann AL. Emergency medical care in developing countries: is it worthwhile? Bull World Health Organ. 2002;80(11):900–5.

    PubMed Central  PubMed  Google Scholar 

  18. Ong ME, Shin SD, Tanaka H, Ma MH, Nishiuchi T, Lee EJ, et al. Rationale, methodology, and implementation of a dispatcher-assisted cardiopulmonary resuscitation trial in the Asia-Pacific (Pan-Asian Resuscitation Outcomes Study Phase 2). Prehosp Emerg Care. 2014;19(1):87–95.

    Article  PubMed  Google Scholar 

  19. Chan PS, McNally B, Tang F, Kellermann A. CARES Surveillance Group. Recent trends in survival from out-of-hospital cardiac arrest in the United States. Circulation. 2014;130(21):1876–82.

    Article  PubMed  Google Scholar 

  20. Ro YS, Shin SD, Song KJ, Park CB, Lee EJ, Ahn KO, et al. A comparison of outcomes of out-of-hospital cardiac arrest with non-cardiac etiology between emergency departments with low- and high-resuscitation case volume. Resuscitation. 2012;83(7):855–61.

    Article  PubMed  Google Scholar 

  21. Jönsson K, Fridlund B. A comparison of adherence to correctly documented triage level of critically ill patients between emergency department and the ambulance service nurses. Int Emerg Nurs. 2013;21(3):204–9.

    Article  PubMed  Google Scholar 

  22. Abrams HC, McNally B, Ong M, Moyer PH, Dyer KS. A composite model of survival from out-of-hospital cardiac arrest using the Cardiac Arrest Registry to Enhance Survival (CARES). Resuscitation. 2013;84(8):1093–8.

    Article  PubMed  Google Scholar 

Download references

Acknowledgement

The authors would like to thank to all the investigators within the Pan-Asian Resuscitation Outcome Study (PAROS) and also convey the greatest appreciation to the Asian EMS Council who had given encouragement and morale support to the preparation of this manuscript.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Nik Hisamuddin Rahman.

Additional information

Competing interests

The authors declare that they have no competing interests.

Authors’ contributions

NHR contributed in the conception of the study, data collection, and manuscript preparation. HT and MEHO contributed in the data collection and review of manuscript. SDS, YYN, TP, and CHL contributed in the data collection. All authors read and approved the final manuscript.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by/4.0), which permits use, duplication, adaptation, distribution, and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rahman, N.H., Tanaka, H., Shin, S.D. et al. Emergency medical services key performance measurement in Asian cities. Int J Emerg Med 8, 12 (2015). https://doi.org/10.1186/s12245-015-0062-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12245-015-0062-7

Keywords