Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Process evaluation of health system costing – Experience from CHSI study in India

  • Shankar Prinja ,

    Contributed equally to this work with: Shankar Prinja, Sehr Brar, Maninder Pal Singh, Pankaj Bahuguna, Lorna Guinness

    Roles Conceptualization, Funding acquisition, Methodology, Project administration, Supervision, Writing – original draft, Writing – review & editing

    shankarprinja@gmail.com

    Affiliation Department of Community Medicine and School of Public Health, Post Graduate Institute of Medical Education and Research, Chandigarh, India

  • Sehr Brar ,

    Contributed equally to this work with: Shankar Prinja, Sehr Brar, Maninder Pal Singh, Pankaj Bahuguna, Lorna Guinness

    Roles Data curation, Formal analysis, Methodology, Validation, Visualization, Writing – original draft, Writing – review & editing

    Affiliation Department of Community Medicine and School of Public Health, Post Graduate Institute of Medical Education and Research, Chandigarh, India

  • Maninder Pal Singh ,

    Contributed equally to this work with: Shankar Prinja, Sehr Brar, Maninder Pal Singh, Pankaj Bahuguna, Lorna Guinness

    Roles Methodology, Project administration, Validation, Visualization, Writing – review & editing

    Affiliation Department of Community Medicine and School of Public Health, Post Graduate Institute of Medical Education and Research, Chandigarh, India

  • Kavitha Rajsekhar ,

    Roles Investigation, Project administration, Writing – review & editing

    ‡ These authors also contributed equally to this work.

    Affiliation Department of Health Research, Ministry of Health and Family Welfare, Government of India, New Delhi, India

  • Oshima Sachin ,

    Roles Investigation, Project administration, Writing – review & editing

    ‡ These authors also contributed equally to this work.

    Affiliation Department of Health Research, Ministry of Health and Family Welfare, Government of India, New Delhi, India

  • Jyotsna Naik ,

    Roles Investigation, Writing – review & editing

    ‡ These authors also contributed equally to this work.

    Affiliation Department of Health Research, Ministry of Health and Family Welfare, Government of India, New Delhi, India

  • Malkeet Singh ,

    Roles Data curation, Writing – review & editing

    ‡ These authors also contributed equally to this work.

    Affiliation Department of Health Research, Ministry of Health and Family Welfare, Government of India, New Delhi, India

  • Himanshi Tomar ,

    Roles Investigation, Writing – review & editing

    ‡ These authors also contributed equally to this work.

    Affiliation Department of Health Research, Ministry of Health and Family Welfare, Government of India, New Delhi, India

  • CHSI Study Collaborating Investigators ,

    ‡ These authors also contributed equally to this work.

    Members of the CHSI Study Collaborating Investigators is listed in the Acknowledgments.

  • Pankaj Bahuguna ,

    Contributed equally to this work with: Shankar Prinja, Sehr Brar, Maninder Pal Singh, Pankaj Bahuguna, Lorna Guinness

    Roles Formal analysis, Validation, Visualization, Writing – review & editing

    Affiliation Department of Community Medicine and School of Public Health, Post Graduate Institute of Medical Education and Research, Chandigarh, India

  • Lorna Guinness

    Contributed equally to this work with: Shankar Prinja, Sehr Brar, Maninder Pal Singh, Pankaj Bahuguna, Lorna Guinness

    Roles Conceptualization, Methodology, Validation, Writing – review & editing

    Affiliation Independent Researcher, Imperial College London, London, England

Abstract

Background

A national study, ‘Costing of healthcare services in India’ (CHSI) aimed at generating reliable healthcare cost estimates for health technology assessment and price-setting is being undertaken in India. CHSI sampled 52 public and 40 private hospitals in 13 states and used a mixed micro-costing approach. This paper aims to outline the process, challenges and critical lessons of cost data collection to feed methodological and quality improvement of data collection.

Methods

An exploratory survey with 3 components–an online semi-structured questionnaire, group discussion and review of monitoring data, was conducted amongst CHSI data collection teams. There were qualitative and quantitative components. Difficulty in obtaining individual data was rated on a Likert scale.

Results

Mean time taken to complete cost data collection in one department/speciality was 7.86(±0.51) months, majority of which was spent on data entry and data issues resolution. Data collection was most difficult for determination of equipment usage (mean difficulty score 6.59±0.52), consumables prices (6.09±0.58), equipment price(6.05±0.72), and furniture price(5.64±0.68). Human resources, drugs & consumables contributed to 78% of total cost and 31% of data collection time. However, furniture, overheads and equipment consumed 51% of time contributing only 9% of total cost. Seeking multiple permissions, absence of electronic records, multiple sources of data were key challenges causing delays.

Conclusions

Micro-costing is time and resource intensive. Addressing key issues prior to data collection would ease the process of data collection, improve quality of estimates and aid priority setting. Electronic health records and availability of national cost data base would facilitate conducting costing studies.

Introduction

India’s three-tier healthcare system serves a population of over 1.3 billion across 28 states and 9 union territories, aimed at providing affordable and accessible primary healthcare. The three levels are primary (sub-centre and primary health centre-PHC), secondary (Community health centre-CHC) and tertiary (district and speciality hospitals). Since its inception, the Indian healthcare system has strived towards universal health coverage and this is being realized with the Ayushman Bharat scheme. It has two components, Health and Wellness Centres (HWCs) for strengthening of primary healthcare services and Ayushman Bharat—Pradhan Mantri Jan Aarogya Yojana (AB-PMJAY), a tax funded health insurance scheme. AB-PMJAY aims to provide a coverage of INR 500000 to over 100 million vulnerable families for secondary and tertiary care services at public and private facilities. [1] Owing to the lack of scientific evidence, the reimbursement rates of AB-PMJAY packages had been defined by a consultative process.

The Government of India has also established a health technology assessment agency called Health Technology Assessment India (HTAIn) to strengthen evidence-based policy making. [2] HTA requires cost data inputs including health system costs and out of pocket expenditure (OOP). Health system cost entails the cost borne by the provider or health system for service delivery while OOP is the financial cost borne by patients in accessing healthcare services. Significant data exists for OOP through various household surveys, such as various rounds of household survey by National Sample Survey Organization (NSSO). However, there is limited information on health system cost since the systems for compiling and collecting cost data are relatively under-developed. [3,4]

Addressing the information gap, studies are now being undertaken more frequently to estimate health system costs in Indian settings. Of the existing cost studies, some are procedure/surgery specific [511], disease/treatments specific [1220] and some estimate public healthcare facility costs [2127]. The existing costing studies largely focus on a single health facility or condition; do not cover all health care services and do not cover all types of health care providers, most importantly the private sector. Hence, they fail to capture the likely heterogeneity in costs across different types of providers and settings. Further, these studies have been conducted using different methodologies and perspectives, in different regions and at different levels of healthcare. As such they lack generalizability and cannot be used in national policy making. To support HTAIn’s efforts in economic evaluation and to aid price-setting of the AB-PMJAY packages, a national level study titled ‘Costing of health services in India’(CHSI) was initiated by Department of Health Research (DHR), Ministry of Health & Family Welfare (MoHFW), New Delhi, in joint collaboration with Post Graduate Institute of Medical Education and Research (PGIMER) Chandigarh and respective Multi-Disciplinary Research Units (MRU) in the 13 states. The aim of the CHSI study is to generate reliable standardised cost estimates for both policy decisions and health system research at a national level and for these cost data to be used in cost-effectiveness analyses for HTA, inform price negotiations for AB-PMJAY as well as budgeting.

Economic costing of health services involves a number of sequential steps. Starting with identifying the input resources, estimating resource use, valuing resources in monetary terms and determining the service outputs. [28] The number of steps and complexities of cost data collection make it a daunting task where there is no routine cost data. Hence it is essential to determine and outline the best way to undertake this complex task, especially in multi-layered health systems. Challenges in data collection have been reported by previous costing studies both in India and elsewhere. [28,29]. These include wide variations in health care delivery infrastructure in India, non-availability of disaggregated data and hospital management information systems (HMIS). Other challenges include obtaining stock-related data, price information and in particular the continued record keeping in physical forms.

Documenting these issues will be critical for shaping cost data collection and improving data quality in the future. Despite the common knowledge around these issues, a comprehensive analysis of challenges faced during health system cost data collection across different states has not be undertaken in Indian settings. This paper reports the findings of the process evaluation of the cost data collection in the CHSI study. It outlines the process followed and challenges faced during data collection for generating national level economic costs, and identifies critical lessons which can feed into subsequent methodological improvement, as well as improve quality of data collection in future studies.

Methodology

Costing of Health Services in India (CHSI) study

The aim of the CHSI study was to generate cost estimates to inform price setting of AB-PMJAY packages and HTA. The study covers 13 Indian states, collecting data from 1 tertiary public hospital, 3 public district hospitals and 3–4 private hospitals in each state. A multi-stage sampling was done to represent heterogeneity based on geography, economic status, health indicators and health service utilization. The cost centres included in the study were outpatient department (OPD), inpatient department (IPD), operating theatre (OT), intensive care unit (ICU) and ancillary services. The study was conducted using an economic perspective and a mixed methodology (top-down and bottom-up). Cost data collection was led by state level team based at each MRU. A central team trained and supported the state teams, collated the state level data and provided quality assurance. Overall analysis was carried out by the central team. Ethical approval for the CHSI study was obtained from the Institutional Ethics Committee of Post Graduate Institute of Medical Education and Research Chandigarh.

Data collection

The present process evaluation was exploratory in nature and used multiple methods. (S1 Data) Three approaches were used for data collection, namely an online survey using a semi-structured questionnaire, group discussion and the review of monitoring data. Data was collected with the aim of understanding the process of cost data collection including: time taken, resources used, challenges and solutions. It was important to take into account that in a costing activity, different kinds of information are required to be collected under each type of input resource. For example, for human resources cost estimation requires information on salary, leave, incentives and time allocation. Similarly, information on quantity, price and utility is required for consumables. Collecting each kind of cost data poses different challenges requiring varied amounts of time and resources. Therefore, in addition to assessing the overall process of data collection, the study aimed to explore differences between input resources. This study was first of its kind, hence tools were developed for data collection and validated in a national consultation with DHR and MRUs. The tools were piloted in 2 sites before initiating data collection in all 11 sites.

Online survey.

A semi-structured tool was designed to elicit information on the data collection process and challenges faced in undertaking cost data collection. The tool had both quantitative and qualitative parameters and comprised of five sections. The first section focused on the basic details of the MRU including location, service capacity, allocated departments and time spent on data collection. The second section focused on inputs or pre-requisites for data collection and associated challenges like institutional permissions, staff recruitment and training. This section contained objective questions to understand the process followed for permissions and subjective questions to understand critical challenging areas. The third section focused on aspects of data collection specific to each input resource or output data. Respondents were required to provide detailed information on the process of data collection for each input resource such as building, human resource, equipment etc. There were 3 parts of the third section. The first part contained multiple choice questions on cost data collection process adopted. The second part comprised of a detailed table to be filled for each resource. (Table 1) The data collection team were required to rate the level of difficulty in cost data collection on a Likert scale of 1 to 10, where 1 implied no difficulty in data collection and 10 implied it was impossible to collect data. The third part captured additional data required for costing of OT procedures.

thumbnail
Table 1. Details on process of cost data collection (section 3 of questionnaire).

https://doi.org/10.1371/journal.pone.0232873.t001

The final two sections of the tool elicited field team experience regarding the training and supervisory support under the CHSI study and suggestions for improvement of data collection. The survey participants included MRUs of 11 states (Andhra Pradesh, Bihar, Delhi, Gujarat, Jammu & Kashmir, Maharashtra, Odisha, Rajasthan, Tamil Nadu, Uttar Pradesh and West Bengal) involved in the CHSI study.

Group discussion.

An open-ended group discussion was conducted after the online survey in order to illicit detailed information on key areas identified. The participants included key personnel involved in the CHSI study (state co-investigators, field staff, central supervisory and data analysis team). The key themes were multiplicity of permissions required; lack of electronic records; identification of the right data source; adapting to changing data requirements and evolvement of data collection tool.

Review of central monitoring data and key stakeholder interviews.

The progress of data collection at field sites was closely monitored by the central (PGIMER) team in terms of time taken for data collection, number of communications required for clarifications, data pendency etc. This data was used in order to infer the time spent on data collection at each site. Secondly, in-depth interviews of central team members were conducted to capture their experiences with a special focus on quality assurance. Findings are reported in conjunction with the findings of the online survey, since core issues that came out of the survey were further elicited from these sources.

Data analysis

Quantitative data.

The quantitative data from the survey was collated and analysed using MS Excel 2017, for descriptive statistics. Summary statistics for questions with discrete variable responses (categorical responses such as yes and no) were summarized in the form of percentages and represented as bar graphs. Summary statistics for questions with continuous variable responses with single response were summarized as mean, median, range and standard error. These are represented as box and whisker plots and in summary tables. Summary statistics for questions with continuous variable responses with multiple responses (such as proportions of sources of price data) were summarized as median and were represented as stacked column graphs. Time was analysed in two ways, firstly the overall time lapse in cost data collection and secondly person hours actually spent in data collection activities. Further, the time spent on data collection of each input resource was compared with its share in the total cost.

Qualitative data.

Qualitative data was obtained from three sources, namely the group discussion, interviews and the online survey. The qualitative data obtained from the group discussion was categorized thematically to delineate key challenges and best practices in data collection. The responses obtained from the group discussion were clubbed under the outlined themes and reported accordingly. The interviews of central team members were focussed on eliciting information on quality assurance and were analysed under one theme. The subjective responses to the online tool were analysed by identifying and comparing the most common themes, across the different sites.

Results

The results are presented according to the key themes identified.

Staff profile and training

There were 65 staff working on the costing study at the MRU level across the sites. On an average there were 5 staff members per MRU (range: 3 to 8), including 3 field officers, 1 administrative assistant guided by 1 co-principal investigator at each MRU. In addition, at some sites regular hospital staff (residents & professors) aided the data collection. Amongst the CHSI staff, 49.23% were centrally trained comprising principal investigators, professors and residents(53.13%) and field data collection staff (46.87%). The remaining staff (50.77%), primarily field investigators and administrative assistants, was trained at the MRU level by centrally trained staff. Amongst the staff, 73.3% were postgraduates and the rest were graduates.

Time horizon of cost data collection

The data collection process was considered under 3 activities; obtaining permissions, actual data collection and entry, queries & final submission which took 0.5 months(Interquartile Range(IQR) 0.28–1.5), 1.56 months (IQR 1.08–3.88) and 5 months (IQR 3.44–5.50) respectively. (Fig 1) On an average, data collection in one department of one facility covering all its services required 355 person-days.

The duration of data collection was disaggregated by the input resource level. (Table 2) Across all input resources, the time spent on negotiation and waiting was more than the actual time spent on data collection. It was found to be the longest for building and consumables, and shortest for overheads.

thumbnail
Table 2. Time required for data collection (days) per input resource.

https://doi.org/10.1371/journal.pone.0232873.t002

Fig 2 compares the proportion of time taken to collect data on each input resource with its share in the total cost. It was observed that while human resources and drugs & consumables contributed to 78% of the total cost, consuming only 31% of data collection time. However data collection of Furniture, Overheads and Equipment consumed 51% of the total time contributing to only 9% of the total cost.

thumbnail
Fig 2. Share of input resources in data collection time and cost.

https://doi.org/10.1371/journal.pone.0232873.g002

Permission for cost data collection

In most MRUs (9/11) the prime person to be contacted for data collection permission was the Dean/Administrator of the institution. Additionally, 4 MRUs also reported the need for obtaining permission from the authorities of concerned departments, generally the head of the departments (HODs). Two MRUs required permissions from Dean of institution, HODs and cost centre (ICU/OT/OPD/IPD) in-charge. In about one-fourth MRUs, additional Institutional Ethics Committee approval was also required. Across all sites, multiple one-on-one meetings were held with the facility head and cost centre in-charge to obtain approval.

Processes in cost data collection

The process of cost data collection has been summarises highlighting the key issues faced by the different sites in collecting data on input resources (Table 3)

thumbnail
Table 3. Overview of cost data collection for different input resources.

https://doi.org/10.1371/journal.pone.0232873.t003

Some of the key results are highlighted here:

Human resources.

Collecting data on human resource time allocation involved personnel interviews. This required willingness and time of the personnel which was challenging. Secondly people generally tend to over report the time devoted to a particular activity in routine. This was evident when the sum total of their activities added to more than their total working hours. This was found in 4 sites and corrections had to be made by re-interviewing the person concerned.

Building area & valuation.

Determination of building rental price was found to be challenging with wide variation in the estimates from different sources. Different sites followed different approaches to determine the building rental price. The most common sources were market survey and housing rentals in the same area. The final estimate was derived from multiple sources (in 9 of 11 sites), with a preference to commercial rates. All sites relied on physical measurement and blue prints of the building or other records that detailed the area measurement.

Equipment & furniture.

Determining the average useful life of equipment & furniture items was reported to be difficult due to the absence of records. All sites relied on expert opinion for this estimate. Secondly, cross checking of equipment & furniture lists with physical observation was found to be essential, to determine the functional status of the items.

Price data sources.

The price data of consumables, furniture items and equipment was often scattered and multiple sources were required to obtain complete information.(Fig 3)The most common source of consumable and equipment prices was the procurement office or central store, followed by online sources (such as online price databases, procurement portals). Additionally, information from the State Medical Procurement and Supplies department was sought for equipment prices. For furniture items prices, the central and departmental stores were the main sources.

thumbnail
Fig 3. Sources of data for price of consumables, equipment and furniture items.

https://doi.org/10.1371/journal.pone.0232873.g003

Form of data sources.

Fig 4 shows the variation in the form of data across input resources. Electronic databases were not available for data collection at most sites. The most common source of data was physical records maintained in respective departments of the hospital. In other cases where the required data was not available, expert opinions were sought.

Shared costs.

Shared resources often require additional data collection so as to enable apportioning of shared costs. The additional data included the duration of procedures conducted in the same premises collected through staff interview, number of patients sharing common resources collated from hospital records, time schedule of operation theatre collected from the hospital administration.

Level of difficulty of data collection.

The levels of difficulty in cost data collection as rated by the MRUs are shown in Fig 5. Input resources with high difficulty rating for data collection included enlisting utility of equipment in different procedures (6.59±0.52), prices of consumables (6.09±0.58), equipment procurement price (6.05±0.72) and prices of furniture items (5.64±0.68).

thumbnail
Fig 5. Rating of different resources on difficulty scale (1–10) in cost data collection.

https://doi.org/10.1371/journal.pone.0232873.g005

Results of the group discussion

Key discussion points (Table 4) included determination of rental price and its uniformity across sites, multiplicity of permissions required for data collection, unavailability of electronic hospital records and operational issues such as release of funds, authorization letters from central agency etc.

Evaluation from field teams on training, supervision and quality assurance

Training for cost data collection.

Hands on training and on-site support from the Central Supervisory team after the initial training was found to be very useful. Key concepts requiring further discussion and training were reported to be time allocation, medical terminologies, cost data collection in specific cost centres such as operation theatres. Participants suggested increase in time spent on practicum during the training and more frequent(quarterly) trainings and workshops.

Supervisory supports: Supervisory support by the central team and co-investigators who were trained at the central level was found to be of paramount importance in addressing day to day queries and conceptual understanding. It was suggested that the central team visit the sites to overlook data collection and provide hand-holding, especially with regards to data collection on revenue, time allocation and shared costs.

Quality assurance.

Quality assurance was of great importance, in particular to ensure standardisation and comparability in this multi-centre study. Key quality assurance strategies were highlighted in stakeholder interviews stating four levels of quality checks, namely checks during data collection, data entry, data sharing and finally on receipt of the data. During data collection, correctness and appropriateness of data was checked, for instance cross checking of time allocation hours. At the time of data entry correctness of entry was checked. Before sharing with the central team, the data was verified by the field supervisor. Finally the central team which carried out the analysis identified data gaps and discrepancies which were then communicated to the field teams. Teams then addressed those gaps by clarifying doubts, recollection of required data and re-entry of any incorrect data.

Discussion

Costing require information on numerous input resources and each of which have distinct data collection methods, making such activities rigorous and time consuming. Additionally, factors such as nature of costing (prospective or retrospective), costing methodology (top down or bottom up), form of data, number of people involved and willingness to share data, define the kind of challenges and time delays that may be faced during data collection. [30] Operational and methodological issues in cost data collection are further enhanced in developing countries like India with increased time taken to collate, input and assure data quality due to limited data availability, multiplicity of sources and unavailability of digitized data. [28,29] This study was targeted at identifying factors and barriers affecting cost data collection and potential solutions, in Indian and similar resource settings.

The study found that the average time required for cost data collection at one tertiary level public facility was 7 months or 355 person-days, inclusive of time required for permissions, actual data collection and data entry, queries & final submission, inclusive of delays such as non-working days. This was found to be higher than some other Indian studies which reported 25 and 45 days, however they had larger data collection teams with 8 and 20 members respectively. [30]

Our study found that a majority of the time was spent on data verification and clarifications to address erroneous data and address new data requirements. This was found to be consistent with other studies which reported that data verification could take longer than planned because incomplete or incorrect data. [30] A thorough pilot test of the tool to ensure its appropriateness and understanding of the data requirement by data collection team is the first step to improve the data quality and avoid erroneous data. Training was found to be essential in building an understanding about basics of costing, data requirements and importance of the costing activity amongst the data collection teams. Practical exposure to cost data collection was especially reported to be useful. The model of Training of Trainers at central level and subsequent training of data collectors by master trainers at local level was found to be useful by the trainees. The continued support from the central level team was also critical in local teams’ activities and the local teams stated the need to have further support in the form of on-site visits. Time was also lost due to the multiple levels of permissions required, gaining full participation of facilities and logistical delays. The permission requirements varied across the settings, however having an official consent letter from the head of the institution, holding stakeholder meetings and assuring confidentiality helped to gain confidence and smoothen the data collection process, ultimately reducing delays. This is keeping with methods employed in other studies. [30]

Implications for future research

The study identified certain areas of data collection that were more challenging than others and might require additional inputs or careful advance planning. The key lessons learned from this are summarised in (Table 5). An order of priority in data collection has been assigned to different data based on their share in total cost, difficulty level and level of uncertainty. The data collectors rated the prices of consumables, furniture, equipment and utility of equipment in different procedures as most difficult data type to collect. Determination of accurate building rental price was also difficult as there was no single source for this data. Prices and average useful life of the old furniture and equipment were not available as back dated records were not maintained at most facilities.

thumbnail
Table 5. Summary findings and implications for cost data collection.

https://doi.org/10.1371/journal.pone.0232873.t005

At the onset of cost data collection, decisions need to be made regarding key input resources to be prioritised, key data sources, time and human resources required for each set of data. As the same assumptions can be followed across centres in multi-centre costing studies, a reference data base for prices and average useful life of equipment and furniture items would save time. A national health system cost database which has been developed for India does make an attempt to bridge this limitation by providing representative data on prices and salaries. [31] This type of streamlining would be a cost-effective solution and is particularly justifiable in the case of equipment which only contributed 5% of the total cost.

Human resource data needs to be collected and handled carefully, since it has a major impact on the total cost. The combined cost of human resources and drugs & consumables accounted for 41% of the total cost, however, it required only 19% of total data collection time. It is essential to carefully tally time allocation tables of all human resources, delineate activities of the personnel and check over-reporting of service provision time. A barrier common to data collection of all resources was data extraction from physical records and registers, which is cumbersome and time-consuming. Sites with insufficient information management system have been known to have a high effort of conducting cost data collection which may even make it non-feasible for the government. [32] Availability of reliable electronic health records would facilitate ease of data collection and prevent time delays and is likely to have beneficial effects in other areas such as patient care and facility management. In addition, it is important to use the skill mix of interviewers for appropriate nature of data collector. Data on items such as overheads, biomedical waste, leave record, consumable indent, building area measurement generally require less expertise and may be collected by non-technical staff. On the other hand, time allocation of human resources and utility for various resources, which are used for apportioning, requires an understanding of the systems and processes and hence need more technical staff for its collection.

Our study findings suggest that during planning of cost data collection, it is essential to remember four important sets of considerations. (Fig 6) The first step is to engage all stakeholders to develop an understanding about the significance of the costing activity, type of data required and streamlining permissions which helps in prevention of delays during data collection. Engagement with stakeholders also builds ownership in the findings for the study. This is particularly relevant if the costing is being done for the purpose of price setting or reimbursement. Next is preparation of all inputs for cost data collection including training, mentoring support and conducting a pilot study. Thirdly, development of a data collection plan specifying cost centres, data sources, norms and suggestions for best practice approaches (Table 3). The final consideration is review and quality check of the data. Based on the CHSI experience, there should be a system of checks in place at four levels: at the time of data collection, data entry, data sharing and finally on receipt of the data. At all steps of the data collection process, it is important to keep a communication system between the site team and the supervisory team, and between the supervisory team and other stakeholders.

The lessons learned are particularly relevant for two sets of reasons. Firstly, the HTA studies are being commissioned in large numbers which require cost evidence to begin with. Secondly, the emergence of publicly financed health insurance schemes such as AB–PMJAY require evidence on cost to set prices for provides. As a result of these twin developments, it is expected that there will be a rise in number of cost analysis studies. Hence, our study findings are likely to be useful in design, conduct and quality assurance of those future studies. While the findings of this study are based only on public tertiary level care, they can be used to infer about any cost assessment to be undertaken at primary and secondary care levels as well. Although there are likely to be differences in the data sources and time required for data collection, the main principles of collecting information on different resources, key assumptions and barriers are likely to be similar.

We would like to note certain study limitations. Firstly, while we assess the level of difficulty faced in data collection, we could not assess and report in quantitative terms. Moreover, this quality assessment, if done according to skill-mix of data collection can provide very useful insight into which type of data collection could be assigned which type of data collection. Further, the difficulty rating was based on a joint decision of the data collection team and not an individual, hence it is not possible to comment on the impact of skill mix of the data collector on data quality. Hence, our current recommendation is based more on the level of difficulty faced and importance of particular data in extent and accuracy of cost, rather than quality of data collected. Secondly, while we collected the time spent in person-days on different tasks for each individual health facility and cost centre, this data was not available disaggregated by individual staff with different level or skill mix. As a result, accurate estimation of the cost of data collection for individual activities or individual cost centres was not possible. Thirdly, this paper focussed on findings from tertiary hospitals where costing was under taken as a part of the CHSI study. Similar studies may be done in the future for district hospitals, private hospitals and other healthcare delivery levels where the CHSI study is being undertaken. Thirdly, there is a significant likelihood of differences in the processes and systems for data collection in private sector hospitals, and more decentralized public sector facilities such as district hospitals, CHCs and PHCs. In view of the large contribution of private sector in service delivery in India, understanding the process of costing in these facilities becomes imperative. Finally, we have highlighted the issues in the context of a study which follows a mixed methodology of top-down and bottom-up. It will be useful to replicate this study in the context of pure bottom-up and top-down costing methodologies, as well as studies from a financial and economic perspective, so as to draw specific lessons for each of the different methods.

Conclusion

The Government of India is committed to health technology assessment as it is the need of the hour. Improving methodology and practical implementation of costing studies will improve the quality of estimates being generated and ultimately aid national priority setting. For implementation of costing studies, having clearly outlined outputs, conducting a pilot test, having a quick communication between the data collectors and central team, specific quality checks and assumptions, are key considerations. In addition to practical aspects, it is essential to strengthen the health system to aid costing studies. Developing electronic health records, having nationally representative cost data databases for reference in case of unavailability of data and developing guidelines on costing in Indian settings are key deliberations.

Acknowledgments

CHSI Study Collaborating Investigators: Kusum Lata Gaur, Jai Prakash Pankaj, Dharmesh Kumar Sharma (Sawai Man Singh Medical College, Jaipur, Rajasthan); Divya Mehrotra (King George's Medical University, Lucknow, Uttar Pradesh); Prakash Patel, Amit C Patel, Rajendra Joshi (Surat Municipal Institute of Medical Education & Research, Surat, Gujarat); Gajanan D Velhal, Amit S Bhondve (Seth G S Medical College & KEM Hospital, Mumbai, Maharashtra); Avijit Hazra, Raghunath Misra (Institute of Postgraduate Medical Education & Research, Kolkata, West Bengal); Sanjay Kumar, Setu Sinha, Varsha Singh (Indira Gandhi Institute of Medical Science, Patna, Bihar); Bhabagrahi Rath, Sashi Bhusan Biswal, Ankita Panigrahy (Veer Surendra Sai Institute of Medical Sciences and Research, Burla, Odisha), Rakesh Bahl, Amit Sachdeva, Sharminder Kaur (Government Medical College, Jammu, Jammu & Kashmir); K. Narayanasamy, A. Chitra (Madras Medical College, Chennai, Tamil Nadu); Kondeti Madhavi, Pulaganti Madhusudana (Sri Venkateswara Medical College, Tirupati, Andhra Pradesh); Akashdeep Singh Chauhan (Post Graduate Institute of Medical Education and Research, Chandigarh)

We would like to acknowledge the contribution of the CHSI Collaborating Investigators’ group which led the data collection at the study sites and comprised of the following members:

  1. A. Chitra, Professor, Institute of Community Medicine, Madras Medical College, Chennai, Tamil Nadu
  2. Akashdeep Singh Chauhan, Department of Community Medicine and School of Public Health, Post Graduate Institute of Medical Education and Research, Chandigarh
  3. Amit C Patel, Department of Community Medicine, Surat Municipal Institute of Medical Education & Research, Surat, Gujarat
  4. Amit S Bhondve, Department of Community Medicine, Seth G S Medical College & KEM Hospital, Mumbai, Maharashtra
  5. Amit Sachdeva, Government Medical College, Jammu, Jammu & Kashmir
  6. Ankita Panigrahy, Department of Pharmacology, Veer Surendra Sai Institute of Medical Sciences and Research, Burla, Odisha
  7. Avijit Hazra, Department of Pharmacology, Institute of Postgraduate Medical Education & Research, Kolkata, West Bengal
  8. Bhabagrahi Rath, Department of Pharmacology, Veer Surendra Sai Institute of Medical Sciences and Research, Burla, Odisha
  9. Dharmesh Kumar Sharma, Sawai Man Singh Medical College, Jaipur, Rajasthan
  10. Divya Mehrotra, King George's Medical University, Lucknow, Uttar Pradesh
  11. Gajanan D Velhal, Department of Community Medicine, Seth G S Medical College & KEM Hospital, Mumbai, Maharashtra
  12. Jai Prakash Pankaj, Sawai Man Singh Medical College, Jaipur, Rajasthan
  13. K. Narayanasamy, Department of Hepatology, Madras Medical College, Chennai, Tamil Nadu
  14. K. Madhavi, Sri Venkateswara Medical College, Tirupati, Andhra Pradesh
  15. Kusum Lata Gaur, Sawai Man Singh Medical College, Jaipur, Rajasthan
  16. P. Madhusudan, Multidisciplinary Research Unit, Sri Venkateswara Medical College, Tirupati, Andhra Pradesh
  17. Prakash Patel, Department of Community Medicine, Surat Municipal Institute of Medical Education & Research, Surat, Gujarat
  18. Raghunath Misra, Institute of Postgraduate Medical Education & Research, Kolkata, West Bengal
  19. Rajendra Joshi, Department of Physiology, Surat Municipal Institute of Medical Education & Research, Surat, Gujarat
  20. Rakesh Behl, Government Medical College, Jammu, Jammu & Kashmir
  21. Sanjay Kumar, Department of Community Medicine, Indira Gandhi Institute of Medical Science, Patna, Bihar.
  22. Sashi Bhusan Biswal, Department of Pharmacology, Veer Surendra Sai Institute of Medical Sciences and Research, Burla, Odisha
  23. Setu Sinha, Department of Community Medicine, Indira Gandhi Institute of Medical Science, Patna, Bihar
  24. Sharminder Kaur, Government Medical College, Jammu, Jammu & Kashmir
  25. Varsha Singh, Department of Community Medicine, Indira Gandhi Institute of Medical Science, Patna, Bihar

References

  1. 1. Press Information Bureau. Government of India. Ministry of Health and Family Welfare. Ayushman Bharat–Pradhan Mantri Jan AarogyaYojana (AB-PMJAY). New Delhi 2018 [Available from: http://pib.nic.in/newsite/PrintRelease.aspx?relid=183624.
  2. 2. Jain S, Rajshekar K, Sohail A, Gauba VK. Department of health research-health technology assessment (DHR-HTA) database: National prospective register of studies under HTAIn. Vol. 148, Indian Journal of Medical Research. Wolters Kluwer Medknow Publications; 2018. p. 258–61. https://doi.org/10.4103/ijmr.IJMR_1613_18 pmid:30425215
  3. 3. Prinja S, Chauhan AS, Angell B, Gupta I, Jan S. A Systematic Review of the State of Economic Evaluation for Health Care in India. Appl Health Econ Health Policy [Internet]. 2015 Dec 8 [cited 2019 Jun 30];13(6):595–613. Available from: http://link.springer.com/10.1007/s40258-015-0201-6 pmid:26449485
  4. 4. Prinja S, Downey LE, Gauba VK, Swaminathan S. Health Technology Assessment for Policy Making in India: Current Scenario and Way Forward. PharmacoEconomics—open [Internet]. 2018 Mar [cited 2019 Jun 30];2(1):1–3. Available from: http://www.ncbi.nlm.nih.gov/pubmed/29464668 pmid:29464668
  5. 5. Prinja S, Gupta A, Bahuguna P, Nimesh R. Cost analysis of implementing mHealth intervention for maternal, newborn & child health care through community health workers: assessment of ReMIND program in Uttar Pradesh, India. BMC Pregnancy Childbirth [Internet]. 2018 Dec 3 [cited 2019 Aug 6];18(1):390. Available from: https://bmcpregnancychildbirth.biomedcentral.com/articles/10.1186/s12884-018-2019-3 pmid:30285669
  6. 6. Kaur G, Prinja S, Ramachandran R, Malhotra P, Gupta KL, Jha V. Cost of hemodialysis in a public sector tertiary hospital of India. Clin Kidney J [Internet]. 2018 Oct [cited 2019 Aug 6];11(5):726–33. Available from: http://www.ncbi.nlm.nih.gov/pubmed/30288270 pmid:30288270
  7. 7. Chatterjee S, Laxminarayan R. Costs of surgical procedures in Indian hospitals. BMJ Open [Internet]. 2013 Jun 20 [cited 2019 Aug 5];3(6):e002844. Available from: http://www.ncbi.nlm.nih.gov/pubmed/23794591 pmid:23794591
  8. 8. Chatterjee S, Laxminarayan R, Gosselin RA. Cost Per DALY Averted in a Surgical Unit of a Private Hospital in India. World J Surg [Internet]. 2016 May 16 [cited 2019 Aug 5];40(5):1034–40. Available from: http://www.ncbi.nlm.nih.gov/pubmed/26675929 pmid:26675929
  9. 9. Gajuryal S, Daga A, Siddharth V, Bal C, Satpathy S. Unit cost analysis of pet-ct at an apex public sector health care institute in India. Indian J Nucl Med [Internet]. 2017 [cited 2019 Aug 5];32(1):1. Available from: http://www.ijnm.in/text.asp?2017/32/1/1/198438 pmid:28242974
  10. 10. Hada M, Chakravarty A, Mukherjee P. Activity based costing of diagnostic procedures at a nuclear medicine center of a tertiary care hospital. Indian J Nucl Med [Internet]. 2014 Oct [cited 2019 Aug 5];29(4):241. Available from: http://www.ncbi.nlm.nih.gov/pubmed/25400363 pmid:25400363
  11. 11. Shelat PR, Kumbar SK. A Retrospective Analysis of Direct Medical Cost and Cost of Drug Therapy in Hospitalized Patients at Private Hospital in Western India. J Clin Diagn Res [Internet]. 2015 Nov [cited 2019 Aug 5];9(11):FC09–12. Available from: http://www.ncbi.nlm.nih.gov/pubmed/26675983 pmid:26675983
  12. 12. Prinja S, Sharma Y, Dixit J, Thingnam SKS, Kumar R. Cost of Treatment of Valvular Heart Disease at a Tertiary Hospital in North India: Policy Implications. PharmacoEconomics—Open [Internet]. 2019 Feb 19 [cited 2019 Aug 6];1–12. Available from: http://link.springer.com/10.1007/s41669-019-0123-6
  13. 13. Chauhan AS, Prinja S, Ghoshal S, Verma R, Oinam AS. Cost of treatment for head and neck cancer in India. Devleesschauwer B, editor. PLoS One [Internet]. 2018 Jan 11 [cited 2019 Aug 6];13(1):e0191132. Available from: https://dx.plos.org/10.1371/journal.pone.0191132 pmid:29324861
  14. 14. Prinja S, Bahuguna P, Duseja A, Kaur M, Chawla YK. Cost of Intensive Care Treatment for Liver Disorders at Tertiary Care Level in India. PharmacoEconomics—Open [Internet]. 2018 Jun 14 [cited 2019 Jun 30];2(2):179–90. Available from: http://www.ncbi.nlm.nih.gov/pubmed/29623618 pmid:29623618
  15. 15. Kaur G, Prinja S, Malhotra P, Lad DP, Prakash G, Khadwal A, et al. Cost of Treatment of Multiple Myeloma in a Public Sector Tertiary Care Hospital of North India. Indian J Hematol Blood Transfus [Internet]. 2018 Jan 3 [cited 2019 Jun 30];34(1):25–31. Available from: http://www.ncbi.nlm.nih.gov/pubmed/29398796 pmid:29398796
  16. 16. Bahuguna P, Khanduja P, Prinja S. Economic Analysis of Delivering Postpartum Intrauterine Contraceptive Device Services in India. Indian J Community Med [Internet]. [cited 2019 Aug 5];44(2):147–51. Available from: http://www.ncbi.nlm.nih.gov/pubmed/31333294 pmid:31333294
  17. 17. Sharma A, Prinja S, Sharma A, Gupta A, Arora SK. Cost of antiretroviral treatment for HIV patients in two centres of North India. Int J STD AIDS [Internet]. 2019 Jul 12 [cited 2019 Aug 6];30(8):769–78. Available from: http://journals.sagepub.com/doi/10.1177/0956462419839852 pmid:31081489
  18. 18. Dandona L, Sisodia P, Prasad T, Marseille E, Chalapathi Rao M, Kumar AA, et al. Cost and efficiency of public sector sexually transmitted infection clinics in Andhra Pradesh, India. BMC Health Serv Res [Internet]. 2005 Dec 5 [cited 2019 Aug 5];5(1):69. Available from: http://www.ncbi.nlm.nih.gov/pubmed/16271151
  19. 19. Bajwala VR, John D, Rajasekar TD, Murhekar M V. Severity and costs associated with hospitalization for dengue in public and private hospitals of Surat city, Gujarat, India, 2017–2018. Trans R Soc Trop Med Hyg [Internet]. 2019 Jul 11 [cited 2019 Aug 5]; Available from: http://www.ncbi.nlm.nih.gov/pubmed/31294808
  20. 20. Kumar P, Jithesh V, Gupta SK. A comparative cost analysis of polytrauma and neurosurgery Intensive Care Units at an apex trauma care facility in India. Indian J Crit Care Med [Internet]. 2016 Jul [cited 2019 Aug 5];20(7):398–403. Available from: http://www.ncbi.nlm.nih.gov/pubmed/27555693 pmid:27555693
  21. 21. Prinja S, Gupta A, Verma R, Bahuguna P, Kumar D, Kaur M, et al. Cost of Delivering Health Care Services in Public Sector Primary and Community Health Centres in North India. Beck EJ, editor. PLoS One [Internet]. 2016 Aug 18 [cited 2019 Aug 6];11(8):e0160986. Available from: https://dx.plos.org/10.1371/journal.pone.0160986 pmid:27536781
  22. 22. Prinja S, Jeet G, Verma R, Kumar D, Bahuguna P, Kaur M, et al. Economic Analysis of Delivering Primary Health Care Services through Community Health Workers in 3 North Indian States. Carlo WA, editor. PLoS One [Internet]. 2014 Mar 13 [cited 2019 Jun 30];9(3):e91781. Available from: https://dx.plos.org/10.1371/journal.pone.0091781 pmid:24626285
  23. 23. Prinja S, Balasubramanian D, Jeet G, Verma R, Kumar D, Bahuguna P, et al. Cost of delivering secondary-level health care services through public sector district hospitals in India. Indian J Med Res [Internet]. 2017 [cited 2019 Jun 30];146(3):354–61. Available from: http://www.ncbi.nlm.nih.gov/pubmed/29355142 pmid:29355142
  24. 24. Prinja S, Manchanda N, Mohan P, Gupta G, Sethy G, Sen A, et al. Cost of neonatal intensive care delivered through district level public hospitals in India. Indian Pediatr [Internet]. 2013 Sep [cited 2019 Jun 30];50(9):839–46. Available from: http://www.ncbi.nlm.nih.gov/pubmed/23502671 pmid:23502671
  25. 25. Prinja S, Mazumder S, Taneja S, Bahuguna P, Bhandari N, Mohan P, et al. Cost of delivering child health care through community level health workers: how much extra does IMNCI program cost? J Trop Pediatr [Internet]. 2013 Dec 1 [cited 2019 Aug 6];59(6):489–95. Available from: https://academic.oup.com/tropej/article-lookup/doi/10.1093/tropej/fmt057 pmid:23872793
  26. 26. Chatterjee S, Levin C, Laxminarayan R. Unit cost of medical services at different hospitals in India. PLoS One [Internet]. 2013 [cited 2019 Aug 5];8(7):e69728. Available from: http://www.ncbi.nlm.nih.gov/pubmed/23936088 pmid:23936088
  27. 27. Sangwan A, Prinja S, Aggarwal S, Jagnoor J, Bahuguna P, Ivers R. Cost of Trauma Care in Secondary- and Tertiary-Care Public Sector Hospitals in North India. Appl Health Econ Health Policy [Internet]. 2017 Oct 13 [cited 2019 Aug 6];15(5):681–92. Available from: http://link.springer.com/10.1007/s40258-017-0329-7 pmid:28409489
  28. 28. Batura N, Pulkki-Brännström A-M, Agrawal P, Bagra A, Haghparast-Bidgoli H, Bozzani F, et al. Collecting and analysing cost data for complex public health trials: reflections on practice. Glob Health Action [Internet]. 2014 [cited 2019 Jun 19];7:23257. Available from: http://www.ncbi.nlm.nih.gov/pubmed/24565214 pmid:24565214
  29. 29. Griffiths UK, Legood R, Pitt C. Comparison of Economic Evaluation Methods Across Low-income, Middle-income and High-income Countries: What are the Differences and Why? Heal Econ (United Kingdom). 2016 Feb 1;25:29–41.
  30. 30. Özaltın A, Cashin C, Acheampong OB, Asenso-Boadi F, Beaulieu K, Chatterjee S, et al. Joint Learning Network for Universal Health Coverage. Costing of Health Services for Provider Payment: A Practical Manual Based on Country Costing Challenges, Trade-offs, and Solutions. 2014.
  31. 31. PGIMER. National Health System Cost Database for India; 2018 [cited 2020 Feb 25] Database [Internet]. Available from https://www.healtheconomics.pgisph.in/costing_web/
  32. 32. Chootipongchaivat S, Tritasavit , Luz A, Teerawattananon , Tantivees S. Factors conducive to the development of health technology assessment in Asia: Impacts and Policy Options. Manila: World Health Organization, Regional Office for the Western Pacific, 2015.