Next Article in Journal
High-Grade Infection after Branched Endovascular Aortic Repair in Patient with Recent COVID-19 Hospitalization
Next Article in Special Issue
The Role of Cardiovascular Magnetic Resonance Imaging in the Assessment of Mitral Regurgitation
Previous Article in Journal
Diagnosis of Human Cytomegalovirus Drug Resistance Mutations in Solid Organ Transplant Recipients—A Review
Previous Article in Special Issue
Experimental Examination of Conventional, Semi-Automatic, and Automatic Volumetry Tools for Segmentation of Pulmonary Nodules in a Phantom Study
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Ultrasound in the Diagnosis of Non-Expandable Lung: A Prospective Observational Study of M-Mode, B-Mode, and 2D-Shear Wave Elastography

by
Jesper Koefod Petersen
1,2,
Katrine Fjaellegaard
1,2,
Daniel B. Rasmussen
1,2,
Gitte Alstrup
1,
Asbjørn Høegholm
1,
Jatinder Singh Sidhu
1,
Pradeesh Sivapalan
3,
Oke Gerke
4,5,
Rahul Bhatnagar
6,
Paul Frost Clementsen
7,
Christian B. Laursen
8,9 and
Uffe Bodtger
1,2,*
1
Respiratory Research Unit, Department of Internal and Respiratory Medicine, Zealand University Hospital, 4000 Roskilde, Denmark
2
Institute of Regional Health Research, University of Southern Denmark, 5000 Odense, Denmark
3
Section of Respiratory Medicine, Department of Medicine, Herlev and Gentofte Hospital, University of Copenhagen, 2900 Hellerup, Denmark
4
Department of Nuclear Medicine, Odense University Hospital, 5000 Odense, Denmark
5
Department of Clinical Research, University of Southern Denmark, 5000 Odense, Denmark
6
Academic Respiratory Unit, University of Bristol, Bristol BS8 1TU, UK
7
Copenhagen Academy for Medical Education and Simulation (CAMES), University of Copenhagen and the Capital Region of Denmark, 2100 Copenhagen, Denmark
8
Department of Respiratory Medicine, Odense University Hospital, 5000 Odense, Denmark
9
Odense Respiratory Research Unit (ODIN), Department of Clinical Research, University of Southern Denmark, 5000 Odense, Denmark
*
Author to whom correspondence should be addressed.
Diagnostics 2024, 14(2), 204; https://doi.org/10.3390/diagnostics14020204
Submission received: 10 December 2023 / Revised: 7 January 2024 / Accepted: 11 January 2024 / Published: 18 January 2024
(This article belongs to the Special Issue Advances in Cardiovascular and Pulmonary Imaging)

Abstract

:
Background: Non-expandable lung (NEL) has severe implications for patient symptoms and impaired lung function, as well as crucial implications for the management of malignant pleural effusion (MPE). Indwelling pleural catheters have shown good symptom relief for patients with NEL; hence, identifying patients early in their disease is vital. With the inability of the lung to achieve pleural apposition following thoracentesis and the formation of a hydropneumothorax, traditionally, chest X-ray and clinical symptoms have been used to make the diagnosis following thoracentesis. It is our aim to investigate whether ultrasound measurement of lung movement during respiration can predict NEL before thoracentesis, thereby aiding clinicians in their planning for the optimal treatment of affected patients. Methods: A total of 49 patients were consecutively included in a single-centre trial performed at a pleural clinic. Patients underwent protocolled ultrasound assessment pre-thoracentesis with measurements of lung and diaphragm movement and shear wave elastography measurements of the pleura and pleural effusion at the planned site of thoracentesis. Results: M-mode measurements of lung movement provided the best diagnostic ROC-curve results, with an AUC of 0.81. Internal validity showed good results utilising the calibration belt test and Brier test. Conclusion: M-mode measurement of lung movement shows promise in diagnosing NEL before thoracentesis in patients with known or suspected MPE. A validation cohort is needed to confirm the results.

1. Introduction

Non-expandable lung (NEL) describes the inability of the lung to fully expand and achieve full pleural apposition. This typically occurs following pleural inflammation and subsequent visceral pleural fibrosis, but can also result from endobronchial obstruction [1,2,3]. NEL presents in recurrent pleural effusions with breathlessness, chest pain, or cough, which limits fluid drainage during thoracentesis. NEL also limits the efficacy of pleurodesis procedures due to the absence of pleural apposition [4,5].
NEL often presents as suspected iatrogenic pneumothorax after thoracentesis, in the form of hydropneumothorax on chest X-ray (CXR) [6]. Predicting the presence of NEL prior to thoracentesis remains challenging, and there is no standard approach [7], although techniques have been proposed to identify the condition both before and during thoracentesis. Previous studies have typically used a combination of radiological findings and intra-procedure clinical symptoms, but definitions have been inconsistent [7,8,9,10].
Intermittent pleural manometry during drainage has been suggested as the diagnostic method of choice, and is used in some centres [7]. Several studies [11,12] have proposed manometric cut-off values diagnostic of NEL; however, discordance between radiological and pleural manometry readings are a factor. While a high degree of pleural apposition on chest radiography predicts successful pleurodesis [5,7,13,14], fewer studies using pleural manometry have compared the same outcomes [7,15]. The recent study by Chopra and colleagues [16] showed that 28% of patients with elevated pleural elastance had complete lung expansion following thoracentesis, and 34% of patients with incomplete lung expansion detected by post-thoracentesis CXR had normal pleural elastance.
TUS assessment prior to thoracentesis is now considered the standard of care, and is included in several national and international recommendations [17,18]. Research groups have proposed TUS as a novel method of identifying NEL [19,20] before thoracentesis. Several ultrasound-based physiological measurements have been developed, including diaphragmatic and pulmonary movement [21,22,23,24]. Lung movement assessed by M-mode has been proposed as a possible method of identifying NEL before thoracentesis [19]. In one study, reduced lung pulse measured with M-mode measurement resulted in sensitivity and specificity values of 50% and 85% [20]. Ultrasound 2D shear wave elastography (SWE) allows for the characterization of tissue according to acoustic resilience, with pathological tissue differing in its mechanic properties compared to the surrounding normal tissue. In general, stiffer tissue results in faster tissue shear wave propagation [25], and this technique has previously been applied to help differentiate benign from malignant breast masses [26], subpleural solid masses [27], pleural effusions, and hilar lymph nodes [28,29]. However, it has no role in current guidelines for the work-up of suspected NEL [18,30].
In this study, we hypothesize that decreased movement of the visceral pleura in the context of NEL will be detectable by ultrasound measurement. We also hypothesize that SWE is superior to B-mode and M-mode in the diagnosis of NEL in patients with suspected MPE as it differentiates normal from fibrotic pleural lining.

2. Materials and Methods

A prospective observational study was conducted at a single secondary care hospital in Denmark (Zealand University Hospital, Naestved). Patients were eligible if they were over 18 years old, with a pleural effusion of over 2 cm depth needing thoracentesis in the context of suspected or known malignancy. Exclusion criteria were an estimated life expectancy of less than three months or a contraindication to thoracentesis. All patients provided informed consent. The study was approved by the Regional Ethics Committee (approval nr. SJ-891) and Region Zealand Data Committee (REG-001-2021). All TUS examination and thoracenteses were performed by two experienced and certified operators (JKP and KF).
Patient examination followed the following steps: pre-procedure baseline data collection and ultrasound measurements at site of planned thoracentesis, incorporating B-mode, M-mode, and SWE. Thoracentesis was performed and the hemithorax was assessed by ultrasound for complete pleural fluid drainage status, defined as less than 0.1 L remaining fluid assessed by visual estimation, and pleural apposition. All patients had a CXR after thoracentesis, except if a chest computer tomography (CT) was booked by the treating oncologist or physician the same day. The presence or absence of NEL was recorded following the four-month follow-up period from thoracentesis using the electronic patient records.

2.1. Pre-Procedure Baseline Data

Patient demographics and patient questionnaires were collected before examination. Medical records were examined for causes of pleural effusion.

2.2. Ultrasound Examinations

Ultrasound assessments were performed using LOGIQ S8 (GE Healthcare, Wauwatosa, WI, USA) with a C1-6-D curved (2–5 Mhz) transducer in the abdominal preset. Patients were scanned while seated upright, leaning forward, and resting upon a support stand. To avoid exertion-induced changes in respiration rate, patients were given least 5 min of rest before the examination was performed.
All measurements were obtained from a designated optimal site of thoracentesis, allowing for some minor transducer movement to optimize imaging (e.g., moving out of rib shadow). In order to identify and mark a safe site for thoracentesis, the diaphragm was located, effusion size and characteristics (e.g., septations, depth of effusion) were noted, and proximity to organs was considered. All effusions were basal.
Pleural effusion was evaluated by pleural thickening (>10 mm), swirling and septations, atelectasis/consolidation of the lung, pleural nodules, and lung sliding apically to the effusion (present or non-present). Four-second movie clips were recorded for measurements.

2.3. B-Mode

Maximum diaphragm movement was assessed from US recordings, and diaphragm amplitude was measured in centimetres. The area method, as described by Skaarup et al. [24], was utilized, with a lateral, mid-axillary view targeting optimal diaphragm visualization. A film clip was saved, and images of maximal inspiration and expiration were identified. A region from above the diaphragm was traced, and the area was calculated using the area-function of the ultrasound machine. The maximum areas of inspiration and expiration were calculated and subtracted, resulting in a measurement of diaphragm movement.

2.4. M-Mode

Diaphragm movement was measured at the area of largest amplitude of the diaphragm with the targeting line. Lung movement (visceral pleura) was assessed at the perpendicular point of thoracentesis, as illustrated in Figure 1. Amplitude results are provided in centimetres with two decimals.

2.5. 2D Shear Wave Elastography (SWE)

Measurements from three structures were made: five recordings of the parietal pleura, the pleural effusion superficially to the lung, and the visceral pleura/consolidation perpendicular to the transducer (Figure 1). Care was taken to apply minimal pressure from the transducer to the chest wall. Readings were obtained while breath was held. A region of interest (ROI) was selected upon the elastogram, providing adequate shear wave propagation, and was adjusted to size. Measurements were obtained in m/s [31].

2.6. Thoracentesis

Thoracentesis was performed according to local guidelines, with US-guided administration of local anaesthesia in the skin and intercostal structures [17]. A 7 French pigtail catheter was inserted and connected to a sealed system. The effusion was drained under gravity until fluid cessation (with a pause for each 1000 mL fluid removed) or until the patient experienced discomfort, e.g., tightness of chest, dyspnoea, or pain. Drainage was discontinued if symptoms were not alleviated by a pause. At premature flow cessation, the drain was flushed with 50 mL saline to check for blockage. CXR was performed within 2 h after thoracentesis.

2.7. Scoring of Non-Expandable Lung (NEL)—Reference Standard

As per the method described by Salamonsen et al. [20], NEL was defined based on a combination of clinical observations and post-thoracentesis radiology (Table A3). Two interventional pulmonologists (AS and JS) independently assessed the study data after four months of follow-up. Post-thoracentesis radiology reports and images obtained both directly following thoracentesis and in the follow-up period were available, and assessors recorded whether they observed air (or hydropneumothorax) in the pleural space surrounding the lower lobe, with examples shown in Figure 2. The assessors were blinded to the results from the index tests, e.g., SWE, M-mode, and B-mode measurements. The outcome was classified as: NEL (definite NEL, probable NEL), expandable lung (EL) (probably expandable, definitely expandable), or “Unable to score”. In the event of disagreeing diagnoses between assessors, a third interventional pulmonologist (UB) assessed the study information and a consensus was achieved. All assessments were made while blinded to the other assessor’s results. The assessors were provided with clinical observations during thoracentesis: development of chest tightness or severe coughing not alleviated by thoracentesis pause, air in chest tube, or undrainable pleural effusion despite unblocked chest tube.

2.8. Statistical Analyses

Normality distribution was tested using density histograms and the Shapiro–Wilk test of normality for continuous variables. Normally and non-normally distributed variables were expressed as means (±standard deviation, SD) and medians (interquartile range 25–75%, IQR), respectively.
As our ultrasound measurements had not been tested before in the context of diagnosing NEL, we allowed for a broad approach to the statistical analysis. Receiver-operating characteristic (ROC) curves were constructed to calculate the areas under the curve (AUC) for M-mode (lung and diaphragm movement), B-mode (diaphragm movement, area method: delta area), and SWE (parietal pleura, pleural effusion, and visceral pleura).
Several methods were explored to approximate the most clinically relevant variables. A cut-off at minimum 80% specificity was chosen as clinically relevant. For an estimation of an optimal cut-off point, the Youden index was chosen [32] to maximise the difference between the sensitivity and the false positive rate. For the best-performing model, a Brier score was calculated to evaluate the goodness of predicted probability. A calibration belt plot for internal validity [33,34] was used to estimate the accuracy of the measurements compared to the true outcome observed in the data.
As the cohort was too small to be stratified into both development and validation cohorts, a logistic regression analysis was performed to propose an optimized model for a future validation study.
The null hypothesis was rejected at the level of 5% (p ≤ 0.05). Statistical analysis was performed using STATA (StataCorp. 2021. Stata Statistical Software: Release 17. College Station, TX, USA).

3. Results

Between May 2021 and December 2021, 132 patients were screened, and 49 patients were included in the study. Of those excluded, 22 had too little fluid, 34 had benign aetiologies, 13 had short life expectancies, 12 declined, and 2 could not give consent. Of the included patients, 14/49 (29%) were categorized as NEL (Table 1). The mean age was 71 years (SD 9), and 25 patients were male (51%). The average volume of pleural fluid removed was 1178 mL (SD 629 mL), and the majority (n = 48; 98%) had post-thoracentesis radiological examinations immediately (Table 1). All ultrasound tests showed right-skewed distribution and tested significantly for non-parametric distribution.
At follow-up, most (n = 44; 90%) of the patients were diagnosed with malignancy (lung cancer: 23; breast cancer: 11; mesothelioma: 4), and the remaining had pleural infection, chronic pleuritis, or congestive heart failure. In total, 13 patients with NEL had malignancy at follow-up: lung cancer (6), mesothelioma (4), and other cancers (3).
Concerning the diagnostic property of TUS in NEL, Table 2 shows the AUCs from all eight ultrasonographic tests. We found SWE to be inferior compared to M-mode of lung movement, M-mode movement, and B-mode of diaphragm movement (AUC 0.81, 0.77, and 0.65, respectively). The AUCs of SWE were 0.57, 0.47, and 0.59 for parietal pleura, pleural effusion, and visceral pleura, respectively. ROC curves are shown in Figure 3.
Table A1 shows the ideal cut-point at which to optimize ultrasound diagnostic yield of TUS measures while allowing for at least 75% and 80% sensitivity and specificity. For each of the three best-performing tests, as given by the AUC, optimal cut-off points were calculated according to Youden’s index. The Brier score was calculated at 0.15, and the calibration belt plot showed a high degree of internal validity, with a test statistic of 0.08 and p-value of 0.78. As seen in Figure A1, the calibration belts encompassed the bisector at all times. This suggests that the predictions of the model did not significantly deviate from the observed findings in our developmental sample.
We found no diagnostic benefit of combining different TUS measures (Table A2). In univariate analyses, M-mode lung movement, M-mode diaphragm movement, and lung sliding were significantly associated with NEL. In multivariate regression, only M-mode lung movement remained significant (OR 7.72, 95% CI 0.84–70.90, p = 0.071). No combination of test modalities significantly increased the AUC above the ability of M-mode lung alone (Table A2).

4. Discussion

NEL represents a unique aspect in pleural disease pathophysiology, involving an increase in pleural elastance and pleural infiltration that results in a stiffer visceral membrane and restricted lung movement [1]. Cancer, which often results in malignant pleural effusion (MPE), is a leading cause of NEL, usually as a consequence of persistent pleural inflammation, disruption of the pleural barrier, atelectatic tumour obstruction, and pleural restriction [3]. Some studies have suggested that upwards of 30% [8,35] of patients with MPE are at risk of developing NEL, and that NEL may be the most frequent cause of pneumothorax following ultrasound-guided thoracentesis [11]. The high incidence of MPE and NEL [36] and the significant impact on quality of life of the affected patients warrant increased focus on these conditions.
Both MPE and NEL have been associated with a greater healthcare burden and shorter survival [37]; MPE’s median survival has been estimated at 5–8 months. Pleurodesis and placement of an indwelling pleural catheter (IPC) are the most common “definitive” procedures for patients with recurrent effusions in a cancer setting. For those patients with NEL, treatment of dyspnoea can be challenging, but in those who gain relief from thoracentesis, IPC has been proven to be the best treatment option [13,38,39]. Early detection of NEL facilitates optimal MPE treatment, which reduces symptom burden and time spent in hospital.
Ultrasonographic elastography was introduced in the 1990s and is now part of routine assessments of liver fibrosis and breast lesions [40]. However, it currently has no role in the work-up of suspected NEL [18,30]. In this first study on 2D shear wave elastography in the diagnosis of non-expandable lung, we found SWE to be inferior to both conventional M- and B-mode measurements of lung and diaphragm movement in predicting NEL. Lung movement measured by M-mode showed good predictive values on both ROC curves, with an AUC of 0.81 and clinical viable cut-off values as applied. Youden’s index showed promising results for sensitivity, specificity, PPV, and NPV of 0.78, 0.71, 0.52, and 0.89, respectively. Thus, our data supports the findings of both Salamonsen [20] and Flora [19], i.e., that hindered movement of the lung is a valid predictor of NEL. This also complies well with the pathophysiological description of NEL.
Table A4 lists selected cut points, allowing for an overview of the predictive capabilities of the measurements. Several methods exist for optimizing cut-off points, all with their individual trade-offs. Youden’s index is frequently used and presents a commonly used theoretical cut-off. Other methods, such as Liu’s and “nearest to (0, 1)”, showed the same cut-off and AUC (Table A3).
Our study, however, differs in several ways from those conducted previously [20]. In the Salamonsen study, TUS was performed by cardiologists or pulmonologists trained by cardiologists. Additionally, the ultrasound measurements which were relied upon (e.g., STI strain analysis) and subsequent calculations were not part of the standard training of the respiratory ultrasound practitioners who typically manage patients with NEL. We present data on a widely available and easily implementable method of measuring lung movement which has shown good discrimination between expansive lung and NEL, and which does not require any further specific training for either experienced professionals or US operators with basic 2D and M-mode competency in TUS.
There are no gold-standard, or even readily agreed-upon, diagnostic criteria for NEL. Clinical symptoms such as excessive coughing, increasing dyspnoea, and general discomfort due to reduction in intrathoracic pressure are all well recognized symptoms during thoracentesis, and may be suggestive of NEL. Radiological signs such as thickened visceral pleura and (hydro)-pneumothorax are most readily distinguishable on post-thoracentesis imaging. They are strongly suggestive of NEL and are usually what is relied upon in the clinical setting. These findings are well established, but are obtained after or during thoracentesis. This presents a potential risk of delaying or overlooking NEL. Pre-existing knowledge of NEL has the potential to alter both procedural approaches and treatment options for those with effusions. Several studies [10,41,42] have suggested pleural manometry as a feasible and viable standard for diagnosing NEL. The standardization of both methods and cut-offs has been suggested, although these vary according to the study, and as of yet, guidelines have not proposed pleural manometry as the reference standard nor shown the ability to prevent pleural-pressure-related complications during thoracentesis [13,18,43]. Furthermore, study data on diagnostic applicability have been sparse and not up to current reporting standards. To our knowledge, these data have only been presented in the studies by Lan et al. [7] (n = 65) and Halford et al. [10] (n = 113).
Our finding of 14/49 (28%) patients with suspected or known MPE being diagnosed with NEL is consistent with the results of other studies [5,35]. It has been suggested that many suspected iatrogenic pneumothoraxes seen after thoracentesis are, in fact, “pneumothorax ex vacuo” in the context of NEL, the process of which is believed to be caused by the drop in pleural pressure, resulting in small, transient tears or pleural-pressure-dependant fistulas in the visceral pleura with subsequent accumulation of air into the pleural space to allow for pressure equalization [1,44]. Being able to identify these patients up front will also allow for procedure avoidance, as attempts to conduct prolonged pneumothorax and suction treatment in an effort to fully expand the lung may be avoided. By utilizing the cut-off values suggested in Table A3 for lung and diaphragm movement, patients who are candidates for indwelling pleural catheters can be identified and planned more easily, and care can be taken for the patients at risk of thoracentesis-related symptoms.
In our study, measurements were performed at the “site of planned thoracentesis”. Although TUS appearances which could have more readily identified areas of suspected NEL may, therefore, have been missed, we believe that this approach is more typical of usual clinical practice and, thus, allows for greater external validity.
The ultrasound measurements utilized by M-mode and B-mode were simple to perform, ensuring reproducible and externally valid results. Measurements and cut-points were presented down to 1/100 cm, which intuitively presented a degree of uncertainty of the reproducibility for clinicians, and we propose that a validation cohort present cut-points at 1/10 cm.
SWE did not result in superior diagnostic ability of NEL in our study. Obtaining usable ROI was reasonable easy. We observed that there was a high degree of measurement unreliability within the same ROI, as exemplified in Figure 1. Even though care was taken to conform to attaining optimal imaging, the patient population was frail, and, for example, breath holding proved difficult.
There are a number of limitations to our study, including having a single centre and a relatively small number of patients included. The study was not designed with a power calculation to explore the number of participants needed. This was due to the unknown nature of the diagnostic accuracy of the tests performed and the uncertainty as to the prevalence of NEL in the study population. The lack of operator reliability measures also limits the validity. However, with the COVID-19 restrictions at the time of the study, the limited personnel availability, and a frail patient group, it was decided to limit the number of health care professionals to one physician and one nurse in the procedure room per exam. Studies on more advanced ultrasound measures have shown high test–retest and interrater reliability in comparable studies [20,22], with similar small measurement sizes in mm being distinguishable. Close collaboration between KF and JKP throughout the study aimed to ensure that procedures were performed as uniformly as possible. SWE also comes with another limitation: the increased price of purchasing and limited availability of some ultrasound machines, especially lower- and mid-grade machines.

5. Conclusions

Our study is the first to investigate and show that thoracic ultrasound—in the hands of chest physicians—of respiratory lung movement is useful in the diagnosis of NEL. M-mode is a basic ultrasound feature; it showed promising diagnostic ability in ruling out NEL and was superior to both B-mode and the more complex and resource-demanding shear wave elastography.
Our study suggests that M-mode should be a basic thoracic ultrasound skill in the hands of respiratory physicians involved in the management of pleural diseases.

Author Contributions

Conceptualization, J.K.P., K.F., P.F.C., C.B.L., R.B. and U.B.; methodology, J.K.P., K.F., C.B.L., R.B. and U.B.; software, J.K.P., D.B.R. and O.G.; validation, D.B.R. and O.G.; formal analysis, J.K.P. and D.B.R.; investigation, J.K.P., K.F., J.S.S. and A.H.; resources, J.K.P., K.F. and G.A.; data curation, J.K.P., G.A., J.S.S. and A.H.; writing—original draft preparation, J.K.P.; writing—review and editing, K.F., G.A., P.S., P.F.C., C.B.L., R.B., O.G. and U.B.; visualization, J.K.P.; supervision, R.B., C.B.L. and U.B.; project administration, U.B.; funding acquisition, U.B. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki. All patients provided informed consent. The study was approved by the Regional Ethics Committee (approval nr. SJ-891) and Region Zealand Data Committee (REG-001-2021).

Informed Consent Statement

All patients provided informed consent.

Data Availability Statement

All data from the study are presented in this manuscript. For questions concerning data, please contact the corresponding author.

Acknowledgments

A humble thank you to all participating patients and clinical staff of the Pleural Clinic at Naestved Hospital.

Conflicts of Interest

The authors declare no conflicts of interest.

Appendix A

Table A1. Ultrasound test modalities with Youden index optimized and pre-determined cut-offs for non-expandable lung (NEL).
Table A1. Ultrasound test modalities with Youden index optimized and pre-determined cut-offs for non-expandable lung (NEL).
Test ModalityCut-Point (cm)AUCSensitivitySpecificityPPVNPV
M-mode Lung movement
Youden index0.750.750.780.710.520.89
Min 0.80 specificity1.000.700.500.860.590.81
Min 0.80 sensitivity0.700.720.810.640.470.89
Min 0.75 specificity1.000.700.500.860.590.81
Min 0.75 sensitivity0.760.710.780.710.520.89
M-mode Diaphragm movement
Youden index0.820.740.760.710.510.88
Min 0.80 specificity1.390.700.540.860.610.82
Min 0.80 sensitivity0.560.660.880.430.380.90
Min 0.75 specificity1.040.720.680.790.560.86
Min 0.75 sensitivity0.830.720.760.710.510.88
B-mode Diaphragm movement
Youden index0.790.670.630.710.460.83
Min 0.80 specificity1.370.670.260.860.430.74
Min 0.80 sensitivity0.430.740.800.500.390.86
Min 0.75 specificity1.120.700.400.790.430.77
Min 0.75 sensitivity0.470.710.770.500.380.84
Table A2. Logistic regression of optimized outcome modelling.
Table A2. Logistic regression of optimized outcome modelling.
Test ModalityUnivariate ModelMultivariate Model
Test ModalityOR95% CIp-ValueOR95% CIp-Value
M-mode
Lung movement21.962.83–170.330.0037.720.84–70.900.071
Diaphragm movement2.991.13–7.930.0271.560.55–4.450.399
B-mode
Diaphragm movement2.310.71–7.480.163
Lung sliding9.311.78–48.720.0083.460.54–22.300.191
∆ Area1.020.94–1.090.661
Shear Wave Elastography
Parietal pleura1.270.61–2.620.532
Visceral pleura1.130.55–2.310.734
Pleural effusion0.760.35–1.650.492
CI = Confidence Interval.
Table A3. Reference Standard Definitions of non-expandable lung (NEL).
Table A3. Reference Standard Definitions of non-expandable lung (NEL).
The diagnosis is either
Full Expansion (A)
Probable Expansive Lung (B)
Definitely NEL (C)
probable NEL (D)
Entrapped lung scoring on chest X-ray or chest CT scan
A. Definitely free. Complete apposition of parietal and visceral pleura
B. Probably free. Apposition of parietal and visceral pleura in most places but some residual pleural fluid
C. Definitely entrapped. Air separating the visceral and parietal pleura around the lower lobe
D. Probably entrapped. Some air between visceral and parietal pleura in places around the lower lobe but residual pleural fluid obscuring some areas
E. Unable to score. Insufficient drainage of pleural fluid to allow designation in one of the prior categories.
Final NEL status: consensus diagnosis made by two interventional pulmonologists with all data—except the exploratory US-data (USE-SW, B- and M-mode)—available including follow-up four months after first visit.
Table A4. M-mode lung movement. Detailed report of sensitivity and specificity.
Table A4. M-mode lung movement. Detailed report of sensitivity and specificity.
Correctly
Cutpoint (mm)SensitivitySpecificityClassifiedLR+LR−
(≥0.07)100.00%0.00%69.57%1.0000
(≥0.14)100.00%14.29%73.91%1.16670.0000
(≥0.19)96.88%28.57%76.09%1.35620.1094
(≥0.21)96.88%35.71%78.26%1.50690.0875
(≥0.28)96.88%42.86%80.43%1.69530.0729
(≥0.4)96.88%50.00%82.61%1.93750.0625
(≥0.42)93.75%50.00%80.43%1.87500.1250
(≥0.44)90.63%57.14%80.43%2.11460.1641
(≥0.46)87.50%57.14%78.26%2.04170.2188
(≥0.56)84.38%57.14%76.09%1.96870.2734
(≥0.63)84.38%64.29%78.26%2.36250.2431
(≥0.7)81.25%64.29%76.09%2.27500.2917
(≥0.74)78.13%64.29%73.91%2.18750.3403
(≥0.76)78.13%71.43%76.09%2.73440.3062
(≥0.83)68.75%71.43%69.57%2.40630.4375
(≥0.9)62.50%71.43%65.22%2.18750.5250
(≥0.97)59.38%71.43%63.04%2.07810.5687
(≥1)50.00%85.71%60.87%3.50000.5833
(≥1.01)46.88%85.71%58.70%3.28130.6198
(≥1.02)43.75%85.71%56.52%3.06250.6562
(≥1.04)43.75%92.86%58.70%6.12500.6058
(≥1.1)40.63%100.00%58.70% 0.5938
(≥1.11)37.50%100.00%56.52% 0.6250
(≥1.12)34.38%100.00%54.35% 0.6563
(≥1.46)28.13%100.00%50.00% 0.7188
(≥1.53)25.00%100.00%47.83% 0.7500
(≥1.6)21.88%100.00%45.65% 0.7813
(≥1.74)15.63%100.00%41.30% 0.8438
(≥1.94)12.50%100.00%39.13% 0.8750
(≥2.08)9.38%100.00%36.96% 0.9063
(≥2.22)6.25%100.00%34.78% 0.9375
(≥3.89)3.13%100.00%32.61% 0.9688
(>3.89)0.00%100.00%30.43% 1.0000
Figure A1. Calibration belt of M-mode lung movement in predicting non-expandable lung (NEL).
Figure A1. Calibration belt of M-mode lung movement in predicting non-expandable lung (NEL).
Diagnostics 14 00204 g0a1

References

  1. Huggins, J.T.; Maldonado, F.; Chopra, A.; Rahman, N.; Light, R. Unexpandable lung from pleural disease. Respirology 2018, 23, 160–167. [Google Scholar] [CrossRef] [PubMed]
  2. Feller-Kopman, D. Therapeutic thoracentesis: The role of ultrasound and pleural manometry. Curr. Opin. Pulm. Med. 2007, 13, 312–318. [Google Scholar] [PubMed]
  3. Pereyra, M.F.; Ferreiro, L.; Valdés, L. Unexpandable lung. Arch. Bronconeumol. 2013, 49, 63–69. [Google Scholar] [CrossRef] [PubMed]
  4. Kennedy, L.; Rusch, V.W.; Strange, C.; Ginsberg, R.J.; Sahn, S.A. Pleurodesis using talc slurry. Chest 1994, 106, 342–346. [Google Scholar] [PubMed]
  5. Roberts, M.E.; Neville, E.; Berrisford, R.G.; Antunes, G.; Ali, N.J.; on behalf of the BTS Pleural Disease Guideline Group. Management of a malignant pleural effusion: British Thoracic Society pleural disease guideline 2010. Thorax 2010, 65 (Suppl. 2), ii32–ii40. [Google Scholar] [CrossRef] [PubMed]
  6. Hassan, M.; Mercer, R.M.; Rahman, N.M. Thoracic ultrasound in the modern management of pleural disease. Eur. Respir. Rev. 2020, 29, 190136. [Google Scholar] [CrossRef] [PubMed]
  7. Lan, R.S.; Lo, S.K.; Chuang, M.L.; Yang, C.T.; Tsao, T.C.Y.; Lee, C.H. Elastance of the pleural space: A predictor for the outcome of pleurodesis in patients with malignant pleural effusion. Ann. Intern. Med. 1997, 126, 768–774. [Google Scholar] [CrossRef]
  8. Muruganandan, S.; Azzopardi, M.; Fitzgerald, D.B.; Shrestha, R.; Kwan, B.C.H.; Lam, D.C.L.; De Chaneet, C.C.; Ali, M.R.S.R.; Yap, E.; Tobin, C.L.; et al. Aggressive versus symptom-guided drainage of malignant pleural effusion via indwelling pleural catheters (AMPLE-2): An open-label randomised trial. Lancet Respir. Med. 2018, 6, 671–680. [Google Scholar] [CrossRef]
  9. Davies, H.E.; Mishra, E.K.; Kahan, B.C.; Wrightson, J.M.; Stanton, A.E.; Guhan, A.; Davies, C.; Grayez, J.; Harrison, R.; Prasad, A.; et al. Effect of an indwelling pleural catheter vs chest tube and talc pleurodesis for relieving dyspnea in patients with malignant pleural effusion: The TIME2 randomized controlled trial. JAMA J. Am. Med. Assoc. 2012, 307, 2383–2389. [Google Scholar] [CrossRef]
  10. Halford, P.J.; Bhatnagar, R.; White, P.; Haris, M.; Harrison, R.N.; Holme, J.; Sivasothy, P.; West, A.; Bishop, L.J.; Stanton, A.E.; et al. Manometry performed at indwelling pleural catheter insertion to predict unexpandable lung. J. Thorac. Dis. 2020, 12, 1374–1384. [Google Scholar] [CrossRef]
  11. Heidecker, J.; Huggins, J.T.; Sahn, S.A.; Doelken, P. Pathophysiology of pneumothorax following ultrasound-guided thoracentesis. Chest 2006, 130, 1173–1184. [Google Scholar] [CrossRef] [PubMed]
  12. Feller-Kopman, D.; Walkey, A.; Berkowitz, D.; Ernst, A. The relationship of pleural pressure to symptom development during therapeutic thoracentesis. Chest 2006, 129, 1556–1560. [Google Scholar] [CrossRef] [PubMed]
  13. Feller-Kopman, D.J.; Reddy, C.B.; DeCamp, M.M.; Diekemper, R.L.; Gould, M.K.; Henry, T.; Iyer, N.P.; Lee, Y.C.G.; Lewis, S.Z.; Maskell, N.A.; et al. Management of malignant pleural effusions: An official ATS/STS/STR clinical practice guideline. Am. J. Respir. Crit. Care Med. 2018, 198, 839–849. [Google Scholar]
  14. Thomas, R.; Fysh, E.T.H.; Smith, N.A.; Lee, P.; Kwan, B.C.H.; Yap, E.; Horwood, F.; Piccolo, F.; Lam, D.; Garske, L.; et al. Effect of an indwelling pleural catheter vs talc pleurodesis on hospitalization days in patients with malignant pleural effusion: The AMPLE randomized clinical trial. JAMA J. Am. Med. Assoc. 2017, 318, 1903–1912. [Google Scholar] [CrossRef] [PubMed]
  15. Lester, M.; Maldonado, F.; Rickman, O.B.; Roller, L.J.; Avasarala, S.K.; Katsis, J.M.; Lentz, R.J. Association between terminal pleural elastance and radiographic lung re-expansion after therapeutic thoracentesis in patients with symptomatic pleural effusion: A post-hoc analysis of a randomised trial. BMJ Open 2022, 12, e053606. [Google Scholar] [CrossRef]
  16. Chopra, A.; Judson, M.A.; Doelken, P.; Maldonado, F.; Rahman, N.M.; Huggins, J.T. The Relationship of Pleural Manometry with Postthoracentesis Chest Radiographic Findings in Malignant Pleural Effusion. Chest 2020, 157, 421–426. [Google Scholar]
  17. Havelock, T.; Teoh, R.; Laws, D.; Gleeson, F. Pleural procedures and thoracic ultrasound: British Thoracic Society pleural disease guideline 2010. Thorax 2010, 65 (Suppl. 2), ii61–ii76. [Google Scholar] [CrossRef]
  18. Bibby, A.C.; Dorn, P.; Psallidas, I.; Porcel, J.M.; Janssen, J.; Froudarakis, M.; Subotic, D.; Astoul, P.; Licht, P.; Schmid, R.; et al. ERS/EACTS statement on the management of malignant pleural effusions. Eur. J. Cardio-Thoracic. Surg. 2019, 55, 116–132. [Google Scholar] [CrossRef]
  19. Flora, B.; Ahmad, S. Ultrasound as a noninvasive tool to diagnose trapped lung. Am. J. Respir. Crit. Care Med. 2017, 195, A6518. [Google Scholar]
  20. Salamonsen, M.R.; Lo, A.K.C.; Ng, A.C.T.; Bashirzadeh, F.; Wang, W.Y.S.; Fielding, D.I.K. Novel use of pleural ultrasound can identify malignant entrapped lung prior to effusion drainage. Chest 2014, 146, 1286–1293. [Google Scholar] [CrossRef]
  21. Aguilera Garcia, Y.; Palkar, A.; Koenig, S.J.; Narasimhan, M.; Mayo, P.H. Assessment of Diaphragm Function and Pleural Pressures During Thoracentesis. Chest 2020, 157, 205–211. [Google Scholar] [CrossRef] [PubMed]
  22. Testa, A.; Soldati, G.; Giannuzzi, R.; Berardi, S.; Portale, G.; Gentiloni Silveri, N. Ultrasound M-Mode assessment of diaphragmatic kinetics by anterior transverse scanning in healthy subjects. Ultrasound Med. Biol. 2011, 37, 44–52. [Google Scholar] [PubMed]
  23. Boon, A.J.; Harper, C.J.; Ghahfarokhi, L.S.; Strommen, J.A.; Watson, J.C.; Sorenson, E.J. Two-dimensional ultrasound imaging of the diaphragm: Quantitative values in normal subjects. Muscle Nerve 2013, 47, 884–889. [Google Scholar]
  24. Skaarup, S.H.; Løkke, A.; Laursen, C.B. The Area method: A new method for ultrasound assessment of diaphragmatic movement. Crit. Ultrasound J. 2018, 10, 15. [Google Scholar] [CrossRef] [PubMed]
  25. Gennisson, J.L.; Deffieux, T.; Fink, M.; Tanter, M. Ultrasound elastography: Principles and techniques. Diagn. Interv. Imaging 2013, 94, 487–495. [Google Scholar] [CrossRef]
  26. Wen, X.; Yu, X.; Tian, Y.; Liu, Z.; Cheng, W.; Li, H.; Kang, J.; Wei, T.; Yuan, S.; Tian, J. Quantitative shear wave elastography in primary invasive breast cancers, based on collagen-S100A4 pathology, indicates axillary lymph node metastasis. Quant. Imaging Med. Surg. 2020, 10, 624–633. [Google Scholar] [CrossRef]
  27. Ozgokce, M.; Yavuz, A.; Akbudak, I.; Durmaz, F.; Uney, I.; Aydin, Y.; Yildiz, H.; Batur, A.; Arslan, H.; Dundar, I. Usability of Transthoracic Shear Wave Elastography in Differentiation of Subpleural Solid Masses. Ultrasound Q. 2018, 34, 233–237. [Google Scholar] [CrossRef]
  28. Yue, C.; Jiang, Y.; Li, P.; Wang, Y.; Xue, J.; Li, N.; Li, D.; Wang, R.; Dang, Y.; Hu, Z.; et al. Dynamic change of PD-L1 expression on circulating tumor cells in advanced solid tumor patients undergoing PD-1 blockade therapy. Oncoimmunology 2018, 7, e1438111. [Google Scholar] [CrossRef]
  29. Jiang, B.; Li, X.-L.; Yin, Y.; Zhang, Q.; Zang, T.; Song, W.-S.; Wang, X.-M.; Kang, J.; Herth, F.J.; Hou, G. Ultrasound elastography: A novel tool for the differential diagnosis of pleural effusion. Eur. Respir. J. 2019, 54, 1802018. [Google Scholar] [CrossRef]
  30. Porcel, J.M. Ultrasound-based elastography: “hard” to implement in the pleural effusion work-up? Eur. Respir. J. 2019, 54, 1901587. [Google Scholar] [CrossRef]
  31. Kollmann, C.; Jenderka, K.V.; Moran, C.M.; Draghi, F.; Jimenez Diaz, J.F.; Sande, R. EFSUMB clinical safety statement for diagnostic ultrasound—(2019 revision). Ultraschall der Medizin 2020, 41, 387–389. [Google Scholar]
  32. Gogtay, N.J.; Thatte, U.M. Statistical evaluation of diagnostic tests—Part 2 Pre-test and post-test probability and odds, likelihood ratios, receiver operating characteristic curve, youden’s index and diagnostic test biases. J. Assoc. Physicians India 2017, 65, 86–91. [Google Scholar]
  33. Nattino, G.; Finazzi, S.; Bertolini, G. A new calibration test and a reappraisal of the calibration belt for the assessment of prediction models based on dichotomous outcomes. Stat. Med. 2014, 33, 2390–2407. [Google Scholar] [PubMed]
  34. Nattino, G.; Lemeshow, S.; Phillips, G.; Finazzi, S.; Bertolini, G. Assessing the calibration of dichotomous outcome models with the calibration belt. Stata J. 2017, 17, 1003–1014. [Google Scholar] [CrossRef]
  35. Dresler, C.M.; Olak, J.; Herndon, J.E.; Richards, W.G.; Scalzetti, E.; Fleishman, S.B.; Kernstine, K.H.; Demmy, T.; Jablons, D.M.; Kohman, L.; et al. Phase III Intergroup Study of Talc Poudrage vs Talc Slurry Sclerosis for Malignant Pleural Effusion. Chest 2005, 127, 909–915. [Google Scholar] [PubMed]
  36. Psallidas, I.; Kalomenidis, I.; Porcel, J.M.; Robinson, B.W.; Stathopoulos, G.T. Malignant pleural effusion: From bench to bedside. Eur. Respir. Rev. 2016, 25, 189–198. [Google Scholar] [CrossRef]
  37. Mummadi, S.R.; Stoller, J.K.; Lopez, R.; Kailasam, K.; Gillespie, C.T.; Hahn, P.Y. Epidemiology of Adult Pleural Disease in the United States. Chest 2021, 160, 1534–1551. [Google Scholar] [CrossRef]
  38. Walker, S.; Bibby, A.C.; Maskell, N.A. Current best practice in the evaluation and management of malignant pleural effusions. Ther. Adv. Respir. Dis. 2017, 11, 105–114. [Google Scholar] [CrossRef]
  39. Bhatnagar, R.; Corcoran, J.P.; Maldonado, F.; Feller-Kopman, D.; Janssen, J.; Astoul, P.; Rahman, N.M. Advanced medical interventions in pleural disease. Eur. Respir. Rev. 2016, 25, 199–213. [Google Scholar] [CrossRef]
  40. Sigrist, R.M.S.; Liau, J.; El Kaffas, A.; Chammas, M.C.; Willmann, J.K. Ultrasound elastography: Review of techniques and clinical applications. Theranostics 2017, 7, 1303–1329. [Google Scholar]
  41. Huggins, J.T.; Sahn, S.A.; Heidecker, J.; Ravenel, J.G.; Doelken, P. Characteristics of trapped lung: Pleural fluid analysis, manometry, and air-contrast chest CT. Chest 2007, 131, 206–213. [Google Scholar] [CrossRef] [PubMed]
  42. Doelken, P. Pleural Manometry. In Principles and Practice of Interventional Pulmonology [Internet]; Springer: New York, NY, USA, 2013; pp. 571–575. [Google Scholar] [CrossRef]
  43. Lentz, R.J.; Lerner, A.D.; Pannu, J.K.; Merrick, C.M.; Roller, L.; Walston, C.; Valenti, S.; Goddard, T.; Chen, H.; Huggins, J.T.; et al. Routine monitoring with pleural manometry during therapeutic large-volume thoracentesis to prevent pleural-pressure-related complications: A multicentre, single-blind randomised controlled trial. Lancet Respir. Med. 2019, 7, 447–455. [Google Scholar] [CrossRef] [PubMed]
  44. Chopra, A.; Doelken, P.; Hu, K.; Huggins, J.T.; Judson, M.A. Pressure-Dependent Pneumothorax and Air Leak: Physiology and Clinical Implications. Chest 2023, 164, 796–805. [Google Scholar] [CrossRef] [PubMed]
Figure 1. Ultrasound examinations. (A) 2D shear wave elastography (SWE) of visceral pleura. (B) SWE of parietal pleura with 5 readings recorded in the lower left corner. (C) M-mode measurement of lung movement. The respiratory pattern is seen in the curves on the bottom of the screen. (D) M-mode measurement of lung movement. Here, absence of lung movement is illustrated by the flat line below the 2-D image. (E) M-mode measurement of diaphragm movement. (F) Area-method measurement of area above the diaphragm during inspiration.
Figure 1. Ultrasound examinations. (A) 2D shear wave elastography (SWE) of visceral pleura. (B) SWE of parietal pleura with 5 readings recorded in the lower left corner. (C) M-mode measurement of lung movement. The respiratory pattern is seen in the curves on the bottom of the screen. (D) M-mode measurement of lung movement. Here, absence of lung movement is illustrated by the flat line below the 2-D image. (E) M-mode measurement of diaphragm movement. (F) Area-method measurement of area above the diaphragm during inspiration.
Diagnostics 14 00204 g001
Figure 2. Radiological images of non-expandable lung (NEL). CXR (image (A,C,E)) and CT-scan (image (B,D,F)). (A) NEL. Lower lobe with thickened visceral lining. (B) NEL. Obstructive tumour of the lower lobe. (C) NEL. Hydropneumothorax following thoracentesis. (D) NEL with thickened pleura. (E) NEL. Small hydropneumothorax following thoracentesis. (F) NEL. Pleural effusion following thoracentesis. Fluid has replaced initial air in the pleural space seen on CXR after thoracentesis.
Figure 2. Radiological images of non-expandable lung (NEL). CXR (image (A,C,E)) and CT-scan (image (B,D,F)). (A) NEL. Lower lobe with thickened visceral lining. (B) NEL. Obstructive tumour of the lower lobe. (C) NEL. Hydropneumothorax following thoracentesis. (D) NEL with thickened pleura. (E) NEL. Small hydropneumothorax following thoracentesis. (F) NEL. Pleural effusion following thoracentesis. Fluid has replaced initial air in the pleural space seen on CXR after thoracentesis.
Diagnostics 14 00204 g002
Figure 3. Best-performing ROC curves. (SWE Shear Wave Elastography).
Figure 3. Best-performing ROC curves. (SWE Shear Wave Elastography).
Diagnostics 14 00204 g003
Table 1. Demographics.
Table 1. Demographics.
VariableExpandable LungNon-Expandable Lungp-Value
n = 35n = 14
Age71 (63–76)76 (68–80)0.071 #
Sex (female)18 (51%)6 (43%)0.59 ¤
Smoking status 0.40 ¤
Current/Former8 (23%)/19 (54%)3 (21%)/10 (71%)
Never8 (23%)1 (7%)
Accumulated smoking pack years34 (15–43)28 (18–40)0.81 #
Known malignancy31 (89%)13 (93%)0.65 ¤
Pleural fluid volume drained (mL)1290 (604)888 (620)0.049 ∩
Data are presented as mean (SD) or median (IQR) for continuous measures, and n (%) for categorical measures. #: Wilcoxon rank-sum test. ¤: Chi-square test. ∩: Two-sample t test.
Table 2. Ultrasound test measurements with ROC-curve characteristics.
Table 2. Ultrasound test measurements with ROC-curve characteristics.
Test ModalityMedian (IQR)AUC95% CI
M-mode
Lung movement (cm)0.94 (0.44–1.11)0.810.68–0.95
Diaphragm movement (cm)1.21 (0.56–2.01)0.770.61–0.93
B-mode
Diaphragm movement (cm)0.86 (0.41–1.27)0.650.46–0.84
Area method—delta (cm2)11.50 (6.69–15.95)0.600.40–0.79
Lung sliding *24 (50%)0.740.61–0.87
Shear Wave Elastography
Visceral pleura (m/s)1.64 (1.47–2.08)0.590.40–0.79
Pleural effusion (m/s)1.64 (1.34–2.06)0.530.34–0.71
Parietal pleura (m/s)2.54 (2.03–3.09)0.570.39–0.75
* n (%) for categorical measures.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Petersen, J.K.; Fjaellegaard, K.; Rasmussen, D.B.; Alstrup, G.; Høegholm, A.; Sidhu, J.S.; Sivapalan, P.; Gerke, O.; Bhatnagar, R.; Clementsen, P.F.; et al. Ultrasound in the Diagnosis of Non-Expandable Lung: A Prospective Observational Study of M-Mode, B-Mode, and 2D-Shear Wave Elastography. Diagnostics 2024, 14, 204. https://doi.org/10.3390/diagnostics14020204

AMA Style

Petersen JK, Fjaellegaard K, Rasmussen DB, Alstrup G, Høegholm A, Sidhu JS, Sivapalan P, Gerke O, Bhatnagar R, Clementsen PF, et al. Ultrasound in the Diagnosis of Non-Expandable Lung: A Prospective Observational Study of M-Mode, B-Mode, and 2D-Shear Wave Elastography. Diagnostics. 2024; 14(2):204. https://doi.org/10.3390/diagnostics14020204

Chicago/Turabian Style

Petersen, Jesper Koefod, Katrine Fjaellegaard, Daniel B. Rasmussen, Gitte Alstrup, Asbjørn Høegholm, Jatinder Singh Sidhu, Pradeesh Sivapalan, Oke Gerke, Rahul Bhatnagar, Paul Frost Clementsen, and et al. 2024. "Ultrasound in the Diagnosis of Non-Expandable Lung: A Prospective Observational Study of M-Mode, B-Mode, and 2D-Shear Wave Elastography" Diagnostics 14, no. 2: 204. https://doi.org/10.3390/diagnostics14020204

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop