Survey PaperRespiratory motion models: A review
Graphical abstract
Highlights
► Respiratory motion models try to model the breathing motion of a region of interest. ► Applied for motion correction in image acquisition or image guided interventions. ► Model formation typically based on imaging data. ► Model application can use direct or indirect correspondence models. ► This paper reviews applications and methodological techniques proposed to date.
Introduction
Advances in imaging technology in recent decades have opened up an increasingly wide range of potential applications for medical images, including diagnosis, treatment planning and image-guided interventions. However, in the thorax and abdomen the problem of organ motion caused by respiration remains a limiting factor. In image acquisition it can cause artefacts in the acquired images (Nehmeh and Erdi, 2008, Scott et al., 2009), thus limiting their practical utility; whereas in image-guided interventions it can cause a misalignment between the static guidance information and the moving anatomy (Hawkes et al., 2005), limiting the accuracy of the guidance.
A number of solutions have been proposed to deal with the problem of respiratory motion. The simplest approach is breath-holding, but this limits acquisition/intervention time to typically less than 30 s, which is inadequate for many situations. Respiratory gating involves only acquiring/using imaging data during a limited window (e.g. end-expiration), based on a simple respiratory signal. However, this significantly increases acquisition/intervention time. An alternative solution is motion tracking. As it can be difficult to image the motion of interest directly during the procedure, markers are often implanted into the region of interest and tracked using an imaging device such as X-ray (Shirato et al., 2000, e.g.). In this case the implantation can be invasive and motion information is only available at the marker(s) and not for the whole region of interest.
Because of the limitations and drawbacks of these techniques, over the past 15 years there has been significant interest in the development of models that can estimate and correct for the effects of respiratory motion. Such models attempt to model the relationship between the motion of interest, i.e. the motion of the internal organ(s), and some ‘surrogate’ data, e.g. the displacement of the skin surface. This relationship is used to estimate the motion based on the subsequent acquisition of the surrogate data. A wide range of different techniques have been proposed for respiratory motion modelling. This paper reviews progress made and attempts to summarise the current state of the art with a view to informing the direction of future research.
Before reviewing the algorithmic components and techniques involved in forming respiratory motion models, it is first necessary to clearly define what is meant by such a model. The term ‘motion model’ has been used in the literature to refer to a number of different concepts, including a series of geometric transformations to different respiratory positions, both with (Rit et al., 2009) and without (Li et al., 2006b, Rohlfing et al., 2004, Zhu et al., 2010) interpolation between them.
In this paper we follow the majority of the literature and use the following definition: a motion model refers to a process that takes some surrogate data as input and produces a motion estimate as output. Motion models are used when it is not possible or practical to directly measure the actual motion of interest with sufficient temporal resolution during the intended procedure (e.g. image acquisition or an image-guided intervention). If the motion can be directly measured then a motion model is not required as motion tracking can be used. When a motion model is used, measurements are made of some surrogate data instead of measuring the motion of interest directly. The surrogate data should be easily measurable and have a strong relationship with the motion of interest. If this relationship can be modelled then the motion of interest can be estimated from the surrogate data. Examples of different sources of surrogate data used in the literature are given in Section 3.
Typically the motion model is based on motion measurements made from imaging data. When forming the model, the surrogate data is normally acquired at the same time as the imaging data (or can be easily derived from the imaging data). The model then approximates the relationship between the surrogate data and the motion of interest. To apply the model only the surrogate data needs to be acquired, and the model estimates the motion from the current surrogate data. This is illustrated in Fig. 1, Fig. 2. The motion model should be capable of making a motion estimate for any value of the surrogate data (although often within a defined range).
Therefore, according to this definition Li et al., 2006b, Rohlfing et al., 2004, Zhu et al., 2010 are not classified as motion models, as they only make motion estimates at a number of discrete respiratory positions; whereas Rit et al. (2009) is classified as a motion model, as the proposed technique is capable of interpolating a motion estimate between these discrete positions.
The goal of the motion model is to approximate the relationship between the surrogate data and the estimated motion by establishing a correspondence model. This correspondence can be ‘direct’ or ‘indirect’.
For a direct correspondence the model estimates the motion as a direct function of the surrogate data as illustrated in Fig. 2a (e.g. King et al., 2009a, Manke et al., 2003). Formally, we can write,where s is the surrogate data, ϕ the direct correspondence model and M the estimate of the motion (i.e. a vector of motion parameter estimates). In this case the number of degrees of freedom of the model is determined by the number and nature of the surrogate data, s. The surrogate values directly parameterise the motion estimates and determine what type of motion can be estimated.
An indirect correspondence model parameterises the motion using a number of internal variables which define the degrees of freedom of the motion model (see Fig. 2b). These variables can have a physiological interpretation, e.g. position in the respiratory cycle (Blackall et al., 2005), or can be a more abstract parameterisation of the motion, e.g. the weights of a statistical model built using principal component analysis (PCA) (King et al., 2012). When the motion model is used to estimate the motion the internal variables are not directly measured. Rather, the surrogate data is a subset of, or can be derived from, the motion estimates made by the model. To apply the motion model the internal variables are optimised to find the best match between the measured surrogate data and the estimates of the surrogate data made by the motion model. Techniques based on indirect correspondence models have sometimes been referred to as ‘image-driven’ approaches in the literature because, to date, they have always used images as the surrogate data, although this need not necessarily be the case.
Formally, we can write,wherein which x is the vector of internal variables, ϕ(x) is a vector of motion parameters estimated from the internal variables, I is a reference image, T is a function that transforms the reference image according to the motion parameters, F is a function which simulates the surrogate data from the transformed reference image, and Sim is a measure of similarity between the simulated surrogate data and the measured surrogate data, s. The function F can vary: it can select a subset of the transformed reference image data corresponding to the surrogate data (King et al., 2008b, King et al., 2010b, Peressutti et al., 2012), or it can simulate the surrogate imaging modality if this is different to the modality used to acquire the reference volume (Blackall et al., 2005, King et al., 2001, King et al., 2010c, Li et al., 2011a, Vandemeulebroucke et al., 2009), e.g. simulating an ultrasound (US) signal from a magnetic resonance (MR) volume (Blackall et al., 2005). The internal variables are optimised to find the values, , that produce the best value of Sim. →doR2.2 The final motion estimate, M, is produced by applying the model ϕ using the values .
This paper will review work carried out to model the respiratory motion of any organ affected by breathing. Predominantly, this means the lungs, the heart and the liver, although work has also been performed to model the respiratory motion of other thoracic organs (see Section 2.1 for a more detailed discussion). In the heart there is the additional problem of motion due to the beating of the heart (which we term cardiac cycle motion). This motion is also approximately repeatable and there are a number of examples of models of cardiac cycle motion in the literature (e.g. Huang et al., 1999). Such models often employ similar modelling techniques to respiratory motion models. Nevertheless, for reasons of clarity and brevity we do not discuss such work in this review. There are also a small number of examples of joint models of both respiratory and cardiac cycle motion (e.g. Odille et al., 2008a, Odille et al., 2010, Shechter et al., 2006). Since these works model breathing motion and are small in number we do include such papers in this review.
Due to some inconsistency in the use of terminology in the literature we wish to clarify the use of the terms motion ‘estimation’ and ‘prediction’. The term ‘prediction’ has been commonly used to refer to estimating the value of a future signal value based on current and/or past values, for example using a technique such as a Kalman filter (Kalman, 1960). Good reviews of such prediction techniques can be found in Ernst and Schweikard, 2009, Verma et al., 2011. However, the term ‘prediction’ has also been used to refer to estimating current values (e.g. motion fields) based on some other simple signals (e.g. surrogate values) (Ahn et al., 2004, Cervino et al., 2009, Cervino et al., 2010, Ehrhardt et al., 2010). Whilst we do not disagree with such use, for the purpose of this review it is necessary to have some clarity in definitions. Therefore, in this paper we use the term motion ‘estimation’ to refer to the estimates of current motion made by the type of motion model described above, and ‘prediction’ to refer to estimating future values of a signal. The subject of this review is models for motion estimation, not motion prediction. However, it should be noted that prediction and estimation can be performed simultaneously. For example, in Isaksson et al. (2005) a motion model was described that could predict future motion estimates based on current surrogate data. This type of approach can be useful, for example, in overcoming latency in motion compensation systems.
Respiratory motion is often assumed to be, at least approximately, the same from cycle to cycle. To a large extent, this assumption is valid, but there are certain variations in breathing motion that should be discussed and defined with relation to the physiology literature. There are two physiological causes of respiration: contraction of the thoracic diaphragm muscle and movement of the rib cage caused by the rib cage muscles (primarily the internal and external intercostals) (West, 2004). These combined effects cause an increase in intrathoracic volume and a consequent inhalation of air into the lungs (De Troyer and Estenne, 1984, West, 2004). The relative contributions of these two causes can vary from breathing cycle to breathing cycle, and can differ greatly depending on the subject’s pose (e.g. supine/upright) (De Troyer and Estenne, 1984, Sharp et al., 1975) and breathing pattern (e.g. deep/shallow) (Sharp et al., 1975). Significant variation also exists between individuals (Konno and Mead, 1967). The existence of these two underlying causes of respiration and their variability means that the motion of organs due to respiration is not perfectly repeatable (Benchetrit, 2000): changes in the relative contributions and their magnitudes cause breathing motion to be slightly different during each breathing cycle. This fact has been confirmed by a number of empirical studies of breathing variation based on imaging data (Blackall et al., 2006, Hughes et al., 2008, McClelland et al., 2011).
Based on this, we now define a number of key terms regarding breathing variation:
- •
Intra-cycle variation: This refers to variation of motion within a single breathing cycle, i.e. the motion path followed during inspiration is different to that followed during expiration. This is often referred to as hysteresis in the literature;
- •
Inter-cycle variation: This refers to variation of motion between breathing cycles, i.e. the motion path followed during one breathing cycle is different to that followed during another breathing cycle.
In addition, in applications such as radiotherapy (RT), the variation in motion within and between treatment fractions is of interest (Sonke et al., 2008):
- •
Intra-fraction variation: This refers to variation of motion within a single fraction, or treatment session;
- •
Inter-fraction variation: This refers to variation of motion between fractions, potentially over a period of days or weeks.
These two concepts are related to that of inter-cycle variation since they refer to variation in motion between cycles. However, the two types of variation need to be addressed in different ways. Intra-fraction variation can potentially be included in the model and estimated from the surrogate data. Estimating inter-fraction variation from surrogate data is prone to error as it is often difficult and sometimes impossible to know how the surrogate data from one fraction corresponds to the surrogate data from another fraction (McClelland et al., 2011).
The remainder of this paper is structured as follows. In Section 2 we review the range of applications and organs for which respiratory motion models have been proposed. In Section 3 techniques for acquiring surrogate data to act as inputs to motion models are detailed. Section 4 describes methods for acquiring motion information and reviews imaging modalities that have been used as sources of such information. In Section 5 we categorise and describe the different modelling approaches that have been employed. Finally, Section 6 discusses the current state of the art and offers some speculation about fruitful future directions.
Section snippets
Uses of motion models
Respiratory motion models, as defined in the previous section, have been proposed for use in a wide range of clinical applications and for several different anatomical regions in the thorax/abdomen. In this section we review these applications and anatomical regions.
Acquiring input surrogate data
In this section the different ways of making physical measurements to be used as surrogate data for respiratory motion models are reviewed. The main requirements for such measurements are that they have a strong relationship with the true motion that the model is intended to estimate, and that they can be acquired relatively easily and with sufficiently high temporal resolution. The surrogate data can be simple scalar values (i.e. a 1-D signal over time (Hoisak et al., 2004, Manke et al., 2002a
Acquiring motion data
This section summarises the different sources of motion measurements that are used to form the model. Normally this entails the use of some imaging data, although there has been at least one paper that has proposed measuring motion data by means of electromagnetic tracking (Timinger et al., 2004) rather than imaging.
It is important to note the distinction between the data used for forming the motion model and that used to apply it. Data used to apply the motion model (i.e. surrogate data) was
Models
There are four components that make up a motion model:
- •
Choice of surrogate data, i.e. s in Eqs. (1), (2), (3): what signals are the input to the model?
- •
Choice of motion representation: what is the internal representation of the motion in the model? E.g. affine transformation coefficients, control point displacements, etc.
- •
Correspondence model: how is the motion representation related to the surrogate signals (i.e. ϕ in Eqs. (1), (2), (3))? How is this relationship parameterised?
- •
Fitting method: how
Summary
In this paper we have presented a review of the state of the art in the field of respiratory motion modelling. In the past 15 years a range of different modelling techniques have been proposed for dealing with the effects of breathing motion in various applications, and much progress has been made. We have attempted to summarise this progress, relate techniques proposed for different application areas and highlight some of the key papers and key concepts involved in developing respiratory motion
Acknowledgements
The authors wish to acknowledge funding by EPSRC programme Grant EP/H046410/1. In addition, thanks are due to Devis Peressutti, James Martin and Christian Buerger for useful comments and feedback on the manuscript. The authors acknowledge financial support from the Department of Health via the National Institute for Health Research (NIHR) Comprehensive Biomedical Research Centre award to Guy’s & St Thomas’ NHS Foundation Trust in partnership with King’s College London and King’s College
References (171)
- et al.
Respiratory motion compensation for 3-D freehand echocardiography
Ultrasound in Medicine and Biology
(2001) - et al.
Correlation between internal fiducial tumor motion and external marker motion for liver tumors imaged with 4D-CT
International Journal of Radiation Oncology Biology Physics
(2007) Breathing pattern in humans: diversity and individuality
Respiration Physiology
(2000)Results of a multi-institution deformable registration accuracy study (MIDRAS)
International Journal of Radiation Oncology Biology Physics
(2010)- et al.
Hierarchical adaptive local affine registration for fast and robust respiratory motion estimation
Medical Image Analysis
(2011) - et al.
Real-time target position estimation using stereoscopic kilovoltage/megavoltage imaging and external respiratory monitoring for dynamic multileaf collimator tracking
International Journal of Radiation Oncology Biology Physics
(2011) - et al.
Quantifying variability in radiation dose due to respiratory-induced tumor motion
Medical Image Analysis
(2011) - et al.
Tissue deformation and shape models in image-guided interventions: a discussion paper
Medical Image Analysis
(2005) - et al.
Correlation of lung tumour motion with external surrogate indicators of respiration
International Journal of Radiation Oncology Biology Physics
(2004) - et al.
Clinical accuracy of the respiratory tumor tracking system of the cyberknife: assessment by analysis of log files
International Journal of Radiation Oncology Biology Physics
(2009)
Determining optical flow
Artificial Intelligence
Assessment of two novel ventilatory surrogates for use in the delivery of gated/tracked radiotherapy for non-small cell lung cancer
Radiotherapy and Oncology
A subject-specific technique for respiratory motion correction in image-guided cardiac catheterisation procedures
Medical Image Analysis
Thoracic respiratory motion estimation from MRI using a statistical model and a 2-D image navigator
Medical Image Analysis
Respiratory motion correction for image-guided cardiac interventions using 3-D echocardiography
Medical Image Analysis
Evaluation of internal lung motion for respiratory-gated radiotherapy using MRI: Part i. Correlating internal lung motion with skin fiducial motion
International Journal of Radiation Oncology Biology Physics
Evaluation of internal lung motion for respiratory-gated radiotherapy using MRI: Part II. Margin reduction of internal target volume
International Journal of Radiation Oncology Biology Physics
Novel breathing motion model for radiotherapy
International Journal of Radiation Oncology Biology Physics
Predictive cardiac motion modelling and correction with partial least squares regression
IEEE Transactions on Medical Imaging
A feasibility study on the prediction of tumour location in the lung from skin motion
British Journal of Radiology
Matrix description of general motion correction applied to multishot images
Magnetic Resonance in Medicine
Residual motion of lung tumours in gated radiotherapy with external respiratory surrogates
Physics in Medicine and Biology
Motion in radiotherapy: particle therapy
Physics in Medicine and Biology
MRI-based measurements of respiratory motion variability and assessment of imaging strategies for radiotherapy planning
Physics in Medicine and Biology
Alignment of sparse freehand 3-D ultrasound with preoperative images of the liver using models of respiratory motion
IEEE Transactions on Medical Imaging
Joint reconstruction of image and motion in gated positron emission tomography
IEEE Transactions on Medical Imaging
Non-rigid motion modeling of the liver from 3D undersampled self-gated golden-radial phase encoded MRI
IEEE Transactions on Medical Imaging
Estimation of the heart respiratory motion with applications for cone beam computed tomography imaging: a simulation study
IEEE Transactions on Information Technology in Biomedicine
Four-dimensional deformable image registration using trajectory modelling
Physics in Medicine and Biology
The diaphragm as an anatomic surrogate for lung tumor motion
Physics in Medicine and Biology
Tumor motion prediction with the diaphragm as a surrogate: a feasibility study
Physics in Medicine and Biology
Relation of external surface to internal tumor motion studied with cine CT
Medical Physics
Real-time tumor tracking using sequential kV imaging combined with respiratory monitoring: a general framework applicable to commonly used IGRT systems
Physics in Medicine and Biology
A monoscopic method for real-time tumour tracking using combined occasional X-ray imaging and continuous respiratory monitoring
Physics in Medicine and Biology
Deformable templates using large deformation kinematics
IEEE Transactions on Image Processing
MR-guided focused ultrasound surgery
Journal of Computer Assisted Tomography
Planning lung radiotherapy using 4D CT data and a motion model
Physics in Medicine and Biology
Prospective navigator correction of image position for coronary MR angiography
Radiology
Coordination between rib cage muscles and diaphragm during quiet breathing in humans
Journal of Applied Physiology
Optical flow based method for improved reconstruction of 4d ct data sets acquired during free breathing
Medical Physics
Statistical modeling of 4D respiratory lung motion using diffeomorphic image registration
IEEE Transactions on Medical Imaging
Correlation between external and internal respiratory motion: a validation study
International Journal of Computer Assisted Radiology and Surgery
Cited by (335)
Deep learning to predict respiratory lung-tissue displacement from multi-phase pulmonary computed tomography images
2024, Biomedical Signal Processing and ControlGaussian Processes for real-time 3D motion and uncertainty estimation during MR-guided radiotherapy
2023, Medical Image AnalysisBreathing-Adapted Imaging Techniques for Rapid 4-Dimensional Lung Tomosynthesis
2023, Advances in Radiation OncologyMotion compensated cone-beam CT reconstruction using an a priori motion model from CT simulation: a pilot study
2024, Physics in Medicine and BiologyRespiratory motion modelling for MR-guided lung cancer radiotherapy: model development and geometric accuracy evaluation
2024, Physics in Medicine and Biology