Identification of Structures in Medical Images 1

The development of automatic systems for medical image processing, which can effectively act as an agent to aid medical diagnosis, is a goal that has been pursued by researchers since the first works on the field of medical image processing in the 80 s. The automation of image analysis tasks can produce very interesting results such as less time spent by specialists, decrease of intra-and inter-observer differences, second opinions to non-specialists and in educational systems. Any automatic system deployed for analysis and visualization of images involves the identification of objects and often the relationships among them. The automatic identification of structures is a research area of image processing that still has great challenges. In tasks that involve primarily activities of calculation, the computer's processing power is incomparably higher than the humans. However, in recognition and analysis tasks, the human brain possesses a strong ability that is not obtained by any computational system. For a given scene, humans possess a unique ability to distinguish the objects that are significant and, among them, those that represent the focus of interest for a particular situation.


Introduction
The development of automatic systems for medical image processing, which can effectively act as an agent to aid medical diagnosis, is a goal that has been pursued by researchers since the first works on the field of medical image processing in the 80´s.The automation of image analysis tasks can produce very interesting results such as less time spent by specialists, decrease of intra-and inter-observer differences, second opinions to non-specialists and in educational systems.Any automatic system deployed for analysis and visualization of images involves the identification of objects and often the relationships among them.The automatic identification of structures is a research area of image processing that still has great challenges.In tasks that involve primarily activities of calculation, the computer's processing power is incomparably higher than the humans.However, in recognition and analysis tasks, the human brain possesses a strong ability that is not obtained by any computational system.For a given scene, humans possess a unique ability to distinguish the objects that are significant and, among them, those that represent the focus of interest for a particular situation.
In the routine of medical image analysis, doctors often deal with images in which the visual perception is not sufficient for identification of some particular structure under study.They need, therefore, conceptual hypotheses so as to perform this task.At each new image analyzed a doctor uses a huge amount of knowledge accumulated over the years of clinical practice.This is a typical feedback and a case base reasoning system, as each new image analyzed gives the doctor more knowledge about a specific problem.A system of medical aid for identification and analysis of structures should also be able to incorporate and use knowledge about the problem domain.

Problem statement
Automatic recognition and identification of structures are fields of Computer Vision.The recognition of structures is a difficult task because many factors influence its computation.
These can include restrictions on permissible forms, the semantics of the context of the scene and the information present in the image itself (Suetens et al, 1992).After recognizing the set of objects in the scene, the next step in the interpretation process is the identification of one or more objects of particular interest in the recognized set.There is a distinction between the concepts of recognition and identification of structures.There are many definitions in the literature for recognition (or discovery) of structures.We use here a simple definition, given by Suetens et al (1992): "The recognition (or detection) of structures is the task of finding and labeling parts of a digital image that correspond to objects in the scene." Once the recognition step has been performed, the next step is to identify one or all instances of a particular structure.The definition of object identification is not precise and, depending on the application, may be mistaken for recognition.In this text, identification is defined by the authors as: The identification of a particular set of structures is to set apart from the recognized structures the one or ones that correspond to the object under study.
The term segmentation is sometimes used by researchers as a synonym to identification as defined above.The canonical definition of segmentation is the subdivision of the image into its components (Gonzales & Woods, 2000;Jain, 1989).Strictly speaking, segmentation corresponds only to the first step of the recognition process.

Automatic identification
The steps involved in the automatic identification of structures are usually divided into different levels.Some authors have proposed a division into two levels: low and high (Ballard & Brown, 1982;Sonka et al, 1998) while others use three processing levels: low, medium and high (Gonzales & Woods, 2000;Niessen, 1997).
The low processing level deals with functions similar to automated human visual reactions that normally require little or no knowledge about the content of the image (Gonzalez & Woods, 2000;Sonka, 1998).It is mainly or almost entirely based on intrinsic image data, without inclusion of a priori information.It includes activities such as the basic process of image formation, filtering for noise reduction and correction of acquisition artifacts.In addition, there are processes to decrease the huge amount of data present in the original image to obtain parametric images describing relevant information.Examples of relevant parameters are local contrast, texture, curvature and movement.At the intermediate processing level the tasks are performed by extraction and characterization of components in the images resulting from the low level process.This stage includes activities such as identification, representation and description of the image information through models.For authors who do not use this classification, these functions are usually defined as low-level processing (Ballard & Brown, 1982;Sonka, 1998).The authors that include the intermediate level point out that the difference to the low level processing is that, based on some restrictions, it is possible to select -or highlight -relevant information to a particular application and formalize this information using models that include some information a priori about the application domain, thus making it easier the subsequent high-level tasks (Niessen, 1997).Moreover, the result of this processing step generates no more pictures, but representation of objects in the picture.The high processing level is based on knowledge, goals and strategies on how to achieve those goals more efficiently.Using knowledge about the image structure, image data is systematically evaluated.Hypotheses are confirmed or rejected by the data.Routines for processing high-level information should be organized to produce rapid searches.The difference between high-level and intermediate level processing is the explicit use of knowledge about the context of the image under analysis.

Automatic identification in medical images
In medical applications, the ultimate goal of automatic image analysis is to obtain not only qualitative results, but also quantitative data for assessing objectively the changes in relation to normal standards.To obtain such results it is necessary first to identify the object of interest and then carefully analyze it.Among the applications for which identification of structures is an important step are: visualization and surgical planning, planning for radiotherapy procedures, identification of tumors, malformations and dysfunctions, classification of organs, tissues and cells, and retrieval of similar cases in image content based databases.
In the case of image volumes, or 3D image data, automatic identification involves a large number of operations starting from the original acquired data: they must be processed so that object slices are formed by applying filters and reconstruction algorithms.Once the slices have been obtained, further transformations in the image are needed in order to make the data more suitable for identification.Such transformations may include processes as edge detection algorithms and structure recognition.Then high level procedures involving the use of a priori knowledge to determine the searched structure are used and, specific analysis of the found structures can be performed The results of methods for identification of structures in medical images can be affected by image artifacts.Images contain noise that can alter the intensity of a pixel such that its classification becomes uncertain.They also have finite pixel size and, for this reason, are subject to partial volume averaging effect where individual pixel volumes contain a mixture of tissue classes so that the intensity of a pixel in the image may not be consistent with any one structure.The intensity level of a single tissue class can vary over the extent of the image, and internal materials (for instance surgical clips) may cause distortion in the imaged organ (Rani et al, 2011;Withey & Koles, 2007).Some of these effects are depicted in Figure 1.All these factors may affect the identification of organs due to poorly defined boundaries, blur or weak edges homogeneity.Moreover, the identification of organs in medical imaging presents some unique challenges as the creation of efficient models for representation of biological structures that are not easily described by mathematical models.All these factors, added to the variability in tissue distribution among individuals in the human population, means that some degree of uncertainty must be attached to all segmentation results.Figure 2 exemplifies the difficulty in identifying brain structures in Computerized Tomography (CT) and Magnetic Resonance (MR) images.Fig. 2. Example challenging identification tasks in brain images.The areas inside the red circles are the areas to be identified.Due to the aforementioned problems, the identification of some tissues in medical images can be difficult even in modalities with high resolution as CT (left image) and MR (middle and right) (images acquired at the Imaging Department -Heart Institute -HCFMUSP-with patient anonymization).

Use of a priori information for identification in medical images
In the context of medical images, the concept of identification, as defined above, involves the discrimination against a particular organ or the discrimination between normality and abnormality for a particular organ, based on the attributes of the image.Although trained clinicians are able to find and classify body structures precisely, there is not a technique that can reproduce precisely their ability in computational environments.The use of simple methods that rely only on information contained in the image data, in general, does not produce satisfactory results.In other words, the intrinsic information of the image is not enough to characterize the structures present in real objects (Suetens et al, 1992).Effective procedures for identifying structures should incorporate knowledge of the model or the context.
There are several studies in the literature aiming the characterization of the knowledge about a particular organ in a medical image modality to aid the automatic processing of the image.Typically, the methods use information about shape, size, texture and position of a given structure.A major difficulty in using a priori information about the forms of the organs is to characterize them geometrically.Unlike the rigid objects from other applications of computer vision, most human organs have few calculable and stable invariants.Apart from that, the features to be detected in images may be small and subtle, sometimes characterized only by tiny changes in gray level.In many medical applications there is a significant amount of known information about the human anatomy.However, there is a major difficulty involving the creation of models to represent biological structures, comprising very complex forms that are not described easily.Furthermore, for a given population these structures usually show variations both in form and size.

A review of applications in medical imaging
Techniques specifically designed for computerized detection of abnormalities in chest radiographs were among the first works in the area of identification of structures in medical images (Duncan & Ayache, 2000).The field has evolved continuously since then and the amount of knowledge has grown constantly with a huge number of methods described in the literature, as pointed out in many survey papers (Bandyopadhyay, 2011;Duncan & Ayache, 2000;Kirbas & Quek, 2002;Lesage et al, 2009;Ma et al, 2008;Oliver et al, 2010;PetitJean & Dacher, 2011;Witney & Koles,2007).
The assumptions and requirements taken by identification methods vary substantially for different body organs and imaging modalities.Therefore, the choice of the method will be dependent on the particular application, regarding the organ or system under study, as well as the imaging modality (Bandyopadhyay, 2011).There have been described general methods that intended to be more comprehensive, and therefore can be applied to a number of applications.However, their performance is worse than that of methods designed for specific applications, as the latter generally use a priori information available about the problem under study (Bandyopadhyay, 2011).
In the next paragraphs we will present some methods proposed in recent years.A general overview of the methods described for medical image identification in the last decades is presented in the first section.The next sections are dedicated to clinical applications with emphasis on the organ under study, presenting the well-known specific challenges for identifying the particular organ and exploring some recently published methods.This text avoids performance comparisons due to fact that existing applications are so different in their purposes and practical applications, that direct comparisons do not make sense.

Evolution of the methods along the last decades
There is a number of papers discussing the evolution of the algorithms for extraction or identification of medical structures.It is out of the scope of this chapter to present and compare all studies and methodologies for classifying the existing methods.Instead, we present a description of the field evolution based on two surveys, presented by Ma et al (2008) and Witney & Koles (2007), which we believe presents the evolution of the field in a simple and informative way.For the other reviews, readers are referred to the literature (Duncan & Ayache, 2002;Petitjean et al, 2011).Withey and Koles (2007) classified the medical image identification approaches in literature into three generations, while Ma et al (2008) classified the methods in three categories that can be fitted to the three generations model of Witney and Koles.
The first generation techniques are those almost based on intrinsic image information, without any a priori information.They include:


Threshold: algorithms belonging to this type assume that the searched structures are related to clearly quantifiable features as image intensity and gradient.


Region growing: from an initial seed located in the image, adjacent pixels are checked against a set of predefined homogeneity criteria.Pixels that meet the criteria are included in the region.


Edge tracing: after the application of an edge detection algorithm, the edge pixels with adjacent neighbor connectivity are followed sequentially and collected into a list to represent an object boundary.
The second generation is composed of algorithms using image models, optimization methods, and uncertainty models, and includes:  Pattern recognition and clustering: as certain structures in medical images can be treated as patterns, segmentation algorithms that combine pattern recognition techniques have been proposed to extract the searched structures.


Deformable models: deformable models use curve evolution to perform segmentation.
A moving equation should be defined to drive the initial curves to the right structure boundaries.


Graph search: image pixels are used to form nodes in a graph and the nodes are interconnected to neighbors, mapping the corresponding pixel associations in the image.Costs are assigned for each interconnection.Algorithms from combinatorial optimization are used to obtain minimum-cost solutions.


Multiresolution methods: multiresolution or multiscale analysis refers to the use of scale reduction to group pixels into image objects.A stack of images is formed by recursively reducing the scale of the original image by blurring followed by down sampling.The central idea in this category is that there may be scales that are more suited for processing some image features.
The third generation is characterized by algorithms that are capable of incorporating knowledge, such as:  Shape and appearance models: the active shape model (ASM) was inspired by deformable models with the added intention of limiting the extent of the model deformation.A statistical representation of an object is formed by identifying a set of landmark points on an object boundary and analyzing the variation of each across a set of training images.


Atlas-based: it is a generic technique for automatic delineation of structures in volumetric images, which starts by registering an anatomical image from an atlas with a target image to be segmented.A critical underlying assumption is that it is possible to find a deformation that aligns the atlas with the target image in such a way that label propagation lines up the objects of interest.


Rule-based: image primitives are usually derived from first-generation and secondgeneration algorithms and then interpreted using anatomical and image knowledge applied as a set of rules.

Brain applications
Environmental factors, age and disease can affect neuroanatomical structures during the process of ageing.These structures are also affected by genetic factors in patients with degenerative diseases (Ashburner et al, 2003).Besides providing anatomical information, medical imaging procedures are able to provide extremely relevant information about brain physiology, which can be used to understand physical and psychological clinical conditions.
In order to capture the extraordinary morphological variability of the human brain, a number of automatic methods for identification and analysis of structures have been developed for different imaging modalities (Ashburner et al, 2003;Bandyopadhyay, 2011;Bresser et al, 2011;Ishii et al, 2009;Lopes et al, 2008;Ribbens et al, 2010;Rousset et al, 2007;Shiee et al, 2010;Tu & Bai, 2010;Yi et al, 2009;Zhang et al, 2011).
Most automatic systems for analysis of brain images have three processing steps: (i) brain tissue identification; (ii) registration: the voxels of interest are matched to a template or to an earlier scan from the same individual; (iii) statistical comparison of different groups of patients or same patient in different times.
In their review of the methods to assess brain structures, Ashburner et al (2003) discuss the strengths and limitations of algorithms for identification and registration of brain images, together with evidence of their usefulness at the clinical and research level.A more recent review of current methods used for computer automated identification in brain anatomical images is presented by Bandyopadhyay (2011).Most methods for extraction of brain structures and tissue segmentation use some kind of a priori information.Some authors, however, try to minimize the need of a priori knowledge.As an example, Scherrer et al (2009) extracted tissue and sub-cortical structures in MR images by considering a local approach to cope with intensity non-uniformity and a multi-agent based implementation.

Cardiovascular applications
Cardiovascular diseases are the leading cause of death in developed countries (Allender et al, 2008).Imaging techniques are nowadays among the most valuable tools in helping diagnosis, treatment and follow-up of cardiovascular pathologies, as they provide qualitative and quantitative information of the heart, which would be impossible to obtain otherwise (Cordero-Grande et al, 2011).Diagnosis and treatment follow-up of these pathologies can rely on numerous cardiac imaging modalities, which include echography, CT (computerized tomography), digital subtractive angiography, coronary angiography and cardiac MRI.
Heart.Quantitative analysis of the heart structure and function can be performed by the extraction of cardiac descriptors from medical images.These descriptors can provide information about global and local function.This includes estimating left ventricle volume, ejection fraction, cardiac output, myocardial mass, myocardial morphology, myocardial uptake rates, tissue viability, blood flow, local motion and deformation (wall thickening).
The determination of most of these descriptors involves an appropriate 3D identification of cardiac structures -being the cardiac chambers, cardiac muscle or cardiac valves -as a prerequisite (Alattar et al, 2010;Cordero-Grande et al, 2011).The imaging modality of choice will depend on the problem under study and the complexity of the identification of the cardiac structures will depend on the modality chosen.
Automatic isolation of the heart structures in MR images is a challenging task because of the inherent noise associated with cine MRI, caused by factors such as patient movement, cardiac dynamics, and complex intensity characteristics (Lynch, 2008).Even in noiseless cases, MR images may present blurred edges due to the partial volume effect and can be affected by motion artifacts.In addition, low contrast between the myocardium and surrounding tissues complicates the identification of the cardiac structures.As an example, the papillary muscles, the trabeculae, or the liver have pixel intensity values which are virtually indistinguishable from those of the myocardium (Cordero-Grande et al, 2011).In the literature there is a huge number of recent works on heart identification in MR images (Alattar et al, 2010;Cordero-Grande et al, 2011;Lekadir et al, 2011;O´Brien et al, 2011;O'Donnell et al, 2006;Rouchdy et al, 2007;Schaerer et al, 2010;Zhao et al, 2009).A comprehensive survey focusing on extracting the heart cavities in short axis view from cardiac MR images has been published recently by Petitjean & Dacher (2011).
The analysis of cardiac conditions using CT images needs a highly consistent identification of the involved surfaces.To measure blood volume, segmentation should consistently follow the cavity border.Wall thickness measurements should be based on a convex hull around the cavity that excludes papillary muscles from the myocardium.For the epicardium, it is important to avoid local confusions with the nearby pericardium or lung transition (Peters et al, 2010).A number of authors have described methods to isolate the cardiac walls and chambers in CT images (Ecabert et al, 2008 Rahman et al (2010).The arteries extraction methods tend to use a strong cylinder assumption.Qian et al (2009) presented a method in which they try to avoid this common strong prior knowledge.The analysis of temporal change in vessel images was addressed by Zhao et al (2009) who described a method to identify subjects with connective tissue disorder in 4D cardiovascular MRI.

Liver applications
Precise measurement of shape and composition of liver is the basis for diagnosis, surgery planning and therapy control of liver pathologies such as cirrhosis, liver cancer, and fulminant hepatic failure (Campadelli et al, 2009).CT images are preferred by the doctors for performing these tasks.Liver identification in CT scans poses problems due to the low contrast and blurred edges that characterize CT images.In their survey of liver extraction in CT scans, Campadelli et al (2009) identified the main applications of the technique as: (1) automatic detection of liver cancer from other liver diseases; (ii) measurement of liver volume -an important index in cases of living donor liver transplantation; (iii) 3D volume rendering of abdominal organs for surgical planning and radiation treatment programs.
Another review of the methods for liver identification in CT images was published by Heimann et al (2009).Recent works in this area have also been described by Ruskó et al (2009), Masuda et al (2010) and Pu et al (2009).Linguraru et al (2009) proposed a method to isolate both the spleen and liver in contrast-enhanced CT images by first aligning the images with models from an atlas and then improving the results by an active contour technique.

Lung applications
Pulmonary nodules are potential manifestations of lung cancer, and their detection and inspection are essential for screening and diagnosis of the disease (Kubota et al, 2011).Once the nodule has been detected, monitoring of its size, density, edge-smoothness and growth rate provide information about what treatment, if any, is appropriate (Murphy et al, 2009).
An extensive review of computer aided diagnosis in chest radiography was published by van Ginneken et al (2001).Automated detection of nodules has been described by Murphy et al (2009) and Kubota et al (2011).Bouma et al (2009) proposed a system for the automatic detection of pulmonary embolism in contrast-enhanced CT images and Pu et al (2011) proposed an automated scheme to extract the airway tree depicted on CT images.

Other applications
Mammography.In mammography, the use of computer-aided diagnosis (CAD) systems is intended to assist radiologists in the automatic detection and classification of mammographic abnormalities.There is a large number of different types of mammographic abnormalities.In the majority of cases the abnormalities are either micro-calcifications or masses (Oliver et al, 2010).Mammography CAD systems usually start by a preprocessing step to extract the breast tissue, background tissue and pectoral muscle (Cheng et al, 2003;Samulski & Karssemeijer, 2010;Timp et al, 2007).Oliver et al (2011) published an extensive review on the theme in which they divide the automatic methods for detection of abnormalities in three groups: (i) Region-based methods: region growing and related methods, watershed methods, split and merge methods; (ii) Contour-based methods; (iii) Clustering methods.
Thyroid.Diseases of the thyroid gland are among the most frequent endocrine disorders.
Ultrasound has become the most important technique for thyroid gland imaging and computerized systems have been described to aid doctors in the task of thyroid image analysis.Hegedüs (2004) and Kollorz et al (2008) presented methods to automatically isolate the thyroid gland in US images, using geodesic active contour level set based approach.
Kidney.In renal applications Oguro et al (2011) described a system to aid CT-guided percutaneous cryoablation of renal tumors.Xie et al (2005) used a priori information about texture and size to segment kidney in ultrasound images.The described method provides the ability to deal with objects with incomplete boundaries by taking into account their texture.
Meniscal tears.The meniscus is an important part of the knee supporting mechanism.The knee joint can be severely damaged by a variety of causes, such as arthritis or knee injury.This can cause pain and inability to walk.In some cases, replacing parts of the joint is the appropriate course of action (Andra et al, 2008).Meniscal tears are a knee injury that is common in both young athletes and the aging population.It requires accurate diagnosis and, if necessary, surgical intervention.Ramakrishna et al (2009) described a novel CAD diagnostic system for automatic detection of meniscal tears in the knee in MR images.Andra et al (2008) described a system for pre-operative surgical planning based on structural simulation of the bone-prosthesis system.The model of the bone was constructed based on information from CT datasets.

Radiotherapy planning.
In the process of radiotherapy planning the most critical issue is the identification of the gross tumor volume and its relationship with surrounding tissues in order to decide the appropriate irradiation angles to minimize non-tumor tissue damage.Zaidi & El Naqa (2010) published a survey on methods for tumor identification in PET images.

Multiscale methods for identification, some research and examples
The fact that real objects appear different depending on the scale of observation has important consequences to their description.The concept of scale is very important when analyzing image data by means of automatic methods (Koenderink, 1984).An object in a scene can be described in several forms, such as: its edges, its geometrical properties, its density or color, or its surface area.Ideally, each object has a set of features that set it aside from the other structures in the scene.Any method for the interpretation of image content must involve a phase for the detection of those features.Such detection is performed by the application of some operator to the image data.The application raises some questions about the type and size of the operators: the result of the operation is strongly dependent on these choices.The use of multiple scales is a way to overcome the problem of determining the size of the operator (Yuille & Poggio, 1986;Witkin, 1983).For optimal detection in a single scale, the size of the structure to be found should be known a priori.Several multiscale methods have been described in the literature for detection and classification of tissues and organs (Amira et al, 2008;Bernard & Friboulet, 2009;Gooya et al, 2008;Ugarriza et al, 2009;Yoon et al, 2008;Yu et al, 2008).In the following section we present an example of a representation for identification of organ structures that uses prior information in a multiscale technique.

Example application
In this example we propose a representation for the identification of medical structures by using a multiresolution approach, the scale-space.We evaluate the use of a data representation that allows the inclusion of a priori knowledge about the structures in several scales and we also develop the idea of an optimal scale to perform the processing.

Methodology
Linear scale-space.The linear scale-space concept was introduced in the Western literature by Witkin (1983) and Koenderink (1984).Weickert (1997) showed that the concept had already been introduced in Japan approximately 20 years before.One of the main requirements of the scale-space formalism is that structures at coarser scales in the multiscale representation should constitute simplifications of corresponding structures at finer scales.This requirement has been stated in different forms by several authors (Babaud et al 1986;Koenderink, 1984;Lindeberg, 1994;Yuille & Poggio, 1987).Adding the requisites of linearity and invariance to spatial displacement, those authors have shown that the Gaussian ( ) is the only filter that leads to monotonic destruction of detail under consecutive blurring.A blurred version of the original image  is obtained by its convolution with the Gaussian filter, for a specific  (Equation 1).In Equation 1, is the intensity of element after convolution.The width  determines the scale of the output image in each image dimension.In an isotropic condition,  assumes the same value for all image dimensions.The stack of images, as a function of increasing scale parameter , is known as linear scale-space.(2 ) 2 (, ) () (, ) In order to keep the property of scale invariance, the sampling of the scale parameter, , must be logarithmic (Keonderink, 1984).In the implementation of the Gaussian filter bank, a new parameter  is usually introduced and it varies linearly throughout scales and relates to  as: , where n is a natural number (N) consisting of zero and all positive integers, that represents the level in the scale space;  is the parameter related to the resolution, or inner scale, of each level and  determines the degree of spacing between the levels of the scale space.Linear scale space can be seen as a generic representation of image data that is common to different types of processing tasks.The main issues at this point are: how to describe properly the hierarchy throughout scales and how to extract important structures in each level of scale space and create a new data representation that can be suitable for high level processing routines.
Multiscale representation -scale space primal sketch.Primal sketch is an image structure first described by Marr (2010) based on the concept of gray level blobs (GLBlob), which are regions of the image that are lighter or darker than the surrounding background.Relating these regions to image structures results from the fact they have an intensity, or range of intensities, that may single them out among the other structures.Qualitatively, the concept of gray level blob for bi-dimensional images can be explained using the watershed operation from Mathematical Morphology (Lindeberg, 1994): the image function is seen as a flooded geographic area.As the water level decreases maxima points appear.At a certain point -or water level -two different peaks become connected.This point is called saddle.The difference between the image intensity at the maximum and at the saddle defines the contrast of each blob.The gray level of the saddle point is called blob base level.The support region of the blob is defined as the region containing the points with intensity values higher than the blob base level and can be reached from a local maximum without crossing points with intensity values l o w e r t h a n t h e b l o b b a s e l e v e l .F o r 2 D i m a g es, the gray level blob is defined as the tridimensional volume defined by the surface of the gray level and the base level.
A more formal definition of the blob concept is: the elliptical region associated to an extreme that is delimited by the above mentioned extreme and a saddle point.Differential geometry theory states that any image characteristic that has a geometrical meaning can be represented by a suitable differential operator.Florack et al (1992) presented an operator for the detection of elliptic regions -or blobs -in the images, the umbilicity operator.In 3D, it is given by Equation 2: Regions with positive values of the umbilicity define the gray level blob regions.To discriminate light from dark blobs the laplacian operator is used (Equation 3): 22 2 00 , 00 Dark and light blobs are not only prominent in the original image, but also tend to maintain this prominence throughout the scales (Koenderink, 1984).That characteristic makes them a suitable choice for the elements of an image representation containing scale information.The Scale Space Primal Sketch (SSPS), first described by Lindeberg (1994), is an extension of the Primal Sketch and describes the characteristics of the blobs and their relations across scale space.Blobs in different scales are linked and form a new structure, the scale space blobs (SSBlob).Figure 3 presents an example of a tree structure that maps blob relations in scale space.Figure 3.c depicts 5 levels of a scale space built for that image and their corresponding GLBlobs. Figure 3b presents the tree mapping of the relevant events occurring when following the blobs up in the scale space: creation, destruction and merging of blobs.These events are called bifurcations.A SSBlob is the object produced by the union of all GLBlobs that exist in the range between two bifurcations and is represented as a node in the tree.Each SSBlob is associated with a maximum scale (appearing scale) and a minimum scale (disappearing scale).The difference, given as number of scales, between the appearance and disappearance scales is called the life time.The information content of the tree nodes comprises the number of scales a structure survives, its appearance and disappearance scales, contrast with background, position at each survival scale, structure volume at each scale and total volume throughout scales.The linking between nodes reflects the behavior of the blobs during the smoothing process: creation, destruction and merging.This tree model is called Scale Space Primal Sketch.
Stable blobs in the SSPS must represent significant structures in the image (Koenderink, 1984, Lindeberg, 1994).Based on the stability of the SSBlobs significance measurement rates can be defined, allowing the classification of structures.The first significance measure proposed by Lindeberg (1994) was solely based on intrinsic characteristics of the image: blob volume and blob life time.The assumptions for this measurement were general and assumed there was not any a priori knowledge about the structures present in the image.As an example, the general significance measure can be applied to extract significant structures in thoracic magnetic resonance images, such as cardiac cavities, arteries and veins, shown in Figure 4 left.The thin arrows in Figure 4 right point to the 15 most significant dark blobs extracted using the general significance rate and the scale where the blob is more distinguished.Figure 4 right shows that almost all target structures are present amongst the set of significant blobs (exception is the atrium).Other anatomic structures are also present in the set, like the lungs (two lower arrows in scale 8).The general approach is not able, therefore, to set aside a particular structure from others present in the image.Furthermore, there are some structures that do not have any anatomic meaning, such as disconnected parts of organs in lower scales (lower arrows at scales 5 and 6) and some spurious structures originated from noise in upper scales, identified by the thicker arrows (Rebelo, 2000).In order to overcome the problems found with the application of the general approach, an important feature of medical images can be embedded in the image representation: the knowledge about the general characteristics and expected location of the target structure.
Including a priori information on SSPS.The SSPS is used as an image structure that serves as a guide to subsequent high level processing step where a priori knowledge about the desired feature is modeled and included in the representation.The image structure is first used to build patterns of known objects present in the image.A particular object can then be identified in new images by matching with symbolic features described in the pattern.Any kind of information can be embedded in the SSPS structure.The question that arises immediately regards defining the kind of information that is useful to include in the representation.The answer to this question is highly dependent on the problem domain, including the modality of the image under analysis, the kind of structure to be identified and the representation chosen for the biological structures.To validate the proposed method we have built a prototype application, which is described in detail in the next section.

Prototype application
In a previous work we proposed a prototype, in which geometrical information of the target structure and its relation with other structures present on the scene were included in SSPS (Rebelo, 2007).The set of parameters chosen for building the pattern was: (i) SSBlob volume, or the sum of the support regions of all GLBlobs; (ii) support region of the GLBlob; (iii) contextual information in the form of relative position of the target structure and the others present in the image, specifically distances and angles.For an image containing N GLBlobs at a given scale level the calculation of the set of parameters has a complexity of O(N 3 ).In addition, the calculation has to be performed for the images in all levels of the scale space.A score is determined for each structure present in the image based on the closeness of its feature values to the pattern.
The matching procedure presents, therefore, a problem related to the huge amount of GLBlobs at the lower levels of scale.This can lead to an explosion in the number of parameters to be processed, especially the number of angles, greatly increasing processing time to detect the node with the highest score.In order to decrease the possible combinations for matching, we have developed the following strategy: (1) decrease the number of nodes for the contextual matching phase by performing a selection of the possible nodes, using a global parameter.The prototype uses SSBlob volume, since the previous experiments with thoracic images showed its potential for recognition of relevant structures in medical images.A set of candidate nodes is created by eliminating the nodes with SSBlob volume too distant from the pattern; (2) choosing a suitable scale in the scale space for the contextual matching.
The concept of suitable scale for processing is based on the idea that the smoothness degree for which an object vanishes in the scale space is related to the object size.This issue links directly to the subject of scale selection, a task that is not trivial and that has been studied by some authors (Lindeberg, 1994;Majer, 2000;Pauly et al, 2003).These authors articles considered the case when image analysis is performed without any a priori knowledge about the structures present in the scene.In the present work the suitable scale is determined by constructing the SSPS representation for the pattern images.The analysis of the survival scales for the structure under study and all other structures representing real entities in the image allows the determination of the best scale for the geometrical matching.The following steps are performed: (1) inspect all scales at which the target structure can be detected as a single blob; (2) the levels of the scale space where all blobs representing real structures in the image are present are considered informative and (3) the upper informative scale is considered the most suitable for the contextual matching.
The reasons behind that choice are: (1) the occurrence of spurious structures in lower scales is higher.This fact would highly increase the number of useless calculation of distances and angles; (2) although structures are smoother in higher scales, the determination of angles and distances is based on the geometrical centers of each structure.These points should not change their position under the smoothness process.There is a problem with this strategy when there is not a single level in which all blobs representing real structures exist as a unique structure.This is the case when a small and low contrast structure is present and its corresponding blobs appear only in low scale levels, when bigger objects do not constitute a single blob anymore due to noise.In such cases, the suitable scale is chosen as the higher scale where the blob representing the target structure appears.

Schema of the prototype application
The identification of structures modeled in the prototype comprises the following steps: (1) creation of the scale space by using the linear Gaussian filter; (2) detection of GLBlobs through the application of the umbilicity operator; (3) creation of the Scale Space Primal Sketch; (4) comparison of the SSBlob volumes from all nodes with the pattern.Elimination of nodes with values outside a defined range; (5) determination of the distance and angles for the remaining SSBlob in the suitable scale; (6) contextual matching between each candidate node and the features' pattern: support region, angles and distances and ( 7) the node with the best matching is identified as the target structure.
Similarity criteria.In the simplification of the SSBlob tree processing phase, the candidate nodes are defined as those having SSBlob volume values lying in a range defined as a function of the pattern.That range was initially set to 20% around the pattern.The suitability of this choice are analyzed in the experiments.The similarity criterion for contextual matching consists in minimizing the Euclidean distance between each of the feature values obtained for each node of the test image and the pattern.
Pattern creation.The pattern is obtained by applying the proposed method to a set of data.The following parameters of the blob representing the target structure are stored in the pattern: SSBlob volume, the suitable scale, the support region of the GLBlob in this scale, all distances and angles between the target structure and the others present in this scale, the spatial resolution of the original image, the value of  used for the construction of the scale space.

Experiments
The Mathematical Cardiac Torso (MCAT) phantom is an anthropomorphic phantom, developed at the University of North Carolina that models size, shape and configurations of the major thoracic structures and organs by using of mathematical formulae (Tsui, 1993).A set of experiments was performed with two-dimensional images obtained from manipulations -adding and mirroring -of MCAT images.The manipulation aimed at creating frames with more organs, to make the matching step more complex.The goal was to analyze the ability of the method to identify organs at different levels of noise and contrast.The results for each case are described in the following sections.
Behavior for different noise levels.Five images were generated from the MCAT frames without noise.Three different levels of additive Gaussian noise with zero mean were then added to each image.For computing the noise level, we used the signal-to-noise ratio (SNR) described by Rangayan (2005).The five groups of images were classified into two distinct sets, according to their geometry.Figure 5 shows two images representatives of these sets: an image from Set 1 in first line and Set 2 in second line.In each set, one structure was chosen to be identified: structure 4 in Set1 and structure 5 in Set 2. The pattern was built based on the original images without noise.In Figure 5   After the construction of the scale space, using  = 0.19, the GLBlobs were detected in 12 levels of scale.Figure 6 depicts the GLBlobs detected for the images of Set 1 and Set 2, presented in Figure 5.In Figure 6 it is possible to observe that the number of blobs in lower scales increases for higher noise images.This effect tends to disapear for upper scale levels.
The application of the proposed method led to the successful identification of the chosen structures in most experiments, exception made to two with the highest noise level.Behavior for different contrast levels.The contrast between a structure and its background is one of the factors affecting the survival of the structure in the scale space, or, in other words, it affects the shape of the SSPS.In order to test the application of the method at several levels of contrast a simpler version of image 3 from Set 1, shown in Figure 5 (first line), was used.From the original image, two sets of images were created to assess how contrast influences SSPS shape.The first set was created by changing the counts of the thoracic region.The second set was created by changing both the counts of the thoracic region and the counts of the background, in such a way that the contrast between thorax and background remained constant.The contrast C(s,b) between a target structure S and its local background B, was calculated by Equation 5: µ S is the mean value of intensity inside the structure; µ B is the mean value of intensity in its background B. For each contrast manipulated image a set of noisy images with three different noise levels was created.For this experiment, noise was quantified by the contrast to noise ratio, defined in relation to the target structure S by Equation 6 (Rangayan, 2005): where  B is the standard deviation of intensity in the background.
The structure number 2 (Figure 5 upper A) was chosen as the target.An interesting observation was that when the contrast between thorax and background remained at the same value, the blobs of the images with low contrast between the internal structures and the thorax had a behavior not very different from the one observed in the images with high contrast.However, all internal structures survived in a lower number of levels as the contrast decreased.
The results of identification were satisfactory.In 87% of the experiments the target structure was identified correctly.The pattern was built based on the original image with the lowest contrast between the internal structures and the thorax.

Conclusion
In this chapter we discussed the challenges, methods and clinical examples of medical image identification.We pointed out the importance of this task as a previous step for medical image applications such as visualization and the quantitative analysis of organs and systems.A classification for the methods described in the literature and a brief review of their clinical applications were presented.
It was shown that identification of structures in medical images is a very difficult task due to the complexity of the anatomical structures and their changing with different physiological states.A simplification of the task can be achieved by dividing it in two phases: first, identification of the desired structures that includes a rough segmentation; then a postprocessing phase for segmentation refinement.
In this text we presented a method for the first phase that uses an image representation based on a scale space approach that embeds a priori knowledge in it.One of the main goals of this proposal was to develop a powerful representation for images that would be suitable for application of high level processing routines, such as the matching used in the prototype implemented.A study with MCAT phantom images was performed in order to assess the method under controlled conditions reproducing anatomical structures.The results we have obtained were quite encouraging.However, further intensive investigation with real patient data should be carried out in order to assess the full clinical usefulness of the method.The presented method is general and can be classified as a third generation method, according to the classification presented in the review, as it is a second generation multiresolution representation capable of incorporating knowledge to be used in a specific application.Although the representation is generic, the information chosen for definition of the pattern will depend on particular applications, and can be easily inserted in the data structure.
The review presented in this chapter was intended to give the reader a general scenario of medical applications.We focused on clinical applications and the issues for identification of particular organs and not on detailed application and modality-dependent aspects.Methods and applications that have appeared in the recent literature were briefly described.The main goal was to to give the readers a glimpse of this broad and extremely active research field.
In spite of the huge number of existing methods, the problem of identification in medical images is still challenging, with no general and unique solution and remains an open and promising area in the field of image processing.The growing research in the clinical areas makes the number of objects of interest in medical images to grow constantly, which generates a continuous demand for automatic solutions to aid clinicians.On the other side, fast advances in radiological imaging systems result in increasingly higher volume (3D) images to be processed.Processing of these images in radiological diagnostic systems requires accurate and fast algorithms.Intense use of apparel techniques must be studied, and the use of new computing platforms, such as computational grids, must be explored.
The goal of automated systems is not to replace the experts, whose experience and knowledge are essential in any automatic systems for identification or analysis.Instead, the development of automated systems to assist the clinicians is intended to reduce their workload in the step of analyzing patient data, saving them time to other valuable tasks as strategies and decision making.

Fig. 1 .
Fig. 1.Examples of effects that can affect structure identification can be seen in the three images: (A) kidney in Ultrasound (US), (B) lung in Nuclear Medicine (NM) and (C) brain in Computed Tomography (CT).The noise associated with each image modality has to be dealt with in different ways.In image (C) the presence of dental metal pieces degraded the reconstructed slice of the brain (images acquired at the Imaging Department -Heart Institute -HCFMUSP-with patient anonymization).

Fig. 3 .
Fig. 3. (a) an image that comprises three regions with higher counts than their background.The lower image of (a) shows the image represented as isosurface; (b) SSPS built for the scale space shown in (c); (c) Six levels of the scale space of image (a) and their corresponding GLBlobs.
signal  is the standard deviation of the intensity values of the original image (not corrupted with noise); noise  is the standard deviation of the intensity values of the noise image added to the original to generate the final corrupted image.The signal-to-noise ratio of the three different levels were 8dB (high noise level), 19 dB (medium level) and 33 dB (low level).
the original images (not corrupted by noise) are shown in column A. The noisy images are presented in column B (low level of noise), C (medium level of noise) and D (high level of noise).

Fig. 5 .
Fig. 5. Two images representative of the two Sets (image 3 from Set 1 and image 4 from Set 2).Column (A) shows the original images (without noise), column (B) the low noise images, column (C) medium noise images and column (D) high noise images.

Fig. 6 .
Fig. 6.Set of GLBlobs determined in 12 levels of scale for two groups of images shown in Figure 5.The set from Set 1 (original without noise, low level noise, mid level noise and high level noise) is shown on the left and the set from Set 2 is shown on the right.
Antunes et al (2010))Peters et al, 2010;Zheng et al, 2010)10).Ultrasound imaging is arguably the hardest medical imaging modality upon which to perform organ identification(Noble, 2010).The main problem in detecting, for example, heart boundaries in ultrasound images is related to the high level of multiplicative noise (mainly speckle noise), low contrast among structures, artifacts such as shadowing from the lungs and, attenuation.All these factors can hinder the automatic analysis of the images(Antunes et al, 2010).A recent work byMora et al (2009)combined supervised neural networks with different image processing techniques to identify heart cavities.Antunes et al (2010)applied a geometric deformable model to identify the four heart chambers of newborns.
Destrempes et al (2009)r identification is a particular challenging problem.The network trees of vessels are complex structures, with high variability in size and curvature.The general tubular organ geometry can be perturbed by stents, calcifications, aneurysms, and stenosis, besides being embedded in complex anatomical scenes, surrounded by other organs.Lesage et al (2009)published a review on vessel lumen segmentation techniques focused on 3D contrastenhanced imaging modalities (Magnetic Resonance Angiography -MRA and Computed Tomography Angiography -CTA).A former survey on vessel extraction techniques was presented byKirbas and Kerk (2002).Destrempes et al (2009)describe a method for extracting carotid artery in ultrasound images.A segmentation and visualization of the left coronary heart arteries in CT images was presented by Crum et al (2004)(2010)described a method for general organ segmentation with level sets that incorporates local intensity statistics and local curvature by means of a point-based tracking mechanism.Campadelli et al (2010)described a method for extracting abdominal organs in CT images, using a gray-level based segmentation framework and an a priori anatomical knowledge that can be adapted to segment different abdominal organs.Noble (2010) made a review on segmentation techniques in ultrasound images as well as ultrasound tissue characterization for general applications.Crum et al (2004)reviewed non-rigid image registration, with emphasis in cardiac and brain applications.Van Rixoort et al (2010) described a framework for multi-atlas segmentation, in which the most appropriate atlases for a target image are automatically selected.The framework was applied to segment heart from non ECG gated volumetric chest CT scans and caudate nucleus from MR brain images.