Zusammenfassung
Epipolar geometry is exploited in several applications in the field of Cone-Beam Computed Tomography (CBCT) imaging. By leveraging consistency conditions between multiple views of the same scene, motion artifacts can be minimized, the effects of beam hardening can be reduced, and segmentation masks can be refined. So far, these conditions have been formulated as optimization criteria to be minimized in post-processing. In this work, we explore the idea of enabling deep learning models to access the known geometrical relations between views in order to improve the prediction. The implicit 3D information contained in the relative pose between views can potentially enhance various projection domain algorithms such as segmentation, detection, or inpainting. Based on this hypothesis, we introduce a differentiable feature translation operator, which uses available projection matrices to calculate and integrate over the epipolar line in a second view. After geometrically translating all channels of a layer’s output feature map, we concatenate these activations to a second instance of the same architecture processing a second view of the same 3D scene. As an example application, we evaluate the effects of this operation on the task of projection domain metal segmentation. By re-sampling a stack of projections into orthogonal viewpairs,we segment each projection image jointly with a second view acquired 90° apart. The comparison with an equivalent single-view segmentation model reveals an improved segmentation performance of 0.95 over 0.91 measured by the dice coefficient. By providing an implementation of this operator as an open-access differentiable layer, we seek to enable future research [1].
Chapter PDF
Similar content being viewed by others
References
Rohleder M, Pradel C, Wagner F, Thies M, Maul N, Denzinger F et al. Enabling Geometry Aware Learning Through Differentiable Epipolar View Translation. Proc MICCAI. 2023:57– 65.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 Der/die Autor(en), exklusiv lizenziert an Springer Fachmedien Wiesbaden GmbH, ein Teil von Springer Nature
About this paper
Cite this paper
Rohleder, M. et al. (2024). Abstract: Enabling Geometry Aware Learning Through Differentiable Epipolar View Translation. In: Maier, A., Deserno, T.M., Handels, H., Maier-Hein, K., Palm, C., Tolxdorff, T. (eds) Bildverarbeitung für die Medizin 2024. BVM 2024. Informatik aktuell. Springer Vieweg, Wiesbaden. https://doi.org/10.1007/978-3-658-44037-4_61
Download citation
DOI: https://doi.org/10.1007/978-3-658-44037-4_61
Published:
Publisher Name: Springer Vieweg, Wiesbaden
Print ISBN: 978-3-658-44036-7
Online ISBN: 978-3-658-44037-4
eBook Packages: Computer Science and Engineering (German Language)