Computer Science > Computer Vision and Pattern Recognition
[Submitted on 31 Aug 2023 (v1), last revised 20 Nov 2023 (this version, v2)]
Title:BTSeg: Barlow Twins Regularization for Domain Adaptation in Semantic Segmentation
View PDFAbstract:Semantic image segmentation is particularly vital for the advancement of autonomous vehicle technologies. However, this domain faces substantial challenges under adverse conditions like rain or darkness, which remain underrepresented in most datasets. The generation of additional training data for these scenarios is not only costly but also fraught with potential inaccuracies, largely attributable to the aleatoric uncertainty inherent in such conditions.
We introduce BTSeg, an innovative, semi-supervised training approach enhancing semantic segmentation models in order to effectively handle a range of adverse conditions without requiring the creation of extensive new datasets. BTSeg employs a novel application of the Barlow Twins loss, a concept borrowed from unsupervised learning. The original Barlow Twins approach uses stochastic augmentations in order to learn useful representations from unlabeled data without the need for external labels. In our approach, we regard images captured at identical locations but under varying adverse conditions as manifold representation of the same scene (which could be interpreted as "natural augmentations"), thereby enabling the model to conceptualize its understanding of the environment.
We evaluate our approach on the ACDC dataset, where it performs favorably when compared to the current state-of-the-art methods, while also being simpler to implement and train. For the new challenging ACG benchmark it shows cutting-edge performance, demonstrating its robustness and generalization capabilities. We will make the code publicly available post-acceptance.
Submission history
From: Johannes Wolf Künzel [view email][v1] Thu, 31 Aug 2023 15:49:53 UTC (9,365 KB)
[v2] Mon, 20 Nov 2023 12:34:56 UTC (10,013 KB)
References & Citations
Bibliographic and Citation Tools
Bibliographic Explorer (What is the Explorer?)
Litmaps (What is Litmaps?)
scite Smart Citations (What are Smart Citations?)
Code, Data and Media Associated with this Article
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub (What is DagsHub?)
Gotit.pub (What is GotitPub?)
Papers with Code (What is Papers with Code?)
ScienceCast (What is ScienceCast?)
Demos
Recommenders and Search Tools
Influence Flower (What are Influence Flowers?)
Connected Papers (What is Connected Papers?)
CORE Recommender (What is CORE?)
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.