Séminaire Image : « ECoLaF: an Evidential Conflict-guided Late Fusion for robust multimodal semantic segmentation », Lucas Deregnaucourt

Nous aurons le plaisir d’écouter Lucas Deregnaucourt, doctorant INSA Rouen.
Il donnera un séminaire IMAGE le jeudi 7 novembre à 14h00 en salle de séminaire F-200.

Titre : « ECoLaF: an Evidential Conflict-guided Late Fusion for robust multimodal semantic segmentation »

Résumé :
This work presents a novel and robust approach to semantic segmentation based on the fusion of different image modalities (conventional and non-conventional images). The robustness of fusion methods and their ability to tolerate sensor failures are crucial challenges for their deployment in real-world environments. It is essential to develop unique fusion models that can operate even in the absence of certain modalities during inference. However, current fusion methods have a strong dependence on the RGB branch, resulting in significant performance losses in case of its unavailability. To address this issue, we introduce ECoLaF (Evidential Conflict-guided Late Fusion), a ’late fusion’ method based on Dempster-Shafer theory. This method adaptively reduces the output of each modality according to their conflicts before fusing them. Experimental results show that ECoLaF outperforms state-of-the-art methods in terms of robustness on the challenging MCubeS and DeLiVER datasets, especially when the RGB sensor is not operational. This study offers new perspectives for improving the robustness of semantic segmentation in multimodal contexts.