Skip to Main content Skip to Navigation
New interface
Conference papers

Conditional Alignment and Uniformity for Contrastive Learning with Continuous Proxy Labels

Abstract : Contrastive Learning has shown impressive results on natural and medical images, without requiring annotated data. However, a particularity of medical images is the availability of meta-data (such as age or sex) that can be exploited for learning representations. Here, we show that the recently proposed contrastive y-Aware InfoNCE loss, that integrates multi-dimensional meta-data, asymptotically optimizes two properties: conditional alignment and global uniformity. Similarly to [33], conditional alignment means that similar samples should have similar features, but conditionally on the meta-data. Instead, global uniformity means that the (normalized) features should be uniformly distributed on the unit hypersphere, independently of the meta-data. Here, we propose to define conditional uniformity, relying on the meta-data, that repel only samples with dissimilar metadata. We show that direct optimization of both conditional alignment and uniformity improves the representations, in terms of linear evaluation, on both CIFAR-100 and a brain MRI dataset.
Complete list of metadata

https://hal.telecom-paris.fr/hal-03523114
Contributor : Pietro Gori Connect in order to contact the contributor
Submitted on : Wednesday, January 12, 2022 - 2:46:15 PM
Last modification on : Thursday, January 27, 2022 - 3:45:13 AM
Long-term archiving on: : Wednesday, April 13, 2022 - 11:05:25 PM

File

2111.05643.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-03523114, version 1

Citation

Benoit Dufumier, Pietro Gori, Julie Victor, Antoine Grigis, Edouard Duchesnay. Conditional Alignment and Uniformity for Contrastive Learning with Continuous Proxy Labels. Med-NeurIPS - Workshop NeurIPS, Dec 2021, Vancouver, Canada. ⟨hal-03523114⟩

Share

Metrics

Record views

101

Files downloads

36