OmniMed-FL Multimodal Federated Learning Framework
OmniMed-FL: A Robust Multimodal Federated Learning Framework for Clinical Diagnosis
Addresses privacy concerns in medical AI by enabling secure fusion of imaging and patient data across hospitals without centralized data sharing.
OmniMed-FL is a multimodal federated learning framework for five-class clinical condition classification using 3,000 chest radiographs and 3,000 synthetic notes.
The framework benchmarks eight fusion strategies, three initializations, and four missing-text imputation rules across 3 to 20 hospital clients.
FedProx achieved the highest macro-F1 score of 0.737±0.
085 with K=5 clients and severe skew.
OmniMed-FL: A Robust Multimodal Federated Learning Framework for Clinical Diagnosis
Simultaneous assessment of medical imaging and patient records is often required in clinical diagnosis. However, standard machine learning algorithms cannot analyze these data types together. Meanwhile, compliance with HIPAA and GDPR can constrain centralized aggregation of sensitive patient data. This leaves a crucial void of secure fusion of visual and textual context across distant networks. Thus, we present OmniMed-FL, a controlled systems study of multimodal federated learning for five-class clinical condition classification (Normal, Pneumonia, COVID-19, Pleural Effusion, Cardiomegaly). Our proxy corpus pairs 3,000 public chest radiographs with 3,000 class-conditioned synthetic notes, matched by class, not by patient. The framework benchmarks eight fusion strategies, three initializations, four missing-text imputation rules, and matched federated baselines under non-IID Dirichlet partitioning across 3 to 20 hospital clients. As all notes are synthetic and pairing is not patient-level, these are descriptive proxy comparisons, not estimates of diagnostic performance or deployment readiness. Within those limits with clients ($K=5$) and severe skew ($α=0.1$), local-only training achieves a macro-F1 score of 0.297, FedAvg achieves $0.662\pm0.074$, FedProx $0.737\pm0.085$, a matched FedMME-style one-shot ensemble $0.647\pm0.080$, and our SCAFFOLD-AdamW adaptation $0.070\pm0.015$, the 0.075 FedProx-FedAvg gap falling inside the wider of the two two-seed standard deviations. Over a $4\times3$ grid, label skew costs up to 0.27 F1 whereas a near-sevenfold client increase costs at most 0.10, while bidirectional volume grows linearly to 183.5 GiB at $K=20$. Multimodal fusion leads on both corpora, scoring 0.956 against 0.934 for text and 0.664 for images on the synthetic corpus and 0.906 against 0.880 and 0.737 on the radiograph corpus, for $2.3\times$ the model state of text alone.
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include OmniMed-FL, federated learning, multimodal, medical AI.
Source attribution: curated from arXiv cs.LG (@Ayush Debnath, Ruelia Saha, Sudip Misra).
