CODEC-FM: Collaborative, Open, and DECentralized training of Foundation Models

§01 About

Training large-scale foundation models today depends on massive, centralized GPU clusters that are inaccessible to most academic institutions, startups, and industries.

This concentration of compute creates high barriers to entry, centralizes AI innovation, and limits broader progress in developing and studying frontier-scale foundation models. Decentralization and resource pooling provide a way to enable large-scale runs without these barriers, by training models across geographically distributed and heterogeneous devices — from coordinated inter-datacenter settings to consumer devices collaborating over the internet.

This workshop brings together researchers and practitioners from the open-model and decentralized-training communities to address the core technical challenges of this paradigm, extending foundation model training beyond the confines of a single datacenter and supporting more scalable, collaborative open model development.

Topics of interest

Open model development and community infrastructure
Communication-efficient distributed training
Asynchronous and local-update optimization
Inter-datacenter training
Internet-scale volunteer collaboration
Federated learning at scale
Heterogeneous devices and interconnects
Fault tolerance, stragglers, and elasticity
Parallelism strategies over low-bandwidth interconnects
Security and robustness in trustless settings
Incentives, governance, and economic models
Benchmarks and evaluation

§02 Important Dates

Submission deadline

August 29, 2026

Reviewing deadline

September 19, 2026

Notification of acceptance

September 29, 2026

Camera-ready deadline

TBA

Workshop day

December 11 or 12, 2026

All deadlines are 23:59 anywhere on Earth. Submit via the OpenReview portal.

§03 Invited Speakers

Peter Richtárik

KAUST

Professor of Computer Science at KAUST, where he leads the Optimization and Machine Learning Lab. One of the original developers of federated learning and an expert in randomized, distributed, and communication-efficient optimization algorithms.

Nathan Lambert

Interconnects · formerly Ai2

Author of Interconnects, a widely read newsletter on the open-model ecosystem, and an expert in RLHF and open language model post-training. Formerly post-training lead at the Allen Institute for AI (Ai2), where he developed RewardBench and led the Tülu 3 post-training recipe.

Max Ryabinin

Together AI

Distinguished Research Scientist at Together AI and a pioneer of fault-tolerant decentralized training, including SWARM parallelism, Hivemind, and Petals — frameworks for training and serving large models over unreliable, heterogeneous devices.

Alexander Long

Pluralis Research

Founder of Pluralis Research and a pioneer of Protocol Learning — open-source foundation model development via decentralized compute collaborations, including model-parallel training over the internet. Previously an applied scientist at Amazon.

Virginia Smith

Carnegie Mellon University

Leonardo Associate Professor of Machine Learning at Carnegie Mellon University and Sloan Research Fellow; an expert in federated learning and communication-efficient distributed optimization, including the CoCoA framework and foundational work on the challenges and methods of federated learning.

§04 Schedule

8:15 – 8:30

Organizers

Opening remarks

MorningKeynotes, Contributed Talks, and Posters

8:30 – 9:00

Invited speaker

Keynote 1

9:00 – 9:30

Invited speaker

Keynote 2

9:30 – 9:45

Coffee break

9:45 – 10:30

Contributed speakers

Contributed talks 1–3 (3 × 15 min)

10:30 – 11:00

Invited speaker

Keynote 3

11:00 – 12:30

All poster authors

Interactive poster session

12:30 – 1:30

Lunch

AfternoonKeynotes, Contributed Talks, and Panel

1:30 – 2:00

Invited speaker

Keynote 4

2:00 – 2:45

Contributed speakers

Contributed talks 4–6 (3 × 15 min)

2:45 – 3:00

Coffee break

3:00 – 3:30

Invited speaker

Keynote 5

3:30 – 4:45

Invited speakers

Panel discussion & Q&A — core research, scalability, and economic sustainability; moderated by the organizers

4:45 – 5:00

Organizers

Awards & closing remarks

Schedule is tentative; keynote speaker assignments will be announced closer to the workshop.

§05 Call for Papers

Submission format

Long papers: up to 6 pages — eligible for oral presentation
Tiny papers: up to 2 pages — poster presentations only
Page limits exclude references and appendix
Use the official NeurIPS 2026 LaTeX style files (linked from the NeurIPS 2026 Call for Papers)
All submissions are non-archival
All authors are required to have a valid OpenReview profile
Submissions are managed through OpenReview

What we accept

New research results
Position papers
Systems and infrastructure descriptions
Benchmark papers
Negative findings that clarify trade-offs

Topics we accept

We encourage contributions addressing the core challenges of collaborative, open, and decentralized training, including but not limited to:

Open model development and community infrastructure: systems, architectures, algorithms, and governance mechanisms that enable large-scale collaborative open model development on geographically distributed compute.
Communication-efficient and asynchronous training: compression, scheduling, and optimization techniques that enable large-scale training across all parallelism axes over low-bandwidth and high-latency interconnects, while improving device utilization through asynchronous execution.
Heterogeneous and fault-tolerant systems: training algorithms and systems that support heterogeneous devices and interconnects in decentralized networks, maximize aggregate throughput, and remain robust to unreliable nodes, stragglers, and intermittent connectivity.
Security and robustness: methods for detecting malicious or faulty participants, ensuring robust convergence, and enabling secure collaboration in trustless or partially trusted decentralized training environments.

Review process

Reviewing is double blind. Each submission will receive three reviews. Our program committee includes international experts with reviewing experience at top-tier conferences and recent workshops such as MCDC@ICLR'25 and FL@FM-NeurIPS'24. Organizers follow the NeurIPS conflict-of-interest policy. Six accepted papers will be selected for oral presentation, and all accepted papers will be presented in the poster session.

Awards

We will present awards for the best paper, best student paper, and best tiny paper. Specific details will be communicated closer to the workshop date.

Call for reviewers

We are recruiting reviewers from the broader community. If you work on distributed or decentralized training, federated learning, communication-efficient optimization, ML systems, or security for collaborative learning, we would be grateful for your help. To volunteer, fill out the reviewer sign-up form.

Dual submission

Because this workshop is non-archival, concurrent submission to other venues is permitted (subject to those venues' policies), and accepting a paper here does not preclude its later publication elsewhere. Accepted papers and the workshop schedule will be made publicly available on this website, and talks will be made public after the workshop.

Code of conduct

All authors, reviewers, and attendees are expected to adhere to the NeurIPS Code of Conduct.

Submissions are due August 29, 2026.

Submit on OpenReview ↗ Email the organizers Volunteer to review ↗

§06 Organizers

Thalaiyasingam Ajanthan

Pluralis Research · ANU

Founding Scientist at Pluralis Research and Visiting Researcher at the Australian National University. Previously a senior ML scientist at Amazon and a research fellow at Oxford and ANU; works on optimization, continual learning, and robust decentralized training algorithms.

Sameera Ramasinghe

Pluralis Research · U Adelaide

Founding Scientist at Pluralis Research and Visiting Research Fellow at the University of Adelaide; PhD from ANU. Previously a research scientist at Amazon; works on implicit neural representations, efficient decentralized training, and optimization on manifolds.

Benjamin Thérien

Mila · UdeM

Final-year PhD student at Mila / Université de Montréal advised by Irina Rish and Eugene Belilovsky; FRQNT PhD scholar. Works on meta-learned optimizers, continual pre-training, and optimizers for cross-datacenter and over-the-internet pre-training; previously at Meta FAIR and Capital One.

Anastasia Koloskova

University of Zurich

Assistant Professor of AI and Optimization at the University of Zurich. Previously a postdoctoral researcher at Stanford; PhD from EPFL in the Machine Learning and Optimization lab with Martin Jaggi. Works on optimization for decentralized and collaborative learning and privacy.

Kaja Gruntkowska

KAUST

PhD student at KAUST advised by Peter Richtárik and a Google PhD Fellow in Algorithms and Optimization. Develops scalable optimization methods with strong theoretical guarantees for distributed, federated, and communication-efficient learning; previously at Warwick and Oxford.

Eugene Belilovsky

Concordia · Mila

Associate Professor at Concordia University and member of Mila, leading a research group on efficient training for large-scale deep learning. Research spans decentralized learning, federated learning, and model parallelism; co-organized workshops at ICML, ICLR, and NeurIPS.

Aakanksha Chowdhery

Reflection AI · Stanford

Researcher at Reflection AI and adjunct professor at Stanford; Program Chair for MLSys 2026. Technical lead of the 540B PaLM model and a lead contributor to Gemini pre-training at Google; now helping build frontier open intelligence accessible to all.

Nicholas Lane

Cambridge · Flower Labs

Professor of Machine Learning Systems at the University of Cambridge, where he leads the CaMLSys lab; co-founder and Chief Scientific Officer of Flower Labs, creators of the Flower federated learning framework. Royal Academy of Engineering Chair in Decentralized AI.

§07 Sponsors

§08 Contact

Email the organizers.

For submission questions, sponsorship enquiries, accessibility requests, or anything else.

codec-fm-neurips-2026@googlegroups.com