Skip to content

About

Sandbox for running federated learning with NVIDIA FLARE through GA4GH WES and TES

Resources

Code of conduct

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

32 Commits

Folders and files

NVFlare on WES and TES testbed

A sandbox for running federated learning jobs with NVIDIA FLARE through GA4GH WES and TES, and an evaluation of the ways to do that.

Warning

This has only run on laptops, against two local Funnel instances, and twice on one cluster, with both sites in the hub's own cluster. It has not touched a partner's node. Its WES trusts one identity provider and gives every caller the same role, it sends one-time links for certificates through task records, and it takes every shortcut it can: all of them are listed. Never for production, and never for real data: its test set is synthetic and tiny.

Quick start

Needs Docker with Compose, on Linux. No Python on your machine, and no free ports. Takes a few minutes, mostly for the image builds.

git clone https://github.com/elixir-cloud-aai/nvflare-wes-testbed.git
cd nvflare-wes-testbed
docker compose --profile test build
docker compose --profile test run --rm e2e
docker compose --profile test down -v

It ends with 11/11 checks passed: one WES run, with one TES task per site, and a federated model that beats models trained at any one site alone. The Funnel containers get the Docker socket, so read compose.yaml first.

The same without NVFlare, as rounds of plain TES tasks that need no network (T10): docker compose --profile test run --rm fedavg-e2e, before down. It ends with 18/18 checks passed.

Links

Docs are written for Read the Docs (MkDocs Material), and readable here as Markdown.

  • Home
  • Background: use case, requirements and design, independent of any product
  • Goals: first the direct run, WES to each TES; then proTES
  • Options: what was evaluated, and the recommendation
  • Shortcuts and hacks: each with what the proper way would be
  • Setup
  • Tests
  • Outcomes: drafts from the local runs and the dress rehearsals; real-site rows follow
  • Issues: the work ahead
  • About

Licence: Apache-2.0. Never for production.

About

Sandbox for running federated learning with NVIDIA FLARE through GA4GH WES and TES

Resources

Code of conduct

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages