Federated Learning

Federated execution lets several independent Kaapana instances collaborate on a computation (training a model, aggregating radiomics features, collecting metadata) without moving raw patient data between them. Only intermediate artifacts – model weights, aggregated statistics, extracted features – ever cross an instance boundary.

Coordinator and Remote Instance Roles

A federation consists of one coordinator instance and one or more remote instances, registered with each other through the Instance Overview component (shared instance tokens and, optionally, Fernet encryption for the data exchanged between them). The web interface calls these the same roles by different names: the coordinator is the job’s Owner Instance, and the remote instances it distributes work to are the Runner instances. Each remote instance independently decides which of its own DAGs and datasets it makes available to the federation – the coordinator cannot execute or read anything the remote instance hasn’t explicitly allowed.

The orchestration flow

sequenceDiagram participant C as Coordinator DAG participant R1 as Remote instance 1 participant R2 as Remote instance 2 participant M as MinIO (presigned URLs) loop each federated round C->>R1: distribute job (local counterpart DAG) C->>R2: distribute job (local counterpart DAG) R1->>R1: run workflow on local data R2->>R2: run workflow on local data R1->>M: upload round result R2->>M: upload round result C->>M: download round results C->>C: aggregate (e.g. federated averaging) end C->>R1: distribute final/next round job

A federated workflow on the coordinator side is built around a round: it distributes a job to every registered remote instance (each remote instance runs the ordinary, non-federated counterpart workflow locally), waits for all of them to finish, downloads their results from MinIO via time-limited presigned URLs, aggregates them, and either starts another round or finalizes. This distribute-wait-aggregate loop is implemented once, in a shared base class, and reused by every federated workflow – individual workflows only implement what “aggregate” means for their use case.

Kaapana ships a handful of federated example workflows built on this base: federated nnU-Net training (which aggregates model weights and, in one mode, dataset intensity statistics across sites), federated radiomics feature extraction, federated metadata collection, and a minimal test pair used to validate that a newly connected federation is wired up correctly.

Note

This orchestration logic currently lives in kaapana-backend, not in the newer workflow-api / data-api services described in Backend Architecture. If you are extending federated execution today, you are extending the legacy backend’s federation endpoints, not the new workflow model.

Security model

Federation relies on the same building blocks as everything else in Kaapana: mutual instance registration with shared secrets and encrypted transport for sensitive payloads (see Access Control). However, federated capabilities are currently only supported in the admin project and only enabled for system admin users. A remote instance is never given direct access to the coordinator’s data or vice versa – all it receives is a job to run against its own data, and all it returns is the artifact its local workflow produced.

Building a new federated workflow

If you need a new federated workflow, look at one of the existing ones as a template rather than starting from scratch: subclass the shared federated-training base class, implement the aggregation step for your use case, and provide the non-federated counterpart DAG that each remote instance runs locally. See Developing Workflow Extensions for how workflow extensions are structured in general.

Future Direction

Federation is currently Kaapana’s own round-based orchestration, implemented once in a shared base class and reused by every federated workflow – it is not built on top of an external FL framework. Making federation agnostic to the FL engine is planned: a coordinator will be able to drive a round through a different, external FL framework instead.