Federated Averaging
Follow the data, decisions, feedback, and validation boundaries before writing the full system.
How to read this diagram
Read left to right for the forward path: raw information becomes a representation, passes through the project’s main computational ideas, and produces an output that can be measured. Then follow the feedback path back toward the trainable or decision-making components.
FedAvg
FedAvg lets clients perform local SGD and combines their parameters with weights proportional to client sample counts. Local work saves communication, but heterogeneous client data causes update drift and changes the optimization problem.
Boundary check: document its accepted input, output shape, mutable state, failure modes, and the metric that proves this stage is correct before connecting it downstream.
Non-IID data
Non-IID data means clients or shards follow different distributions. It tests whether aggregation remains stable when local gradients disagree and whether reported global accuracy hides poor subgroup behavior.
Boundary check: document its accepted input, output shape, mutable state, failure modes, and the metric that proves this stage is correct before connecting it downstream.
Communication rounds
Communication rounds defines one of the project’s main information transformations. Understand its input representation, objective, numerical invariants, computational cost, and failure modes before relying on a library implementation.
Boundary check: document its accepted input, output shape, mutable state, failure modes, and the metric that proves this stage is correct before connecting it downstream.
Architecture review checklist
- Every arrow has a documented shape, dtype, unit, or schema.
- Training and evaluation paths cannot leak information into each other.
- Randomness is seeded and captured in experiment metadata.
- Expensive stages expose timing, memory, throughput, and error metrics.
- Each feedback loop has a stop condition and a rollback strategy.
- Small reference implementations exist for numerical comparisons.