The most useful surgical AI models require data from many institutions. A model trained at a single academic center learns that center's patient population, its scanner protocols, and its referral patterns. It frequently performs worse elsewhere. Multi-institutional training is the accepted solution.
The obstacle has never been technical. It is that moving patient data between institutions requires approvals that most institutions will not grant. Federated learning removes the need to move the data at all, and in doing so removes the specific approvals that stop most multi-institutional studies before they start.
How It Works
Conventional model training collects data in one place. Every institution sends its images to a central repository, and the model trains on the pooled dataset. Federated learning inverts this arrangement. The model travels to the data instead of the data traveling to the model.
The training cycle proceeds in five steps, repeated many thousands of times over a training run.
First, each participating site receives a copy of the current model. Second, each site trains that model on its own local patients, using images that never leave the institution. Third, rather than sending the images, each site computes a numerical update describing how the model should change based on what it just saw. Fourth, each site transmits only that update to a coordinating server, which averages the updates received from all sites. Fifth, every site receives the averaged update and applies it. The cycle then repeats.
The update is not a compressed image or a statistical summary of the data. It is a set of numerical adjustments to the model, the same quantity the model would compute during ordinary training, transmitted instead of applied locally. No image, no report, and no identifier crosses an institutional boundary at any point.
Weights and Gradients
Two terms recur in this literature and the distinction between them determines how well a federated system performs.
- The weights of a model are its current state. They are the full set of internal numbers that define what the model has learned so far. A model with ten million weights is described entirely by those ten million values at any given moment.
- A gradient is a proposed adjustment to those weights. After the model examines a batch of images, it calculates one number for every weight, indicating the direction and magnitude by which that weight should change to improve performance on what it just saw. The gradient is therefore not the model itself but a correction to it.
The two dominant federated approaches differ in which of these they transmit, and how often.
- Under federated weight averaging, each site trains independently for an extended period, commonly ten full passes through its local dataset. Each site arrives at its own set of weights, and those sets are no longer the same. The sites then transmit their final weights and the server averages them. The difficulty is that during the long interval of independent training, each site has drifted toward a model suited to its own patient population. Averaging weights that have diverged is an approximation, and the resulting model is not equivalent to one trained on the pooled data.
- Under federated gradient averaging, each site computes the gradient from a single batch and transmits it immediately, before applying it. The server averages the gradients from all sites and returns the result, which every site then applies. Because all sites begin from the same weights and apply the same averaged correction, they remain identical to one another throughout training. No drift occurs because no interval exists in which drift could accumulate.
Why Gradient Averaging Reproduces Pooled Training
A reasonable objection is that any federated approach must produce a worse model than training on pooled data. Under gradient averaging, it does not.
The mathematical reason is that the average of the gradients computed at each site equals the gradient that would have been computed from all of the data at once. When every site applies that same averaged gradient starting from identical weights, the training trajectory matches what centralized training would have produced.
Remedios and colleagues demonstrated this directly in 2020 [1]. They constructed an adversarial test of data heterogeneity in which one site held only handwritten digits 0 through 4 and the other held only digits 5 through 9. There was no overlap between the two datasets. This is a more extreme imbalance than any realistic hospital network would produce.
The federated model matched the centrally trained model exactly. Final model weights differed by less than one part in a trillion, which is the scale of ordinary floating-point rounding. This is not comparable performance but the same model. Federated weight averaging, which synchronizes far less frequently, lost 5.4 percentage points, and sequential site-to-site training lost roughly 15.
The authors also applied the method to clinical data, training a head CT hemorrhage segmentation model across the NIH Clinical Center in Maryland and Vanderbilt University in Tennessee, on different hardware and different software versions. A centralized comparison was not possible for that dataset because the data legally could not be transferred. Against the alternative federated methods, gradient averaging produced a Dice score of 0.76 compared with 0.72, a statistically significant improvement.
What Changes in Institutional Governance
The governance implications are the reason this matters for surgeon-researchers, and they are substantial.
- No data use agreement for transfer is required. The negotiation between legal departments that typically consumes months exists to govern the movement of data. When no data moves, that agreement is unnecessary. A federation participation agreement governs the collaboration instead, and it is a materially simpler document.
- No institutional approval for data egress is required. This is the highest bar in multi-institutional research and the one most frequently denied outright. A request to send patient imaging outside the institution invokes privacy, security, compliance, and liability review simultaneously. A request to run computation on data that remains in place invokes none of them in the same way.
- No institution sells or licenses its data. Health systems are increasingly reluctant to transfer data assets to commercial partners, and reasonably so. Valuation is difficult, downstream use is hard to constrain, and the institution permanently loses exclusive control. Federated participation requires none of this. The institution contributes to a model without surrendering the underlying asset.
- Institutional review is still required, but the pathway changes substantially. Research on patient data continues to require IRB review or a formal determination of exemption. What changes is which pathway applies. Because no identifiable data leaves the institution and no external party gains access to the records, federated studies are considerably more likely to qualify for exemption or expedited review rather than full board review. This is a meaningful reduction in administrative burden, though it is a change in the review pathway rather than an elimination of oversight.
For international collaboration the difference is larger still. Cross-border transfer of health data triggers GDPR obligations in Europe and national data-residency requirements in many other jurisdictions. Those requirements are frequently prohibitive. When the data never crosses a border, the analysis changes fundamentally.
The Costs
Three tradeoffs deserve explicit mention.
Training is slower. In the published implementation, federated gradient averaging ran approximately twice as slowly as methods that synchronize less frequently. The network coordination is the cost of the exactness. Whether that is acceptable depends on the study, but a factor of two is a reasonable price for access to data that would otherwise be unavailable.
Synchronization frequency is a design parameter rather than a fixed property of the method. The demonstrated implementation synchronizes after every training batch, which maximizes both exactness and communication cost. Synchronizing every ten or fifty batches reduces communication substantially while introducing far less deviation than methods that synchronize once per training epoch. Gradient compression techniques reduce the transmitted volume further, with published reductions of two to three orders of magnitude [2].
Privacy protection must be engineered deliberately. Transmitting model updates rather than images is a meaningful privacy improvement, but it is not by itself a privacy guarantee. Published work has demonstrated that model updates can leak information about the data that produced them. Systems intended for clinical use require differential privacy, a formal method of adding calibrated statistical noise that bounds how much any individual patient can influence what is transmitted [3].
Why This Is the Next Frontier
The models that will meaningfully change surgical decision-making require training populations that no single institution possesses. Scanner variation, population differences, surgical technique variation, and rare presentations all demand breadth that a single center cannot supply.
What has been missing is a way to obtain that breadth without asking every participating institution to accept a governance burden most will refuse. Federated learning supplies it. It converts the question from whether an institution will send its data to whether it will permit computation on data it continues to control. That is a question with a very different answer at most institutions.
For several AI builds in particular, this is the difference between a collaboration that can be assembled and one that cannot.
References
[1] Remedios SW, Butman JA, Landman BA, Pham DL. Federated Gradient Averaging for Multi-Site Training with Momentum-Based Optimizers. In: Domain Adaptation and Representation Transfer, and Distributed and Collaborative Learning (DART/DCL 2020), MICCAI. Lecture Notes in Computer Science, vol 12444. Springer; 2020:170–180. PMID 34531637.
[2] Lin Y, Han S, Mao H, Wang Y, Dally WJ. Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training. In: International Conference on Learning Representations (ICLR). 2018. arXiv:1712.01887.
[3] Abadi M, Chu A, Goodfellow I, McMahan HB, Mironov I, Talwar K, Zhang L. Deep Learning with Differential Privacy. In: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security (CCS). 2016:308–318.