When the Server waits until enough model updates are available, aggregates them, and replaces the global model parameters with the new aggregate.
My question is: In the aggregation process, Should all weights at clients, for example, be the same size? Can we aggregates different weights size from different clients?
If “size” refers to the shape of the NumPy ndarray’s returned from the clients, then yes, all of them should have the same shape.
This is only true if you use the built-in strategies. More advanced approaches that deal with differently shaped ndarray’s/tensors coming back from different clients can be implemented by creating custom strategies: How to implement strategies