Thanks for the response! That makes sense! Another thing i have noticed, which I can’t really explain is something related to experimenting with one bad client.
So i am simulating 5 clients, one of them is considered bad, so they have drastically worse labels to train on than the others. So the bad properties learned from the bad client is aggregated, and makes the aggregated model (federated loss) worse on the testset as expected. But what is quite weird is when i track the training progress on each client, the bad performance is shifting between clients for each round, which can be seen in the image.
But the way i have set it up, all clients is getting the newest same aggregated model before training next round, so it doesn’t really make sense, why the bad performance is shifting between clients and not just the same (worse) ish for all clients?