Implementing leave-one-out cross-validation (LOOCV). How do I configure clients?

This question was migrated from the Slack channel #questions.

Original Question:

Hi everyone, I’m exploring the Flower framework for federated learning using this example, where each client has its own training and validation sets. I’m interested in implementing leave-one-out cross-validation (LOOCV) with this setup. Is it possible to configure it so that all clients hold their own training set, while one client at a time is used for evaluation, allowing each client a turn for validation? I would appreciate any insights or suggestions!

Yes, you can partition the dataset to split the dataset to clients, check out Flower Datasets 0.0.2 we have popular datasets ready to be used in a federated setting.
To sample evaluation for one client only, you can change the server code during evaluate round, so that you only send evaluate message to one client.