Hello guys. I am working on a project where each participant in FL wants to get a unique model after training.
I currently have a preliminary idea like this paper’s Architecture Sharing part: only a portion of the model participates in FL. Taking BERT-based text classification as an example, we typically define a BERT+classification head model. We could only share the BERT parameters during get_weights and set_weights operations.
This basic approach (sharing only part of the model) doesn’t seem difficult to implement in Flower, but the problem is how to ensure the models obtained by each party ultimately have sufficiently significant differences. In conventional horizontal FL where all parties have identical task definitions and data formats, the resulting models might not show much difference if the backbone is shared.
Has anyone had experience with similar tasks, especially those that can be implemented within Flower? Thanks for any suggestions and help 
1 Like