Strategy with multiple communication rounds per iteration

Flower Help - Beginners
1

Hi everyone,
I’m working on implementing the GIANT algorithm ([1709.03528] GIANT: Globally Improved Approximate Newton Method for Distributed Optimization), a second-order distributed optimization algorithm that aims to apply the Newton Method in an approximate way for federated learning. To implement this, I need to perform multiple communication rounds between the server and clients before updating the global model parameters.
I’ve reviewed the Flower documentation, and it seems to me that the library is designed around a single communication step, where only model parameters are exchanged.
I’m wondering if it’s possible to create a custom Strategy in Flower that supports these additional communication rounds.

Here’s what I specifically need to achieve:

  1. Clients compute local gradients and send them to the server.
  2. The server aggregates the gradients into a global gradient and sends it back to the clients.
  3. Clients use the global gradient together with their local data to compute an approximate Newton direction (ANT).
  4. Clients send their ANTs back to the server for aggregation, which will determine the direction for the next optimization step.

Thanks in advance for your advice!

2 Likes
2

Hello, I am also interested in the use of second-order optimization methods in FL. If you have any updates, please let me know. Thank you.

2 Likes
3

Hi @rmazzier @tslehbdphj3px53 , this is a great question and something that very well fits the capabilities of the newer (but still preview) low-level API in flower. In short: it allows you to define your ServerApp as a standard “for loop” enabling you to customize what a “round” is. You could have multiple communication exchanges within a round.

We are working on an example for this. If you are keen to dive into it, take a look at this brach. Note that this low-level API also enables you to construct Messages directly, for which you’ll have to use RecordSet as their payload.

The code linked above is part of this PR, that you are welcome to comment on if you’d like. My plan is to merge this within a couple of days.

1 Like
4

I will update this thread once the example if finalised. Looking forward to hearing from your !! Let me know if this is useful :smiling_face:

1 Like
5

Thank you for your kind answer!
Unfortunately, I needed this for a relatively small sized project with a close deadline, so I ended up coding a couple of custom Server and Client classes without using any functionality from Flower, as I am still not very experienced with it.

Nonetheless I will surely remember to check out the new low-level APIs for potential future projects :+1:t2:

6

Thank you for your reply.
I would be happy to try Flower and its new features in my project. If I have any experiences or encounter any issues while using the features mentioned above, I will gladly share them with the community.