Server-Edges-Clients Settings

[mzonth]

Hello everyone,

I am currently attempting to implement (server-edges-clients) in Flower instead of (server-client) settings. However, I find myself having to perform many steps manually, which results in numerous errors. I believe that there must be a more efficient way to manage this process. My question is: How can I optimize the setup involving a main server, 2 edges, and 10 clients? Here’s what I’ve implemented so far:

1. Initially, both edges serve as servers for their clients, conducting federated learning and saving final models.

2. Then, the edges switch roles to become clients with the main server, sending their models for aggregation.

3. Finally, the server combines these models into one global model and shares it back with the edges.

The server.py and client.py files contain typical code. Here is a brief overview of the Edge1.py code:

#Edge1.py code: 

def run_server(): 

  ......... 

def save_model(): 

................ 

def run_client(): 

.................... 

   

if __name__ == "__main__": 

    run_server()  # run the edge server as a server 

    save_model() # Save the model into Edge_model to be sent to the server  

     run_client(Edge_model) #Run as a client with the main server 

     Global_model = ...# The global model after aggregation  

    run_server(Global_model) # run the edge server  again as a server

I have to do the same manually in Edge2.py, Could you please advise me on how to improve it?

Thank you in Advance.

[mzonth]

Hello Everyone,
I’ve made progress and answered my questions here. When I finish it, I am planning to make the code public. However, I find myself having some additional questions.

The setup involving a main server, 2 edges, and 10 clients. Initially, both edges serve as servers for their clients, conducting federated learning and saving their final edge models as Edge1_checkpoints and Edge2_checkpoints following thr step in How do I save the global model after training?

#Server.py:

ROUNDS=1

def run_main_server() -> None:

strategy = fl.server.strategy.FedAvg( )

fl.server.start_server(
server_address="0.0.0.0:8888",
config=fl.server.ServerConfig(num_rounds=ROUNDS),
strategy=strategy,
)


if __name__ == "__main__":
run_main_server()

#Edge.py:
def run_client()-> None:

class FlowerClient(fl.client.NumPyClient):
      def get_parameters(self, config): 
            # "parameters_round_2.pkl" is the file with weights in the last round on the Edge1_checkpoints
            with open(os.path.join('HFL/Edge1_checkpoints', "parameters_round_2.pkl"), 'rb') as f:
                model_dict = pickle.load(f)
                weights = model_dict['globa_parameters']
                return weights


fl.client.start_numpy_client(server_address="127.0.0.0:8088", client=FlowerClient())


if __name__ == "__main__":
run_client()

My questions are:

1. As described here: https://flower.ai/docs/framework/tutorial-series-use-a-federated-learning-strategy-pytorch.html Flower, by default, initializes the global model by asking one random client for the initial parameter through def get_parameters(self, config), How can I ensure that both models are sent to the server for aggregation?

2. In global federated Averaging, after summing all edge parameters, do we divide it by the number of edges(2) or the number of training data under each edge?

3. Is it possible for the client to have only the get_parameters(self, config) function if the intention is solely to send the edge model to the server for aggregation? In such a scenario, would including functions like evaluate() and fit() be meaningless, considering there is no training conducted on the edge and no evaluation required?

Thank you in Advance.

[yan-gao]

Hi @mzonth,
For question 1 and 3: apart from overriding get_parameters() method as you did, you could also override fit() as below:

def fit(self, parameters, config):
    return self.get_parameters(config={}), 1, {}

This will just return the loaded models to the server without further training. Then set the number of rounds to 1, which will aggregate the two models on the server.

For question 2, it depends on the second value returned from fit(). For now, it’s 1 (return self.get_parameters(config={}), 1, {}), which will divide by the number of edges(2). If we want to do it based on the number of training data, you could get the sample numbers and pass it to fit().