Announcing Flower 1.10

General
1

Announcing Flower 1.10
Announcing Flower 1.101200×628 44.1 KB

The Flower Team is excited to announce the release of Flower 1.10 stable and it’s packed with updates! Flower is a friendly framework for collaborative AI and data science. It makes novel approaches such as federated learning, federated evaluation, federated analytics, and fleet learning accessible to a wide audience of researchers and engineers.

Thanks to our contributors

We would like to give our special thanks to all the contributors who made the new version of Flower possible (in git shortlog order):

Adam Narozniak, Charles Beauville, Chong Shen Ng, Daniel J. Beutel, Daniel Nata Nugraha, Danny, Gustavo Bertoli, Heng Pan, Ikko Eltociear Ashimine, Javier, Jiahao Tan, Mohammad Naseri, Robert Steiner, Sebastian van der Voort, Taner Topal, Yan Gao

What’s new?

  • Introduce flwr run (beta) (#3810, #3826, #3880, #3807, #3800, #3814, #3811, #3809, #3819)

    Flower 1.10 ships the first beta release of the new flwr run command. flwr run can run different projects using flwr run path/to/project, it enables you to easily switch between different federations using flwr run . federation and it runs your Flower project using either local simulation or the new (experimental) SuperExec service. This allows Flower to scale federatated learning from fast local simulation to large-scale production deployment, seamlessly. All projects generated with flwr new are immediately runnable using flwr run. Give it a try: use flwr new to generate a project and then run it using flwr run.

  • Introduce run config (#3751, #3750, #3845, #3824, #3746, #3728, #3730, #3725, #3729, #3580, #3578, #3576, #3798, #3732, #3815)

    The new run config feature allows you to run your Flower project in different configurations without having to change a single line of code. You can now build a configurable ServerApp and ClientApp that read configuration values at runtime. This enables you to specify config values like learning-rate=0.01 in pyproject.toml (under the [tool.flwr.app.config] key). These config values can then be easily overridden via flwr run --run-config learning-rate=0.02, and read from Context using lr = context.run_config["learning-rate"]. Create a new project using flwr new to see run config in action.

  • Generalize client_fn signature to client_fn(context: Context) -> Client (#3779, #3697, #3694, #3696)

    The client_fn signature has been generalized to client_fn(context: Context) -> Client. It now receives a Context object instead of the (now depreacated) cid: str. Context allows accessing node_id, node_config and run_config, among other things. This enables you to build a configurable ClientApp that leverages the new run config system.

    The previous signature client_fn(cid: str) is now deprecated and support for it will be removed in a future release. Use client_fn(context: Context) -> Client everywhere.

  • Introduce new server_fn(context) (#3773, #3796, #3771)

    In addition to the new client_fn(context:Context), a new server_fn(context: Context) -> ServerAppComponents can now be passed to ServerApp (instead of passing, for example, Strategy, directly). This enables you to leverage the full Context on the server-side to build a configurable ServerApp.

  • Relaunch all flwr new templates (#3877, #3821, #3587, #3795, #3875, #3859, #3760)

    All flwr new templates have been significantly updated to showcase new Flower features and best practices. This includes using flwr run and the new run config feature. You can now easily create a new project using flwr new and, after following the instructions to install it, flwr run it.

  • Introduce flower-supernode (preview) (#3353)

    The new flower-supernode CLI is here to replace flower-client-app. flower-supernode brings full multi-app support to the Flower client-side. It also allows to pass --node-config to the SuperNode, which is accessible in your ClientApp via Context (using the new client_fn(context: Context) signature).

  • Introduce node config (#3782, #3780, #3695, #3886)

    A new node config feature allows you to pass a static configuration to the SuperNode. This configuration is read-only and available to every ClientApp running on that SuperNode. A ClientApp can access the node config via Context (context.node_config).

  • Introduce SuperExec (experimental) (#3605, #3723, #3731, #3589, #3604, #3622, #3838, #3720, #3606, #3602, #3603, #3555, #3808, #3724, #3658, #3629)

    This is the first experimental release of Flower SuperExec, a new service that executes your runs. It’s not ready for production deployment just yet, but don’t hesitate to give it a try if you’re interested.

  • Add new federated learning with tabular data example (#3568)

    A new code example exemplifies a federated learning setup using the Flower framework on the Adult Census Income tabular dataset.

  • Create generic adapter layer (preview) (#3538, #3536, #3540)

    A new generic gRPC adapter layer allows 3rd-party frameworks to integrate with Flower in a transparent way. This makes Flower more modular and allows for integration into other federated learning solutions and platforms.

  • Refactor Flower Simulation Engine (#3581, #3471, #3804, #3468, #3839, #3806, #3861, #3543, #3472, #3829, #3469)

    The Simulation Engine was significantly refactored. This results in faster and more stable simulations. It is also the foundation for upcoming changes that aim to provide the next level of performance and configurability in federated learning simulations.

  • Optimize Docker containers (#3591)

    Flower Docker containers were optimized and updated to use that latest Flower framework features.

  • Improve logging (#3776, #3789)

    Improved logging aims to be more concise and helpful to show you the details you actually care about.

  • Refactor framework internals (#3621, #3792, #3772, #3805, #3583, #3825, #3597, #3802, #3569)

    As always, many parts of the Flower framework and quality infrastructure were improved and updated.

Documentation improvements

Deprecations

  • Deprecate client_fn(cid: str)

    client_fn used to have a signature client_fn(cid: str) -> Client. This signature is now deprecated. Use the new signature client_fn(context: Context) -> Client instead. The new argument context allows accessing node_id, node_config, run_config and other Context features. When running using the simulation engine (or using flower-supernode with a custom --node-config partition-id=...), context.node_config["partition-id"] will return an int partition ID that can be used with Flower Datasets to load a different partition of the dataset on each simulated or deployed SuperNode.

  • Deprecate passing Server/ServerConfig/Strategy/ClientManager to ServerApp directly

    Creating ServerApp using ServerApp(config=config, strategy=strategy) is now deprecated. Instead of passing Server/ServerConfig/Strategy/ClientManager to ServerApp directly, pass them wrapped in a server_fn(context: Context) -> ServerAppComponents function, like this: ServerApp(server_fn=server_fn). ServerAppComponents can hold references to Server/ServerConfig/Strategy/ClientManager. In addition to that, server_fn allows you to access Context (for example, to read the run_config).

Incompatible changes

  • Remove support for client_ids in start_simulation (#3699)

    The (rarely used) feature that allowed passing custom client_ids to the start_simulation function was removed. This removal is part of a bigger effort to refactor the simulation engine and unify how the Flower internals work in simulation and deployment.

  • Remove flower-driver-api and flower-fleet-api (#3418)

    The two deprecated CLI commands flower-driver-api and flower-fleet-api were removed in an effort to streamline the SuperLink developer experience. Use flower-superlink instead.

4 Likes
2

Hi @daniel , thank you for this great news,
I wonder how I can now communicate the id to my dataset_partition function, since cid: str is deprecated in get_client_fn.
Is there any other way?
Thank you in advance.
Best regards,
Viktor

1 Like
3

Hi Viktor, I missed your reply!

There are several ways to do what you asked about, but the approach we use in code examples is to start each client (SuperNode) with a node config that has the details it needs:

flower-supernode [other args] --node-config "partition-id=0 num-partitions=3"

Those two key names are what the simulation engine inserts by default. So if you start your clients (SuperNodes) like that, and your app uses both to load a data partition using Flower Datasets, then your app will automatically be compatible with both simulation mode and deployment mode.

1 Like