A prototype implementation of a “network of ML networks” - an internet-like protocol for federated learning where nodes can discover, join, and migrate between different learning groups based on performance metrics.
Want do you think of this? Kind of a network build on Flower AI learning groups. It could be cool to build a Napster/BitTorrent-like app on this to collaboratively train and share arbitrary machine learning models. Would love to hear your opinion.
Best
blueberry
@blueberry very interesting suggestion! Is there a way/have you considered an approach to simulate/test this in some sort of sandbox version and see how it scales? Also, what is the data type that is being fed into the model(s)?
@williamlm Well, in an earlier version of the project, I simulated it with around 10 nodes, but back then not with flower.ai. The ecosystem quickly stagnated and all nodes joined the same learning group. Additionally, the microservices were rather large, so already running around 10 nodes brought my laptop to its knees.
So yeah, the first attempt was interesting (basic functionality working; not so surprisingly), but did not scale well (the much more interesting question). That’s what I will try to figure out with this new prototype. But I’m only at the beginning.
Regarding the data type. I would say songs for a start. You give it a prompt and your local songs and it starts to train a model, which is then used to output a song (or a whole playlist). Depending on how well the song is rated by other users, nodes choose their learning group.
And the model changes are shared as gradients, not as whole models to reduce bandwidth.
Not based on flower ai but the following paper combines BitTorrent with gossip learning: https://dl.acm.org/doi/pdf/10.1145/3703790.3703813