In this post Ziran Wang, an assistant professor in the College of Engineering and member of the Purdue Institute for a Sustainable Future, discusses his recently published research “Decentralized Federated Learning: A Survey and Perspective,” which appears in IEEE Internet of Things with the support of the Office of Naval Research, the NSF – Directorate for Computer and Information Science and Engineering, and the NSF – Division of Computer and Network Systems.
What did you want to know?
We aimed to explore the evolving landscape of federated learning (FL), specifically focusing on decentralized federated learning (DFL). FL, a decentralized learning paradigm proposed by Google researchers in 2016, shares model weights rather than user data, preserving user privacy. Our interest lies in DFL, a variant of FL that eliminates the need for a central server, allowing direct communication between clients. This survey seeks to understand the methodology, challenges, and various implementations of DFL. We aimed to compare DFL with centralized FL (CFL), examine different communication protocols, network topologies, and iteration orders in DFL, and identify potential application scenarios and future research directions. Additionally, we wanted to investigate the benefits and drawbacks of DFL in different settings and propose solutions to address the challenges faced in implementing DFL systems effectively.
What did you achieve?
Our comprehensive survey revealed that DFL can operate with various network topologies such as line, ring, mesh, star, and hybrid configurations, each with its own advantages and challenges. We identified multiple communication protocols including pointing, gossip, broadcast, and hybrid protocols, which determine how clients exchange model parameters and update their local models. DFL can follow sequential, random, cyclic, or parallel iteration orders, affecting the convergence and performance of the learning process. DFL has been successfully applied in diverse domains such as connected and automated vehicles, healthcare, industrial IoT, mobile services, unmanned aerial vehicles, satellites, social networks, and artificial general intelligence. Each application benefits from unique attributes of DFL like improved privacy, reduced communication overhead, and enhanced robustness against central server failures. The main challenges in DFL include high communication overhead, computational and storage burdens, cybersecurity vulnerabilities, lack of incentive mechanisms, and management issues. We proposed solutions such as model compression, selective client communication, and dynamic topology adjustments to address these challenges and improve the efficiency and security of DFL systems.
What is the impact of this research?
This research provides a detailed perspective on DFL, highlighting its potential to revolutionize collaborative learning and data privacy across various domains. By addressing the challenges and proposing solutions, this survey guides future research and development in DFL, promoting more efficient and secure implementations. Practitioners in fields such as healthcare, transportation, and IoT can leverage DFL to enhance their systems’ performance and security. Ultimately, DFL fosters a more inclusive and equitable approach to machine learning, enabling widespread collaboration without compromising user privacy. The insights and findings from this research contribute to the growing body of knowledge on DFL, offering a foundation for researchers and practitioners to develop and deploy more effective and robust federated learning systems. This, in turn, can lead to improved outcomes in various applications, from healthcare to smart cities, by ensuring that sensitive data remains private while still enabling powerful, data-driven insights and innovations.