Carbon-aware Federated Learning with Model Size Adaptation
Date
2024-07-23
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Developing machine learning models heavily depends on the availability of data. Establishing a responsible data economy and safeguarding data ownership are essential to facilitate learning from distinct, heterogeneous data sources without centralizing data. Federated learning (FL) provides a collaborative framework that enables model development using data from geographically distributed clients, each characterized by unique carbon footprints associated with varying energy sources which can lead to significant carbon emissions when learning from these decentralized data from edge clients like smart phones and IoT devices. This variability in carbon intensity poses a substantial challenge in striving for environmentally sustainable model training with minimal carbon emissions. This thesis introduces innovative carbon-aware strategies within FL to mitigate total carbon emissions through strategic client engagement and resource allocation. We propose two distinct methods: (1) clustering clients based on data distribution and offsetting high carbon emissions with those exhibiting lower emissions, implemented through a client similarity matrix (FedGreenCS), and (2) adapting model sizes based on the carbon intensity of client locations (FedGreen), employing model compression techniques. Our results affirm the effectiveness of both approaches in harmonizing model performance with environmental impact, underscoring their potential as sustainable solutions in distributed learning scenarios. We conduct a theoretical analysis of the trade-offs between carbon emissions and convergence accuracy, taking into account the carbon intensity disparities across different regions to optimally select parameters. Empirical studies reveal that model size adaptation significantly reduces the carbon footprints of FL, surpassing contemporary methods while maintaining competitive accuracy. This research also highlights the viability of client selection and model adaptation as sustainable strategies in distributed learning contexts.
Description
Keywords
federated learning, carbon emission, model compression, ordered dropout
Citation
Abbasi, A. (2024). Carbon-aware federated learning with model size adaptation (Master's thesis, University of Calgary, Calgary, Canada). Retrieved from https://prism.ucalgary.ca.