A Client Selection Method Based on Loss Function Optimization for Federated Learning

查看全文 作　　者：Yan [1,2,3]Zeng;Siyuan [1]Teng;Tian [4]Xiang;Jilin [1,2,3]Zhang;Yuankai [5]Mu;Yongjian [1,2,3]Ren;Jian [1,2,3]Wan 机构地区：[1]School of Computer Science and Technology,Hangzhou Dianzi University,Hangzhou,310018,China;[2]Key Laboratory of Complex System Modeling and Simulation,Ministry of Education,Hangzhou,310018,China;[3]Zhejiang Engineering Research Center of Data Security Governance,Hangzhou,310018,China;[4]Intelligent Robotics Research Center,Zhejiang Lab,Hangzhou,311100,China;[5]HDU-ITMO Joint Institute,Hangzhou Dianzi University,Hangzhou,310018,China 出　　处：《Computer Modeling in Engineering & Sciences》索引2023年第10期,共18页 基　　金：supported by the National Natural Science Foundation of China under Grant No.62072146;The Key Research and Development Program of Zhejiang Province under Grant No.2021C03187;National Key Research and Development Program of China 2019YFB2102100;The State Key Laboratory of Computer Architecture(ICT,CAS)under Grant No.CARCHB202120. 摘　　要：Federated learning is a distributedmachine learningmethod that can solve the increasingly serious problemof data islands and user data privacy,as it allows training data to be kept locally and not shared with other users.It trains a globalmodel by aggregating locally-computedmodels of clients rather than their rawdata.However,the divergence of local models caused by data heterogeneity of different clients may lead to slow convergence of the global model.For this problem,we focus on the client selection with federated learning,which can affect the convergence performance of the global model with the selected local models.We propose FedChoice,a client selection method based on loss function optimization,to select appropriate local models to improve the convergence of the global model.It firstly sets selected probability for clients with the value of loss function,and the client with high loss will be set higher selected probability,which can make them more likely to participate in training.Then,it introduces a local control vector and a global control vector to predict the local gradient direction and global gradient direction,respectively,and calculates the gradient correction vector to correct the gradient direction to reduce the cumulative deviationof the local gradient causedby theNon-IIDdata.Wemake experiments to verify the validity of FedChoice on CIFAR-10,CINIC-10,MNIST,EMNITS,and FEMNIST datasets,and the results show that the convergence of FedChoice is significantly improved,compared with FedAvg,FedProx,and FedNova. 关 键 词：Federated learning model aggregation Non-IID