I trained a distilbert , using 3 raspberry as clients , training only the final classifier ,while keeping base model frozen, after aggregation and after testing , there is small difference in the accuracy of local evaluation and centralized server evaluation by 4%,i used the same batch size in testing in both these evaluations.any one has an idea why there is a shift in accuracy.??
1 Like
There are several possibilities:
- Are you using the same evaluation dataset on both the server and the client? Using different datasets would naturally lead to different evaluation results.
- Are you evaluating the locally trained model on the client and the aggregated model on the server? If you use the same dataset on both the client and the server, but you evaluate the locally trained model on the client vs the aggregated model on the server, then you should also expect a difference in result.
Dear Daniel, thank you for the response. I was using a different dataset for local and server evaluation, which was causing this shift,the issues is now solved.
1 Like
This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.