Learning from Heterogeneous Sources

Federated and collaborative learning systems mark a shift from classical data analysis scenarios, where we view samples as coming from a single large underlying population. Instead, techniques for federated and collaborative learning necessitate new techniques for effectively and efficiently learning from multiple, siloed heterogeneous data sources. In recent years there has been a proliferation of methods for federated optimization and learning that aim to enable efficient, accurate training of machine learning models in practice in heterogeneous networks. However, our theoretical understanding of these approaches lags behind, with existing results either making strong assumptions on the problem setting or failing to adequately reflect the impressive empirical performance seen in practice. There is also a pressing need to more clearly define and characterize realistic forms of heterogeneity, and to develop a taxonomy of approaches in the field by carefully relating these definitions/assumptions to corresponding methodology. This workshop will draw broadly on the optimization, learning theory, and statistics communities to explore principled approaches for characterizing, modeling, adapting to, and analyzing the effect of heterogeneity in collaborative learning and analytics.