The Modulus of Transfer

We show that some basic moduli of continuity $\delta$---which measure how fast target risk decreases as source risk decreases---appear to be at the root of many of the classical relatedness measures in transfer learning and related literature. Namely, bounds in terms of $\delta$ recover many of the existing bounds in terms of other measures of relatedness and can often be tighter. Interestingly, while $\delta$ itself might not be efficiently estimated, adaptive procedures exist---based on reductions to confidence sets---which can get nearly tight rates in terms of $\delta$ with no prior distributional knowledge. Such adaptivity to unknown $\delta$ immediately implies adaptivity to many classical relatedness notions. We additionally propose a new quantity, termed the strong modulus of transfer, which captures a further benefit of access to some target data.