How can we create a learning algorithm that generalises “out of distribution”, i.e. beyond the training data? What can we expect about errors when a model generalises OoD? And what should happen when an error is made? In this talk, we will discuss how OccamLab’s approach to these questions. We will discuss solutions that are prescribed by a Bayesian perspective, through work on learning invariance/equivariance properties, uncertainty quantification, and continual learning. We will discuss how the Bayesian point of view points in the same direction as perspective from Minimum Description Length (MDL) and Kolmogorov Complexity, and finish with some open problems to work and collaborate on.