This talk is an outline of statistical modeling, in which the objective is to extract information from a set of data by fitting models from a selected class. This means that no model is right or wrong or true or false. Instead the models have varying performance, which can be measured in terms of the yardstick, the bit. When properly formalized we can say that the data have so many bits of complexity, which breaks up into so many bits of information that can be learned with the class leaving the remaining amount as noise.

I'll discuss first such a formalization due to Kolmogorov in the algorithmic theory of information, which is easy and elegant, and then the more difficult but practical formalization in terms of parametric statistical model classes. Time permitting I'll discuss an application to the so-called denoising problem, where noise is defined as that part in the data that looks random in light of the model class selected and cannot be compressed by the means given.