You may have heard of the idea of a genetic bottleneck. It is when genetic variation is lost by sampling a very small subset of a population. Just by chance, the frequencies of alleles on the subset of the population will be different to those in the whole population, and overall genetic variation is likely to be lost by population sampling.
This is why small population sizes are not good, and species on conservation lists are required to not drop to too small population sizes. The limited genetic variation remaining in a small population poses a risk because it makes the species susceptible to whatever relies on genetic variation for adaptation, such as environmental changes or diseases.
Here is a visualisation of a bottleneck.
A more recent way to think about bottlenecks is that you don't need a catastrophic reduction in population size to achieve them. Population expansion can have the same result, because expansion is usually from a subset of the population, for example the individuals in the edge of the species range.
Furthermore, if the expanded individuals establish a population locally, it is difficult for genetic variation to be reintroduced at the expanded range, by more individuals migrating from the original population centre, because there is no space for them by the time they arrive. That is why, in terms of genetic variation, there is “Southern richness and Northern purity” in European species (see Hewitt 2000).
It is quite obvious once you think about it.
But you don't need to think anymore! You can now run a simulation illustrating this on your browser. Head here and press the play button and see for yourself that the multicoloured centre of a disk, representing ancestral genetic variation, quickly turns into random sized unicolour pizza slices, i.e. genetic variation is lost by population growth as the disk grows.
The authors have a more formal explanation...