Since indexing is supplemental, it can be made a background task that can occur at more or less any time. So we run it as soon as possible.
Largely refactoring work to make the code somewhat nicer to work with.