The overarching plot makes it inevitable. Candidates for TES5 were Summerset, Valenwood and Skyrim, but Summerset was eliminated when Crystal Tower fell and now Skyrim is obviously eliminated by its plot.
Every plot features the destruction of a "tower," a massive artefact tied with the world-bones that keeps the world from unraveling (or possibly, as we recently discovered going back over the lore on a place called Lyg, just keeping Tamriel from sinking into the ocean). Some of these towers were Numidium (destroyed in the events of Daggerfall), Red Mountain (deactivated when Nerevarine displaces the Heart of Lorkhan in Morrowind), White-Gold (deactivated as a sacrifice by Martin Septim when he becomes the Avatar of Akatosh in Oblivion), Crystal-Like-Law (or the Crystal Tower, destroyed behind the scenes during Oblivion's events by a legion of Dremora, possibly with Thalmor involvement) and Snow-Throat (or the Throat of the World, deactivated shortly before Skyrim's events, we don't know what its key or "stone" was but we know that it had to be destroyed in order for Alduin to return, so it's already down). Only two such towers remain, Green-Sap in Vardenfell (which is a giant walking tree, but put down roots between Oblivion and Skyrim, likely to defend itself from the Thalmor), and Ada-Mantia in High Rock. The latter is made of a virtually indestructible material and was the first Tower, as well as being, according to semi-canonical sources, a spaceship on which Akatosh landed on Nirn, so it's almost certainly going to be saved for last.