This question proceeds from an assumption that the length of time a couple spends together makes it married. It doesn't. The state may have a provision for "common law marriages" under which a couple which lives together long enough becomes regarded by the state as married or pseudo-married, but God does not. If a couple lives together for years without getting married in God's eyes, the are still unmarried in God's eyes.
This is the principle which allows an annulment after a period of years. If a couple were never sacramentally married in God's eyes, the mere passage of time does not create the sacrament between them. Thus, if it can be established that their initial contracting of the marriage was invalid (due to an impediment or the fact that the couple did not exchange valid matrimonial consent), then, no matter how much time has elapsed, the two are not married because there never was a marriage in the first place.
Consider an analogy. A marriage is a special kind of contract (an especially serious kind known as a covenant), but, even with regular human contracts that bring about a partnership, if the contract were illegal for some reason then the partnership it attempted to bring about does not exist, no matter how long the two "partners" have been working together. In the same way, a marriage must be valid before God from the time it was contracted, or it is not valid at all and thus can be annulled-that is, determined to have been invalid from the beginning.