There are two different issues: For comparing competitors in a single race, only the variation of the pool size over the various lanes matters (and the quality of time keeping). For comparing competitors over several races (eg over time in different olympics) the variation over different swimming pools also matters. Generally, since swimming pools in different countries are built by different contractors, it's plausible that the variation between pools is higher than the variation between individual lane lengths. The time keeping tolerances are much smaller, so can be ignored, at least over the last 30-50 years I'd say, and anything older is not worth bothering with.

--
$E(X_t|F_s) = X_s,\quad t > s$