From what I know, the populations were moved well after the 20s. It was a see-saw affair of "polite" "population exchange" almost decade by decade until the 1970s when the Albanian majority was cemented. Kosovo was depopulated of much of its Albanian population because of their support for the Nazis. At that point, and for several years, many Serb refugees moved in.

That province/country has been vexed for a very very long time.