How long do you think the rights should last? Do you think the Poles in the postwar period with their talk of 'recovered territories' and the 'repatriation' of the German 'settlers' were correct?

Only a tiny minority of Germans would actually return to the birthplace of their great-parents. That said the Chech laws after 1945 didn´t distinguish between supporters of Hitler and people resisting him. So the laws might be a touch insensitive...
The number actively involved (as opposed to passively supporting) was a small minority of the population. In any case depending on how the question was asked a clear majority of the expellees wished to return into the sixties. A right of return was the official policy of the German government until Brandt, and remained the official policy of the expellee movement long after that. And if the expellees had been kept in refugee camps as second class citizens (Jordan, occuppied territories) or non-citizens (everywhere else) I suspect that plenty would still want to 'return.' Conversely, if the Arab governments had had a deliberate policy of scattering the refugees throughout the rest of the population, breaking up old local communities geographically, and providing financial aid and partial compensation for lost property I imagine we wouldn't be all that worried about the Palestinian refugee problem today. (To be exact the Bundesregierung had a dual, contradictory policy for the expellees. On the one hand the practical one described above. On the other a propaganda/cultural one seeking to preserve their local identities so that they would want to return - hence the large scale funding of the Landsmannschaft movement and its associated organizations.)

Re. the Baltics - you do realize that Vilnius had a Lithuanian population of at most 5% in 1939, and no I don't think the Poles (or rather their children and grandchildren) expelled from Lithuania, Belarus, and Ukraine have any rights with respect to those countries.

And as a final note, a poem by the Polish poet Anna Frajlich, about her home town of Szczecin, formerly Stettin. (translation mine)
City

A sharp western light
strikes my eyes
a different view greeted me
from my childhood windows
from one side gardens
stretching out in the distance
from the other end of a street
planted so thickly with lindens
that their crowns formed a tent
the light as in a tunnel
somewhere in the distance
round and promising
the city was not ours
but taken from others
who fled here in the wartime
chaos and left everything
either buried in the gardens
or covered by the rubble
or simply on the table
red wine in crystal glasses
unfinished
dark stains on the sides
the city was not ours
but it flowered for us
berries and apple trees
in countless gardens
violets and lilies
in the hedges' shadows
flowered
the city by the river
spilled out along the borders
and in this city one heard
various languages
--like the bushes -- transplanted
from east to west
someone drawled in from Wilno
someone's Lwow gentry maners
-- in a whisper someone still spoke in German
and the survivors' Yiddish
filled the streets
and on the riverbanks
the city slang
already grew like grass
among the stones
and it is such an image
that endures in my mind
sometimes somber and then again
full of summer glare
spring autumn in the smoke
of burning branches
the city of my childhood

taken from someone
so that someone else's childhood
would pass by elsewhere

My answer would be that it is because of the incredibly destabilizing effect of the Israeli-Palestinian conflict. But that only makes sense if the objective is to seek a resolution. And that brings me to the second paragraph about the Israeli consensus against a Palestinian right of return. Insisting on such a right is the equivalent of saying that there will be no peace for the forseeable future since an overwhelming majority of Israelis sees it as tantamount to the destruction of their country.  You could counter by saying the same applies in reverse to the Palestinians. I don't think that is the case, but if it is then there is no real hope for settling the conflict and international efforts should be limited to just keeping it at as low a level as possible.  Forcing Israel to accept a right of return strikes as absolutely insane - seeking the destruction of a nuclear armed, conventionally powerful state, smack in the middle of an important and volatile region is not exactly a recipe for world stability.  I'll also note that its been almost forty years since the UN Security Council abandoned its insistence on a right of return - switching to a conveniently vague demand for a 'just solution' to the refugee problem.

In the case of Germany I agree with you on one point - the expulsions took place in the immediate aftermath of Germany's crimes during WWII. Germans only began seeing it in that light in the sixties. In the first ten-fifteen years of the Bundesrepublik the Germans themselves were presented as the primary victims of the war. The interstate agreement (i.e. the Potsdam declaration) was made by the victorious great powers, Germany wasn't involved, nor was Poland a party to the agreement shifting its borders, though the ethnic cleansing of Germans did have the support of virtually all Poles. West Germany's constantly repeated official policy (repeated both verbally and in German law) was that the 1937 borders remained valid, that expellees retained ownership over all property in the Vertreibungsgebiete, that the compensation they received in the Lastenausgleich did not change that one bit, that the Poles in the former German lands were illegal squatters. With respect to the Sudetenland the policy was that Munich was not valid but that the expulsion was illegal and that the Sudeten Germans had a right to exercize their Selbstbestimmungsrecht - i.e. to vote to join Germany. The expellee movement rejected the idea that the Poles or Czechs had any claims on private property, using the analogy that no matter how long a thief keeps posession of his stolen goods, and even if they pass on to his children, they remain the property of the rightful owner and his heirs.  It was only in 1991 that Germany accepted the Potsdam borders de jure (1970 was only de facto since the constitutional court ruled that West Germany couldn't give de jure recognition to the postwar borders)

And no, I don't think Israel should offer compensation to the refugees, and if it should, then think for a second just what the proportionate compensation should be for a relatively bloodless (Kosovo level) ethnic cleansing, vs what Israel would deserve from Germany, or indeed other nations complicit in one degree or another in the Holocaust. Or what the Armenians should get from the Turks, or pretty much every nation of the former (and in the case of the Caucasus - current) Russian colonial empire.

The hellish mess of the Israeli-Palestinian conflict is in large part the direct result of the attitudes and actions of European societies. Israel should withdraw to borders based on the 1967 status quo ante with one-for one compensation for any (minor) adjustments, but that's it. Paying for the monetary cost of making the agreement work should be the rest of the world's problem, and particularly that of Europe.