There are climatological reasons for this, it seems. To make a long story short, sound is deflected downwards by the inversion which acts sort of like a lid on the bustle of the city in the morning. Up the mountain, one is above the inversion and hence above, literally, the madding crowd.
The phenomenon can be explained in equations, but I prefer Carl Sandburg’s poem Fog, from his Chicago Poems.
The fog comes on little cat feet.
It sits looking over harbor and city on silent haunches and then moves on.
Photo: Fog in the summer cemetery