Midday and Midnight are not meaningless and have never been based on the numbers of the clock. Those are terms for "about halfway between sunrise and sunset" and "about half way between sunset and sunrise" respectively.
The measurement of elapsed time is a science. The time of day however is a number that represents how far along you are in one rotation of the planet, relative to an arbitrary 0 position +/- an offset.
The arbitrary 0 position has always been political.
I would say time of day is not completely arbitrary, but it also does not require exact alignment with physical phenomena to have meaning.
For example, if one were to say it is 12pm, people would understand roughly where we are in the diurnal cycle. People will understand it is not night time. People will know roughly where the sun is in the sky.
Rough information is still information. Information doesn't need exact boundaries and discrete rules to carry meaning.
The measurement of elapsed time is a science. The time of day however is a number that represents how far along you are in one rotation of the planet, relative to an arbitrary 0 position +/- an offset.
The arbitrary 0 position has always been political.