So now you "just" need to memorize a lookup table holding information about time when your clocks was first set. And share said lookup table with any visitors. It sounds super convenient :-)
It is convenient. As you can see on this page, I was simply responding to someone else that I also don't change the clocks - like them. Other sibling posters commented similarly.
I'm sure you remember passwords, next to which the number 1 is quite easy to recall immediately. Don't make it complicated when it is easy.