Point is, accordign to description, its not "most used" but one you "recalled most time to". So, in other words, your most common destination.
My theory is that it somehow counts "enter the light" as teleporting to serbule (even tough it dont chnage your "last recall" target). That coudl explain it, sinc ei (and probably everyone else) use this method to get to serbule fast very often.