Think about when the last time you used a wall clock or even a public clock pole/tower or something. My brain doesn’t even register them anymore, I just look at my phone and reset brain to whatever magic hand box says.
To think there used to be a time where you had to physically go outside and look at a clock tower if you wanted accurate ish local time
edit: hourly workers who live by the time card, disregard this post
What error are we talking about here?
I don’t think most people have a seconds clock up on their phone when they set their watch to their phone time (also meaning OP is correct that phone time is base time for people).
That means they aren’t syncing the seconds hand and there is ± 60 seconds of error. Also probably some error in ± 1 minute of hand placement error, especially for small watch faces, plus mechanical timing error for all but the high end mechanical watches. There are tons of sources of error. NTP time is accurate to the 10s of milliseconds, so around 3 orders of magnitude difference for most people.