A few factors:

• Human population centers historically were built by natural waterways and/or by the sea, to enable access to trade, seafood, and obviously, water for drinking and agriculture
• When the fastest mode of land transport is a horse (ie no railways or automobiles), the long-distance roads between nations which existed up to the 1700s were generally unimproved and dangerous, both from the risk of breakdown but also highway robbery. Short-distance roads made for excellent invasion routes for an army, and so those tended to fall under control of the same nation.
• Water transport was (and still is) capable of moving large quantities of tonnage, and so was the predominant form of trade, only seeing competition when land transport improved and air transport was introduced.

So going back centuries when all the “local” roads are still within the same country (due to conquest), and all the long-distance roads were treacherous, slow, and usually uncomfortable (ie dysentery on the Oregon Trail), the most obvious way to get to another country would have been to get a ride on a trading ship. An island nation would certainly regard all other countries as being “overseas”, but so would an insular nation hemmed in by mountains but sitting directly on the sea. When land transport is limited, sea routes are the next best. And whereas roads only connect places situated along the route, the sea (and the sky) allow point-to-point trading, exposing faraway countries to each other when their ships arrive at the port.

TL;DR: for most of human history, other countries were most reasonably reached by sea. Hence “overseas”.