I realized I can see all the addresses of people whose phone number I have on Google Maps. I doubt these people intentionally made their addresses available publicly.
How does the Google Maps app populate the places found under the My Places tab? Where did Google get these from?
As mentioned on Izzy's comment, Google Maps can read the addresses from your Google Contacts. The feature was added on April 27, 2016 (v9.25.1). From AndroidPolice,
Contacts Appear in Your Places and Maps
One of the biggest complaints about using Google Maps for navigation to a friend's house is that the address had to be manually saved or searched; otherwise, the only way to pull it up was through a contacts app. Forget those days because all of your friends' addresses are now listed under the LABELED page on the Your places screen.
Previously, this list contained standard entries for Home and Work, plus any locations with manually added labels. Those items are now followed by a list of addresses pulled from the contacts database. There doesn't appear to be any fast way to sift through them, so there are probably going to be a few more changes to this screen in future releases.
Each entry has an overflow menu with the option to hide the contact from Maps so it won't show up in this list or anywhere else in Maps, and another to remove the address from the contact entirely.
The contacts will also appear as landmarks while looking at maps. You'll have to be zoomed in reasonably close to see them, but they appear like any small business.
One important thing is, Google doesn't publicly show your address to everyone (exception: Google+). It's only taken from the contact info (whether it's right or wrong address), and only shown to you.
Q & A