Summing up from the comments:
There's more to it then just getting rid of "Google Apps" – though running a device without them certainly already massively reduces that data collection. A complete elimination can only be achieved by running the device in airplane mode (i.e. not permitting any network connections).
Using a custom ROM without flashing GApps will massively reduce the data collection, yes. But then you also should avoid all apps from Playstore (e.g. by using F-Droid solely), and don't browse websites (only few, like mine, have no Google trackers involved). As @beeshyams already wrote, a Firewall (root: AFWall+ / no-root: NetGuard) can help reducing it further. Getting it to absolute zero is probably only possible by going into airplane mode.
Even if you disable Javascript completely in the browser, there are still Web bugs to consider: tiny, invisible images (1x1 pixel) embedded into web pages for tracking purposes. Websites which are completely free of trackers are rare; at least two thirds of all existing websites integrate at least one (if not more) Google Trackers: Google Analytics, Google Tag Manager (both to be found even here at SE), Google Ads, Google Fonts, miscellaneous Javascripts (ajax.googleapis.com), Google Maps, Youtube, and so on.
TL;DR: You can massively reduce it, but you can't get fully rid of it without breaking stuff. Using a custom ROM without GApps is a good start, avoiding apps from Playstore helps a lot, using a firewall app (AFWall+ if rooted, NetGuard otherwise) helps further. Selectively enable Javascript for websites in your browser (e.g. using the uBlock Origin addon) massively improves on it.