How can I limit WiFi data usage to activities I explicitly allow?

If you don't have root access on the device, then you can use mobiwol to control the access various apps have to the Internet. Mobiwol creates a 'virtual' VPN connection on the device which allows control of the individual app connections. However, although the app starts when the device starts it is possible that there may be a brief period prior to Mobiwol executing in which apps could get Internet access despite the Mobiwol configuration.

Also there is NetGuard which can generally forbid an app access to WiFi or mobile data, permit an app access to WiFi or mobile data only when screen is on, regardless of above settings, keep it from accessing mobile data when roaming.

It's by far the best rated app in this section – and though I haven't tried it myself, I'd say it's the one to recommend if your device is not rooted.

If you have root access, there are more alternatives:

Remove the 'internet' permission using an app like Permissions Denied
If your kernel supports IPTables, you can use an app such as AFWall+ to control which apps have access to the internet.