How to stop update notifications for disabled apps on a Family-Link managed device?

Have you tried resetting the Play Store?

It is possible (although very unlikely) that the Play Store has failed to detect that the apps have been disabled.

Go to Settings → Apps → Play Store → Storage → Tap the Clear data button.

Note: This will reset all your Play Store settings (update settings, notification settings etc.) to default. You might have to adjust these settings manually again.

If the above steps still don't solve your problem, try the below steps —

First, you'll need to enable USB debugging from the phone settings and download the Android debugging platform-tools package on your PC.
The official documentation for this can be found here.

Then, connect your phone to your PC with a cable and make sure your phone is listed as device as given in the above documentation.

Uninstall the culprit apps for the primary user account

adb shell

This will bring up a command line shell prompt of your phone. Then enter the following commands —

pm uninstall --user 0 com.google.android.videos # Remove Play Movies

pm uninstall --user 0 com.google.android.music # Remove Play Music

pm uninstall --user 0 com.google.android.apps.magazines # Remove Play Newsstand

pm uninstall --user 0 com.google.android.apps.books # Remove Play Books

exit

The apps should not appear in your Play Store's apps list anymore.

EDIT - For Family Link managed devices (Based on OP's observations)

If the device is being managed by Google Family Link, then there is no way to access Developer Options on the managed device because Google Family Link makes advanced settings (like Developer Options, Install from unknown sources etc.) inaccessible on the managed device.

Also, as mentioned by OP in the comments below, if these settings are modified before enabling Family Link management, they are reset to their defaults as soon as management is enabled.

However, these advanced settings can be controlled by parents from the Family Link app for parents, and they can be accessed using the following steps —

Open the Family Link app for parents → Select your child → Look for your child's managed device → Tap Settings on the device card

Here, you should find controls for most of the settings which are not accessible from the managed device anymore. Look for the USB Debugging developer option and turn it on.

Then, by following the steps mentioned above to connect the managed device to a PC via ADB, you should be able to run the commands to get rid of the apps from the managed device.