How to bind mount a folder inside /sdcard with correct permissions?


Question

I am using a Sony Xperia M4 Aqua. As is well known, the internal memory is rather small. Especially the Media directory of WhatsApp uses a lot of precious space, therefore I am trying to move it to the SD card.
I am using Android 6 and I have formatted the SD card to have an adopted storage partition.



Adopted storage would be normally used to migrate all /data to it, as discussed here.
I am however interested in moving just the single directory WhatsApp/Media somewhere else (possibly in the adopted storage partition), and then bind-mount it to its original location.



For this purpose, I moved the WhatsApp/Media directory to the adopted storage. Then, following this discussion, I have modified the script /system/etc/init.qcom.post_boot.sh adding the following mountcommands (the phone has no support for init.d scripts)



mount -o bind /mnt/expand/4fdb2500-9aa7-44bc-a2c4-80aeae28e764/WhatsAppMedia /storage/emulated/0/WhatsApp/Media
mount -o bind /mnt/expand/4fdb2500-9aa7-44bc-a2c4-80aeae28e764/WhatsAppMedia /mnt/runtime/write/emulated/0/WhatsApp/Media
mount -o bind /mnt/expand/4fdb2500-9aa7-44bc-a2c4-80aeae28e764/WhatsAppMedia /mnt/runtime/read/emulated/0/WhatsApp/Media
mount -o bind /mnt/expand/4fdb2500-9aa7-44bc-a2c4-80aeae28e764/WhatsAppMedia /mnt/runtime/default/emulated/0/WhatsApp/Media
mount -o bind /mnt/expand/4fdb2500-9aa7-44bc-a2c4-80aeae28e764/WhatsAppMedia /data/media/0/WhatsApp/Media


Notice: /mnt/expand/4fdb25.... points to the adopted storage partition.



This only apparently works: if I open a shell with adb I can correctly see that the WhatsApp/Media directory contains the mounted directory. Also, I see no additional views that contains the WhatsApp directory, as can be checked by doing, in an adb shell



find / -type d -name WhatsApp


Nevertheless, WhatsApp is not able to access the Media gallery. For instance, in the chat I just see blurred pictures (like a preview), and clicking on it I do not see the full picture. Moreover, if somebody sends me a picture, all I can see is a blurred preview with the download icon. Clicking on the download icon does not produce anything.



Wrong permissions are presumably the source of problems. For example, some permissions/group ownership appear to be incorrect on some views:



root@E2303:/ # ls -n /storage/emulated/0/WhatsApp
drwxrwx--x 0 1015 2019-09-04 14:37 Backups
drwxrwx--x 0 1015 2019-10-19 02:00 Databases
drwxrwx--x 0 1015 2019-09-04 20:24 Media
root@E2303:/ # ls -n /mnt/runtime/write/emulated/0/WhatsApp
drwxrwx--- 0 9997 2019-09-04 14:37 Backups
drwxrwx--- 0 9997 2019-10-19 02:00 Databases
drwxrwx--x 0 1015 2019-09-04 20:24 Media
root@E2303:/ # ls -n /mnt/runtime/read/emulated/0/WhatsApp
drwxr-x--- 0 9997 2019-09-04 14:37 Backups
drwxr-x--- 0 9997 2019-10-19 02:00 Databases
drwxrwx--x 0 1015 2019-09-04 20:24 Media
root@E2303:/ # ls -n /mnt/runtime/default/emulated/0/WhatsApp
drwxrwx--x 0 1015 2019-09-04 14:37 Backups
drwxrwx--x 0 1015 2019-10-19 02:00 Databases
drwxrwx--x 0 1015 2019-09-04 20:24 Media
root@E2303:/ # ls -n /data/media/0/WhatsApp/
drwxrwxr-x 1023 1023 2019-09-04 14:37 Backups
drwxrwxr-x 1023 1023 2019-10-19 02:00 Databases
drwxrwx--x 0 1015 2019-09-04 20:24 Media


Permissions and group ownership of the Media directory should be the same of the other (non bind-mounted) directories Backups and Databases.
Oddly enough, an attempt to remount the directories with the correct gid



root@E2303:/ # mount -o remount, gid=9997 /mnt/runtime/write/emulated/0/WhatsApp/Media
root@E2303:/ # mount -o remount, gid=9997 /mnt/runtime/read/emulated/0/WhatsApp/Media
root@E2303:/ # mount -o remount, gid=1023 /data/media/0/WhatsApp/Media


does not produce any changes in the group ownership: ls -n as above gives identical results.



Even more strange, but maybe unrelated, omitting the space between remount and gid=... results in



mount: Invalid argument


How to bind mount WhatsApp/Media folder from external SD card with correct permissions?


Answer

I have been using two different approaches (in fact many with small differences) on my older Android versions to mount whole /sdcard/WhatsApp directory from external SD card. I have tested, it works on Android 9 too, but the storage things have changed on Android 10.



Before getting into practical details, we need to keep in mind a few points:




  • /sdcard isn't an actual but emulated filesystem. Android uses sdcardfs (or FUSE) to emulate actual storage on /sdcard. See What is /storage/emulated/0/?

  • Both of the above filesystems have a fixed SELinux context: u:object_r:sdcardfs:s0 (or u:object_r:fuse:s0).

  • Normally /data/media is emulated over /sdcard, but in case of Adoptable Storage when data is migrated, /mnt/expand/[UUID]/media is emulated. See How to free Internal Storage by moving data or using symlink / bind-mount with Adoptable Storage?

  • Files and directories on /sdcard have fixed ownership and permissions which depend on if the app has android.permission.[READ|WRITE]_EXTERNAL_STORAGE permission granted or not. Files have never executable permission. Apps' data directories in /sdcard/Android/data/ have ownership set to respective app's UID. For details see What is the “u#_everybody” UID?



    We assume here that every app is allowed to write to /sdcard by setting ownership 0/9997 (user/group) and permissions 0771/0660 (directories/files).


  • To achieve above said behavior, since Android 6 every app is run in an isolated mount namespace and /storage/emulated is bind mounted to a different VIEW: /mnt/runtime/[default|read|write]/emulated with private/slave mount propagation. So mounting directly to /storage/emulated won't appear in apps' mount namespaces unless you enter every app's mount namespace explicitly. The same is true if you mount from some app's isolated mount namespace. See Partition gets unmounted automatically in Android Oreo.



    We are going to mount from root mount namespace to /mnt/runtime/write/emulated which is propagated to all apps' mount namespaces.


  • read and default views have different permissions than write (1), but mounting to both is usually unnecessary. Permissions READ_EXTERNAL_STORAGE and WRITE_EXTERNAL_STORAGE belong to same permission group. So granting one to an app through GUI also grants the other, and all apps with Storage permission will only see write view. default is only to let apps (which don't have READ/WRITE Storage permission) traverse /sdcard/Android/data directories. So mounting to default will let such apps just pass through subdirectories on /sdcard/, no files will be visible.



    Also at least with sdcardfs emulation, read and write are bind-mounted (2) from default and mounting to one also mounts to other two. So it's not possible to mount all of three with different permissions.


  • /sdcard does not support Extended Attributes (xattr) and Access Time (atime). Other mount options include nosuid, nodev and noexec. See mount manpage for details.






First of all make sure you are in root mount namespace as explained in the link given above. Or use nsenter to get a root shell in global namespace:



~# [ $(readlink /proc/1/ns/mnt) = $(readlink /proc/self/ns/mnt) ] || busybox nsenter -t 1 -m /system/bin/sh


1. MOUNT SD CARD MANUALLY:



The straightforward way is to format external SD card as portable storage with exFAT or FAT32. Since these filesystems aren't native to Linux, their in-kernel driver implementations support uid, gid, fmask and dmask mount options. You can use exfat or sdfat drivers with exFAT and vfat with FAT32. Former has also a userspace implementation mount.exfat-fuse which requires only FUSE support from kernel. Check with grep fuse /proc/filesystems.



Let's say /dev/block/sda1 is your exFAT partition:



~# mount -t exfat -o nosuid,nodev,noexec,noatime,context=u:object_r:sdcardfs:s0,uid=0,gid=9997,fmask=0117,dmask=0006 /dev/block/sda1 /mnt/runtime/write/emulated/0/WhatsApp
~# mv /data/media/0/WhatsApp/* /sdcard/WhatsApp/


* Replace u:object_r:sdcardfs:s0 with u:object_r:fuse:s0 or whatever label your /sdcard has.



You can also create multiple partitions on SD card. Or after mounting with required mount options you may also bind mount a directory instead of whole partition. Let's say you first mount SD card to /mnt/my_sdcard, then bind mount WhatsApp directory:



~# mount -o bind /mnt/my_sdcard/WhatsApp /mnt/runtime/write/emulated/0/WhatsApp


Downside with this approach is that vold mounts external SD card on boot, so you need to un-mount first.

Secondly the data on SD card isn't encrypted, though there are multiple ways to encrypt manually. See Decrypting microSD card on another Android device or desktop computer.



2. ADOPTABLE STORAGE:



An easy way to counter the above said downsides is to make use of kernel's built-in FDE by formatting the SD card as Adoptable Storage, but only if you don't want to migrate all /data/media/ to external SD card. Once formatted, external SD card will be mounted at /mnt/expand/[UUID] (filesystem UUID is a 16 bytes number). But we can't simply bind mount a directory from there to /sdcard because the filesystem on Adoptable Storage is ext4, which follows UNIX permissions model but the apps can't handle those as explained above. Even if you make it work somehow (using chown, chmod etc.), every app will create files with its own UID which won't be accessible to other apps e.g. Gallery may not be able see pictures downloaded by WhatsApp.



For this method to work your kernel must support sdcardfs (check with grep sdcardfs /proc/filesystems). Create a directory on Adopted SD card and emulate it to /sdcard/WhatsApp:



~# mkdir /mnt/expand/[UUID]/media/0/WhatsApp
~# mv /sdcard/WhatsApp/* /mnt/expand/[UUID]/media/0/WhatsApp/
~# restorecon -rv /mnt/expand/[UUID]/media/
~# mount -t sdcardfs -o nosuid,nodev,noexec,noatime,mask=7,gid=9997 /mnt/expand/[UUID]/media/0/WhatsApp /mnt/runtime/write/emulated/0/Whatsapp


Please note that we necessarily need to use path /mnt/expand/[UUID]/media/ because it's labeled as media_rw_data_file (3) (like /data/media (4)) which is allowed by SELinux policy to be accessed by apps (5). If you use a different path, you need to modify SELinux policy. Unlike other filesystems sdcardfs itself doesn't change SELinux context when accessing underlying filesystem.



3. PORTABLE / ADOPTABLE STORAGE:



This method is the most flexible, it works if you want to use:




  • Portable storage but don't want to mount whole partition, instead a directory.

  • Adoptable storage but your kernel doesn't have sdcardfs support.

  • Adoptable storage but don't want to use /mnt/expand/[UUID]/media/ path necessarily.



Use a third party tool named bindfs (which uses FUSE) to simulate the behavior of emulated filesystem. Actually Android's built-in tool /system/bin/sdcard does exactly this on pre-sdcardfs releases but it has some fixed paths and other unnecessary things, so its source code needs to be modified to achieve what we want. You can build bindfs yourself or try this this one.



~# DIR=/mnt/media_rw/[UUID]
 # for Portable Storage
~# DIR=/mnt/expand/[UUID] # for Adoptable Storage
~# mkdir $DIR/WhatsApp
~# mv /sdcard/WhatsApp/* $DIR/WhatsApp/
~# bindfs -o nosuid,nodev,noexec,noatime,context=u:object_r:sdcardfs:s0 -u 0 -g 9997 -p a-rwx,ug+rw,ugo+X --create-with-perms=a-rwx,ug+rw,ugo+X --xattr-none --chown-ignore --chgrp-ignore --chmod-ignore $DIR/WhatsApp /mnt/runtime/write/emulated/0/WhatsApp


Side notes:




  • sdcardfs also works partially with this method except it doesn't support SELinux context= option (so far). So it depends on what is the SELinux label of backing directory on SD card.

  • Other tools like rclone, encfs, sshfs etc. which make use of FUSE can also be mounted inside /sdcard the same way. Related: How to mount rclone on Android?

  • -u and -g options require /etc/passwd and /etc/group to exist on bindfs < v1.14.2.






So you may go with whatever method suits you. Usually in-kernel drivers perform better than userspace solutions, and methods native to Linux kernel are more robust and stable. FUSE (over FUSE) may exert performance penalty sometimes e.g. if SD card itself supports high speed data transfers.



You can place the required mount commands in an init.d script or define and init service. See How to run an executable on boot and keep it running?



Note:




  • Apps which don't scan /sdcard filesystem themselves but rely on Android's MediaProvider for any changes may need a forced media scan for new files on mounted filesystem to appear immediately.

  • If you use multiple users or profiles, you need to mount new filesystem for every User_ID. For device owner it's 0.






RELATED:




Topics


2D Engines   3D Engines   9-Patch   Action Bars   Activities   ADB   Advertisements   Analytics   Animations   ANR   AOP   API   APK   APT   Architecture   Audio   Autocomplete   Background Processing   Backward Compatibility   Badges   Bar Codes   Benchmarking   Bitmaps   Bluetooth   Blur Effects   Bread Crumbs   BRMS   Browser Extensions   Build Systems   Bundles   Buttons   Caching   Camera   Canvas   Cards   Carousels   Changelog   Checkboxes   Cloud Storages   Color Analysis   Color Pickers   Colors   Comet/Push   Compass Sensors   Conferences   Content Providers   Continuous Integration   Crash Reports   Credit Cards   Credits   CSV   Curl/Flip   Data Binding   Data Generators   Data Structures   Database   Database Browsers   Date &   Debugging   Decompilers   Deep Links   Dependency Injections   Design   Design Patterns   Dex   Dialogs   Distributed Computing   Distribution Platforms   Download Managers   Drawables   Emoji   Emulators   EPUB   Equalizers &   Event Buses   Exception Handling   Face Recognition   Feedback &   File System   File/Directory   Fingerprint   Floating Action   Fonts   Forms   Fragments   FRP   FSM   Functional Programming   Gamepads   Games   Geocaching   Gestures   GIF   Glow Pad   Gradle Plugins   Graphics   Grid Views   Highlighting   HTML   HTTP Mocking   Icons   IDE   IDE Plugins   Image Croppers   Image Loaders   Image Pickers   Image Processing   Image Views   Instrumentation   Intents   Job Schedulers   JSON   Keyboard   Kotlin   Layouts   Library Demos   List View   List Views   Localization   Location   Lock Patterns   Logcat   Logging   Mails   Maps   Markdown   Mathematics   Maven Plugins   MBaaS   Media   Menus   Messaging   MIME   Mobile Web   Native Image   Navigation   NDK   Networking   NFC   NoSQL   Number Pickers   OAuth   Object Mocking   OCR Engines   OpenGL   ORM   Other Pickers   Parallax List   Parcelables   Particle Systems   Password Inputs   PDF   Permissions   Physics Engines   Platforms   Plugin Frameworks   Preferences   Progress Indicators   ProGuard   Properties   Protocol Buffer   Pull To   Purchases   Push/Pull   QR Codes   Quick Return   Radio Buttons   Range Bars   Ratings   Recycler Views   Resources   REST   Ripple Effects   RSS   Screenshots   Scripting   Scroll Views   SDK   Search Inputs   Security   Sensors   Services   Showcase Views   Signatures   Sliding Panels   Snackbars   SOAP   Social Networks   Spannable   Spinners   Splash Screens   SSH   Static Analysis   Status Bars   Styling   SVG   System   Tags   Task Managers   TDD &   Template Engines   Testing   Testing Tools   Text Formatting   Text Views   Text Watchers   Text-to   Toasts   Toolkits For   Tools   Tooltips   Trainings   TV   Twitter   Updaters   USB   User Stories   Utils   Validation   Video   View Adapters   View Pagers   Views   Watch Face   Wearable Data   Wearables   Weather   Web Tools   Web Views   WebRTC   WebSockets   Wheel Widgets   Wi-Fi   Widgets   Windows   Wizards   XML   XMPP   YAML   ZIP Codes