How to install a firmware without losing your data and restore the previous one in case it goes wrong?


Question

I have a OnePlus 5T and I can't install OTA updates anymore because my firmware is too old. In addition to that I don't have a full backup of my device because I'm using encryption and TWRP doesn't understand it.



I would like to simply try the new firmware and revert in case anyhting goes wrong without losing any data. Is that possible?



I'm not exactly sure what firmware means in this context and where to get it, but looking at the website, there is this download for a zip file with the following contents:
file tree



I'm pretty sure that this is the complete stock ROM, meaning if I were to install it, it would wipe everything and replace my LineageOS with the stock ROM.



What I would like to do is to simply flash "the firmware" and only the firmware without losing data on my internal storage and also not my app data, contacts, sms, wifi passwords, bluetooth pairings etc etc.



And before that I would like to create a backup of my old firmware so that I can go back in case it goes wrong.



Can this be done? Can it be done with fastboot?



Maybe using fastboot flash PARTITION [FILENAME] and some other command to do the backup?



Edit:



The updater-script in META-INF/com/google/android/update-script contains the following code:



getprop("ro.display.series") == "OnePlus 5T" || abort("E3004: This package is for \"OnePlus 5T\" devices; this is a \"" + getprop("ro.display.series") + "\".");

is_part_existed("/dev/block/bootdevice/by-name/vendor") || abort("vendor partition is not existed, exit ota!!");
show_progress(0.650000, 0);
ui_print("Patching system image unconditionally...");
block_image_update("/dev/block/bootdevice/by-name/system", package_extract_file("system.transfer.list"), "system.new.dat", "system.patch.dat") ||
abort("E1001: Failed to update system image.");
show_progress(0.100000, 0);
ui_print("Patching vendor image unconditionally...");
block_image_update("/dev/block/bootdevice/by-name/vendor", package_extract_file("vendor.transfer.list"), "vendor.new.dat", "vendor.patch.dat") ||
abort("E2001: Failed to update vendor image.");
show_progress(0.050000, 10);
show_progress(0.050000, 5);
package_extract_file("boot.img", "/dev/block/bootdevice/by-name/boot");
show_progress(0.200000, 10);
ui_print("Writing static_nvbk image...");
package_extract_file("RADIO/static_nvbk.bin", "/dev/block/bootdevice/by-name/oem_stanvbk");

# ---- radio update tasks ----

ui_print("Patching firmware images...");
ifelse(msm.boot_update("main"), (
package_extract_file("firmware-update/cmnlib64.mbn", "/dev/block/bootdevice/by-name/cmnlib64");
package_extract_file("firmware-update/cmnlib.mbn", "/dev/block/bootdevice/by-name/cmnlib");
package_extract_file("firmware-update/hyp.mbn", "/dev/block/bootdevice/by-name/hyp");
package_extract_file("firmware-update/pmic.elf", "/dev/block/bootdevice/by-name/pmic");
package_extract_file("firmware-update/tz.mbn", "/dev/block/bootdevice/by-name/tz");
package_extract_file("firmware-update/abl.elf", "/dev/block/bootdevice/by-name/abl");
package_extract_file("firmware-update/devcfg.mbn", "/dev/block/bootdevice/by-name/devcfg");
package_extract_file("firmware-update/keymaster.mbn", "/dev/block/bootdevice/by-name/keymaster");
package_extract_file("firmware-update/xbl.elf", "/dev/block/bootdevice/by-name/xbl");
package_extract_file("firmware-update/rpm.mbn", "/dev/block/bootdevice/by-name/rpm");
), "");
ifelse(msm.boot_update("backup"), (
package_extract_file("firmware-update/cmnlib64.mbn", "/dev/block/bootdevice/by-name/cmnlib64bak");
package_extract_file("firmware-update/cmnlib.mbn", "/dev/block/bootdevice/by-name/cmnlibbak");
package_extract_file("firmware-update/hyp.mbn", "/dev/block/bootdevice/by-name/hypbak");
package_extract_file("firmware-update/tz.mbn", "/dev/block/bootdevice/by-name/tzbak");
package_extract_file("firmware-update/abl.elf", "/dev/block/bootdevice/by-name/ablbak");
package_extract_file("firmware-update/keymaster.mbn", "/dev/block/bootdevice/by-name/keymasterbak");
package_extract_file("firmware-update/xbl.elf", "/dev/block/bootdevice/by-name/xblbak");
package_extract_file("firmware-update/rpm.mbn", "/dev/block/bootdevice/by-name/rpmbak");
), "");
msm.boot_update("finalize");
package_extract_file("firmware-update/logo.bin", "/dev/block/bootdevice/by-name/LOGO");
package_extract_file("firmware-update/NON-HLOS.bin", "/dev/block/bootdevice/by-name/modem");
package_extract_file("firmware-update/adspso.bin", "/dev/block/bootdevice/by-name/dsp");
package_extract_file("firmware-update/BTFM.bin", "/dev/block/bootdevice/by-name/bluetooth");
set_progress(1.000000);


Should I just take the msm.boot_update("main"), and the msm.boot_update("finalize"), part and flash them like this?



# main
fastboot flash cmnlib64 ./firmware-update/cmnlib64.mbn
fastboot flash cmnlib ./firmware-update/cmnlib.mbn
fastboot flash hyp ./firmware-update/hyp.mbn
fastboot flash pmic ./firmware-update/pmic.mbn
fastboot flash tz ./firmware-update/tz.mbn
fastboot flash abl ./firmware-update/abl.elf
fastboot flash keymaster ./firmware-update/keymaster.mbn
fastboot flash xbl ./firmware-update/xbl.elf
fastboot flash rpm ./firmware-update/rpm.mbn

# finalize
fastboot flash LOGO ./firmware-update/logo.bin
fastboot flash modem ./firmware-update/NON-HLOS.bin
fastboot flash dsp ./firmware-update/adspso.bin
fastboot flash bluetooth ./firmware-update/BTFM.bin


Or would this already overwrite a partition that contains some sort of user data? Can I make a backup of these partitions first?
And what about RADIO/static_nvbk.bin, boot.bin, system.transfer.list, vendor.transfer.list and all that msm.boot_update("backup"), stuff?



Edit:
I ended up not being able to flash those firmware files via fastboot because it requires more than a "normal" unlocked bootloader. If I had done the necessary step to unlock the bootloader further it would have wiped all my data.
Fortunately you can bypass that by creating a zip with the firmware files and a custom OpenRecoveryScript (updater-script).
I found an awesome GitHub project that automates the whole process: https://github.com/angela-d/firmware_oneplus



I simply sideloaded the generated zip via adb sideload firmware-update-oneplus5T.zip and finally I was able to install my OTA updates.



https://github.com/angela-d/firmware_oneplus


Answer

1) You can boot into TWRP recovery from fastboot



fastboot boot twrp.img


2) Then you can backup your whole emmc from adb



adb pull /dev/block/mmcblk0


twrp_adb_pull_mmcblk0.bmp




Does /dev/block/mmcblk0 include my internal storage, my app data, contacts, sms, wifi passwords, bluetooth pairings, system settings etc? Have you personally tried that method on an encrypted device?




It is all stored in userdata partition. If TWRP is able to decrypt, you should pull /dev/block/dm-0 (=userdata) to get the unencrypted backup.



Otherwise, you still can separate encrypted USERDATA partition (and correlated partition EFS/METADATA required for decryption) from mmcblk0



Do a research how encryption works for your device, i can't help with which partitions are required!



3) print partition table with start/size



parted mmcblk0 unit B print



Warning: Not all of the space available to mmcblk0 appears to be used, you can fix the GPT to use all of the space (an extra 991 blocks) or continue with the current setting? 
Fix/Ignore? i
Model: (file)
Disk mmcblk0: 15300820992
Sector size (logical/physical): 512/512
Partition Table: gpt
Disk Flags:

Number Start End Size File system Name Flags
1 524288 3670015 3145728 proinfo msftdata
2 3670016 8912895 5242880 nvram msftdata
3 8912896 19398655 10485760 ext4 protect1 msftdata
4 19398656 29884415 10485760 ext4 protect2 msftdata
5 29884416 30146559 262144 seccfg msftdata
6 30146560 30539775 393216 lk msftdata
7 30539776 47316991 16777216 boot msftdata
8 47316992 64094207 16777216 recovery msftdata
9 64094208 64618495 524288 para msftdata
10 64618496 73007103 8388608 logo msftdata
11 73007104 83492863 10485760 expdb msftdata
12 83492864 84541439 1048576 frp msftdata
13 84541440 118095871 33554432 ext4 nvdata msftdata
14 118095872 159383551 41287680 metadata msftdata
15 159383552 161480703 2097152 oemkeystore msftdata
16 161480704 167772159 6291456 secro msftdata
17 167772160 176160767 8388608 keystore msftdata
18 176160768 2801795071 2625634304 ext4 system msftdata
19 2801795072 3070230527 268435456 ext4 cache msftdata
20 3070230528 15283519487 12213288960 ext4 userdata msftdata
21 15283519488 15300296703 16777216 flashinfo msftdata


4) copy the partitions into single files (partitions vary for each device model, this is just example for FDE. sometimes metadata is a file located at EFS partition, sometimes encryption footer is concatenated to userdata partition itself)



dd if=mmcblk0 of=metadata.bin skip=118095872 count=41287680 bs=1
dd if=mmcblk0 of=userdata.img skip=3070230528 count=12213288960 bs=1


this is just example for better understanding and is very slow (10 kB/s). of course dd will copy faster when we increase block size, so calculate bytes into 8k blocks



12213288960 / 8192 = 1490880



dd if=mmcblk0 of=metadata.bin skip=14416 count=5040 bs=8k
dd if=mmcblk0 of=userdata.img skip=374784 count=1490880 bs=8k


5) repeat this for all required partitions. for successful decryption system and vendor may required too. beware you can only decrypt with correlating rom on this unique device itself. decryption on other device (even same model won't work)



6) for restoring you can flash this partition dumps from fastboot



fastboot flash userdata userdata.img
fastboot flash metadata metadata.bin


another method is restoring from adb



adb push userdata.img /dev/block/bootdevice/by-name/userdata
adb push metadata.bin /dev/block/bootdevice/by-name/metadata


or even from within twrp terminal



dd if=/external_sd/userdata.img of=/dev/block/bootdevice/by-name/userdata
dd if=/external_sd/metadata.bin of=/dev/block/bootdevice/by-name/metadata


FIRMWARE UPDATE



for firmware update i recommend to flash complete stock rom (you will lose all data), then flash LineageOS, then restore twrp backup data.ext4.win* (which of course you have created from working TWRP before)



another method is using sdat2img to convert system.new.dat and vendor.new.dat into ext4 images system.img and vendor.img, which you can flash along with other partitions from fastboot



please also read the questions from Android Enthusiasts Linked and Related section


Topics


2D Engines   3D Engines   9-Patch   Action Bars   Activities   ADB   Advertisements   Analytics   Animations   ANR   AOP   API   APK   APT   Architecture   Audio   Autocomplete   Background Processing   Backward Compatibility   Badges   Bar Codes   Benchmarking   Bitmaps   Bluetooth   Blur Effects   Bread Crumbs   BRMS   Browser Extensions   Build Systems   Bundles   Buttons   Caching   Camera   Canvas   Cards   Carousels   Changelog   Checkboxes   Cloud Storages   Color Analysis   Color Pickers   Colors   Comet/Push   Compass Sensors   Conferences   Content Providers   Continuous Integration   Crash Reports   Credit Cards   Credits   CSV   Curl/Flip   Data Binding   Data Generators   Data Structures   Database   Database Browsers   Date &   Debugging   Decompilers   Deep Links   Dependency Injections   Design   Design Patterns   Dex   Dialogs   Distributed Computing   Distribution Platforms   Download Managers   Drawables   Emoji   Emulators   EPUB   Equalizers &   Event Buses   Exception Handling   Face Recognition   Feedback &   File System   File/Directory   Fingerprint   Floating Action   Fonts   Forms   Fragments   FRP   FSM   Functional Programming   Gamepads   Games   Geocaching   Gestures   GIF   Glow Pad   Gradle Plugins   Graphics   Grid Views   Highlighting   HTML   HTTP Mocking   Icons   IDE   IDE Plugins   Image Croppers   Image Loaders   Image Pickers   Image Processing   Image Views   Instrumentation   Intents   Job Schedulers   JSON   Keyboard   Kotlin   Layouts   Library Demos   List View   List Views   Localization   Location   Lock Patterns   Logcat   Logging   Mails   Maps   Markdown   Mathematics   Maven Plugins   MBaaS   Media   Menus   Messaging   MIME   Mobile Web   Native Image   Navigation   NDK   Networking   NFC   NoSQL   Number Pickers   OAuth   Object Mocking   OCR Engines   OpenGL   ORM   Other Pickers   Parallax List   Parcelables   Particle Systems   Password Inputs   PDF   Permissions   Physics Engines   Platforms   Plugin Frameworks   Preferences   Progress Indicators   ProGuard   Properties   Protocol Buffer   Pull To   Purchases   Push/Pull   QR Codes   Quick Return   Radio Buttons   Range Bars   Ratings   Recycler Views   Resources   REST   Ripple Effects   RSS   Screenshots   Scripting   Scroll Views   SDK   Search Inputs   Security   Sensors   Services   Showcase Views   Signatures   Sliding Panels   Snackbars   SOAP   Social Networks   Spannable   Spinners   Splash Screens   SSH   Static Analysis   Status Bars   Styling   SVG   System   Tags   Task Managers   TDD &   Template Engines   Testing   Testing Tools   Text Formatting   Text Views   Text Watchers   Text-to   Toasts   Toolkits For   Tools   Tooltips   Trainings   TV   Twitter   Updaters   USB   User Stories   Utils   Validation   Video   View Adapters   View Pagers   Views   Watch Face   Wearable Data   Wearables   Weather   Web Tools   Web Views   WebRTC   WebSockets   Wheel Widgets   Wi-Fi   Widgets   Windows   Wizards   XML   XMPP   YAML   ZIP Codes