Introduction
Anyone that already tried to root their phone using Magisk's Patching Images method would have already encountered this issue.
Magisk's Patching Images method ask you to copy boot.img or recovery.img (depending on your boot ramdisk availability), but when you extract the ZIP archive of your favorite ROM you see only something like this:
1 2 3 4 5 6 7 META-INF/com/android/metadata META-INF/com/android/metadata.pb apex_info.pb care_map.pb payload.bin payload_properties.txt META-INF/com/android/otacert
Newer version of Android OTA packages come with a payload.bin file that contains the individual image files of different firmware partitions.
payload.bin appeared when Google introduced the concept of A/B seamless system updates (Android Oreo (8)).
A/B update mechanism introduced a new format for the Android OTA/firmware update files. Earlier, these update packages came with partition images, which could easily be extracted and used for manual flashing. Nowadays, these update packages come with a single payload.bin file that contains the individual partition images.
Various open-source tools allow to extract the images from payload.bin.
For the sake of this examples I extracted a MIUI ROM (miui_ALIOTHGlobal_V13.0.3.0.SKHMIXM_07afcb74cd_12.0.zip) to a folder named miui.
This article will cover only the case of full OTA (not incremental OTA).
My hardware (for extraction time comparison):
CPU: AMD Ryzen 5 1600X 6c/6t
Memory: 32 GB
Disk: Medium SATA SSD
vm03/payload_dumper
vm03/payload_dumper is a python script.
The requirements, outside of Python 3 itself, are:
To install them on ArchLinux:
1 $ sudo pacman -S python-protobuf python-six --needed --asdeps && pikaur -S python-bsdiff4 --asdeps
With pip (I advice using a virtual environment ):
1 $ pip install --user protobuf six bsdiff4
There is also a docker image.
Beware of the size of storage available you need to be able to extract, for example with my MIUI ROM:
the unextracted ZIP weight 3.2 GB
the files once extracted have roughly the same size of 3.2 GB
the extracted images weight 7.6 GB
Launching the extraction is easy:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 $ python payload_dumper/payload_dumper.py miui/payload.bin --out payload_dumper_test Processing abl partition.Done Processing aop partition.Done Processing bluetooth partition.Done Processing boot partition................................................................Done Processing cmnlib partition.Done Processing cmnlib64 partition.Done Processing devcfg partition.Done Processing dsp partition................................Done Processing dtbo partition................Done Processing featenabler partition.Done Processing hyp partition.Done Processing imagefv partition.Done Processing keymaster partition.Done Processing modem partition.........................................................................................................................................Done Processing odm partition................................................................Done Processing product partition........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................Done Processing qupfw partition.Done Processing system partition....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................Done Processing system_ext partition..........................................................................................................................................................................................................................................................................................Done Processing tz partition..Done Processing uefisecapp partition.Done Processing vbmeta partition.Done Processing vbmeta_system partition.Done Processing vendor partition......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ..................................................................................................................................................Done Processing vendor_boot partition................................................Done Processing xbl partition..Done Processing xbl_config partition.Done
Extraction time on my machine: 5m 54s .
If you are interested in a few images only, you can extract only those to save a lot of time and space:
1 2 3 $ python payload_dumper/payload_dumper.py miui/payload.bin --out payload_dumper_test --images boot,vendor_boot Processing boot partition................................................................Done Processing vendor_boot partition................................................Done
Pros:
Works
Can extract individual images
Cons:
Slow (single-threaded)
Can't list images
LineageOS/scripts
LineageOS/scripts is a collection of Android python scripts from LineageOS.
This time we only requires protobuf as a dpendency.
Before extracting you can list the available images:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 $ python scripts/update-payload-extractor/extract.py miui/payload.bin --list_partitions abl (208896 bytes) aop (204800 bytes) bluetooth (421888 bytes) boot (134217728 bytes) cmnlib (397312 bytes) cmnlib64 (516096 bytes) devcfg (57344 bytes) dsp (67108864 bytes) dtbo (33554432 bytes) featenabler (90112 bytes) hyp (446464 bytes) imagefv (2097152 bytes) keymaster (282624 bytes) modem (285831168 bytes) odm (134217728 bytes) product (1693982720 bytes) qupfw (57344 bytes) system (3363110912 bytes) system_ext (590237696 bytes) tz (3190784 bytes) uefisecapp (126976 bytes) vbmeta (8192 bytes) vbmeta_system (4096 bytes) vendor (1733591040 bytes) vendor_boot (100663296 bytes) xbl (3563520 bytes) xbl_config (102400 bytes)
Launching the extraction is similar to the previous solution:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 $ python scripts/update-payload-extractor/extract.py miui/payload.bin --output_dir ./lineageos_extractor Extracting abl Extracting aop Extracting bluetooth Extracting boot Extracting cmnlib Extracting cmnlib64 Extracting devcfg Extracting dsp Extracting dtbo Extracting featenabler Extracting hyp Extracting imagefv Extracting keymaster Extracting modem Extracting odm Extracting product Extracting qupfw Extracting system Extracting system_ext Extracting tz Extracting uefisecapp Extracting vbmeta Extracting vbmeta_system Extracting vendor Extracting vendor_boot Extracting xbl Extracting xbl_config
Extraction time on my machine: 3m 49s .
If you are interested in a few images only, you can extract only those to save a lot of time and space:
1 2 3 $ python scripts/update-payload-extractor/extract.py miui/payload.bin --output_dir ./lineageos_extractor --partitions boot vendor_boot Extracting boot Extracting vendor_boot
Pros:
Works
Can extract individual images
Can list images
Cons:
ssut/payload-dumper-go
ssut/payload-dumper-go is a tool written in go.
The only system dependency is xz (LZMA lib).
To install it on ArchLinux:
1 $ sudo pacman -Syu xz --needed --asdeps
For Ubuntu:
1 $ sudo apt-get install liblzma-dev
There is also a Docker image.
If you get trouble compiling with go you can either use the Docker image or
download the prebuild binary .
Before extracting you can list the available images:
1 2 3 4 5 6 7 $ ./payload-dumper-go -l miui/payload.bin payload.bin: miui/payload.bin Payload Version: 2 Payload Manifest Length: 223511 Payload Manifest Signature Length: 267 Found partitions: abl (209 kB), aop (205 kB), bluetooth (422 kB), boot (134 MB), cmnlib (397 kB), cmnlib64 (516 kB), devcfg (57 kB), dsp (67 MB), dtbo (34 MB), featenabler (90 kB), hyp (446 kB), imagefv (2.1 MB), keymaster (283 kB), modem (286 MB), odm (134 MB), product (1.7 GB), qupfw (57 kB), system (3.4 GB), system_ext (590 MB), tz (3.2 MB), uefisecapp (127 kB), vbmeta (8.2 kB), vbmeta_system (4.1 kB), vendor (1.7 GB), vendor_boot (101 MB), xbl (3.6 MB), xbl_config (102 kB)
Launching the extraction is similar to the previous solution, but payload-dumper-go is multi-threaded and will use 4 parallel workers by default (that can be tweaked with the -c option).
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 $ ./payload-dumper-go -o payload-dumper-go_test miui/payload.bin payload.bin: miui/payload.bin Payload Version: 2 Payload Manifest Length: 223511 Payload Manifest Signature Length: 267 Found partitions: abl (209 kB), aop (205 kB), bluetooth (422 kB), boot (134 MB), cmnlib (397 kB), cmnlib64 (516 kB), devcfg (57 kB), dsp (67 MB), dtbo (34 MB), featenabler (90 kB), hyp (446 kB), imagefv (2.1 MB), keymaster (283 kB), modem (286 MB), odm ( 134 MB), product (1.7 GB), qupfw (57 kB), system (3.4 GB), system_ext (590 MB), tz (3.2 MB), uefisecapp (127 kB), vbmeta (8.2 kB), vbmeta_system (4.1 kB), vendor (1.7 GB), vendor_boot (101 MB), xbl (3.6 MB), xbl_config (102 kB) Number of workers: 4 aop (205 kB) [============================================================================================================================================================================================================] 100 % bluetooth (422 kB) [============================================================================================================================================================================================================] 100 % abl (209 kB) [============================================================================================================================================================================================================] 100 % boot (134 MB) [============================================================================================================================================================================================================] 100 % cmnlib (397 kB) [============================================================================================================================================================================================================] 100 % cmnlib64 (516 kB) [============================================================================================================================================================================================================] 100 % devcfg (57 kB) [============================================================================================================================================================================================================] 100 % dsp (67 MB) [============================================================================================================================================================================================================] 100 % dtbo (34 MB) [============================================================================================================================================================================================================] 100 % featenabler (90 kB) [============================================================================================================================================================================================================] 100 % hyp (446 kB) [============================================================================================================================================================================================================] 100 % imagefv (2.1 MB) [============================================================================================================================================================================================================] 100 % keymaster (283 kB) [============================================================================================================================================================================================================] 100 % modem (286 MB) [============================================================================================================================================================================================================] 100 % odm (134 MB) [============================================================================================================================================================================================================] 100 % product (1.7 GB) [============================================================================================================================================================================================================] 100 % qupfw (57 kB) [============================================================================================================================================================================================================] 100 % system (3.4 GB) [============================================================================================================================================================================================================] 100 % system_ext (590 MB) [============================================================================================================================================================================================================] 100 % tz (3.2 MB) [============================================================================================================================================================================================================] 100 % uefisecapp (127 kB) [============================================================================================================================================================================================================] 100 % vbmeta (8.2 kB) [============================================================================================================================================================================================================] 100 % vbmeta_system (4.1 kB) [============================================================================================================================================================================================================] 100 % vendor (1.7 GB) [============================================================================================================================================================================================================] 100 % vendor_boot (101 MB) [============================================================================================================================================================================================================] 100 % xbl (3.6 MB) [============================================================================================================================================================================================================] 100 % xbl_config (102 kB) [============================================================================================================================================================================================================] 100
Extraction time on my machine: 2m 6s .
If you are interested in a few images only, you can extract only those to save a lot of time and space:
1 2 3 4 5 6 7 8 9 10 $ ./payload-dumper-go -o payload-dumper-go_test -p boot,vendor_boot miui/payload.bin payload.bin: miui/payload.bin Payload Version: 2 Payload Manifest Length: 223511 Payload Manifest Signature Length: 267 Found partitions: abl (209 kB), aop (205 kB), bluetooth (422 kB), boot (134 MB), cmnlib (397 kB), cmnlib64 (516 kB), devcfg (57 kB), dsp (67 MB), dtbo (34 MB), featenabler (90 kB), hyp (446 kB), imagefv (2.1 MB), keymaster (283 kB), modem (286 MB), odm (134 MB), product (1.7 GB), qupfw (57 kB), system (3.4 GB), system_ext (590 MB), tz (3.2 MB), uefisecapp (127 kB), vbmeta (8.2 kB), vbmeta_system (4.1 kB), vendor (1.7 GB), vendor_boot (101 MB), xbl (3.6 MB), xbl_config (102 kB) Number of workers: 4 vendor_boot (101 MB) [==============================================================================================================================================================================================================] 100 % boot (134 MB) [==============================================================================================================================================================================================================] 100
Pros:
Works
Can extract individual images
Can list images
Fast (multi-threaded)
Cons:
Can be hard to build from sources
tobyxdd/android-ota-payload-extractor
tobyxdd/android-ota-payload-extractor is a tool written in go.
If you get trouble compiling with go you can download the prebuild binary .
Launching the extraction is easy:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 $ ./android-ota-extractor miui/payload.bin 2022/08/03 21:55:07 Parsing payload... 2022/08/03 21:55:07 Block size: 4096, Partition count: 27 2022/08/03 21:55:07 Extracting abl (1 ops) ... 2022/08/03 21:55:07 Extracting aop (1 ops) ... 2022/08/03 21:55:07 Extracting bluetooth (1 ops) ... 2022/08/03 21:55:07 Extracting boot (64 ops) ... 2022/08/03 21:55:09 Extracting cmnlib (1 ops) ... 2022/08/03 21:55:10 Extracting cmnlib64 (1 ops) ... 2022/08/03 21:55:10 Extracting devcfg (1 ops) ... 2022/08/03 21:55:10 Extracting dsp (32 ops) ... 2022/08/03 21:55:11 Extracting dtbo (16 ops) ... 2022/08/03 21:55:11 Extracting featenabler (1 ops) ... 2022/08/03 21:55:11 Extracting hyp (1 ops) ... 2022/08/03 21:55:11 Extracting imagefv (1 ops) ... 2022/08/03 21:55:11 Extracting keymaster (1 ops) ... 2022/08/03 21:55:11 Extracting modem (137 ops) ... 2022/08/03 21:55:22 Extracting odm (64 ops) ... 2022/08/03 21:55:23 Extracting product (808 ops) ... 2022/08/03 21:56:24 Extracting qupfw (1 ops) ... 2022/08/03 21:56:24 Extracting system (1604 ops) ... 2022/08/03 21:58:54 Extracting system_ext (282 ops) ... 2022/08/03 21:59:13 Extracting tz (2 ops) ... 2022/08/03 21:59:13 Extracting uefisecapp (1 ops) ... 2022/08/03 21:59:13 Extracting vbmeta (1 ops) ... 2022/08/03 21:59:13 Extracting vbmeta_system (1 ops) ... 2022/08/03 21:59:13 Extracting vendor (827 ops) ... 2022/08/03 22:00:26 Extracting vendor_boot (48 ops) ... 2022/08/03 22:00:27 Extracting xbl (2 ops) ... 2022/08/03 22:00:27 Extracting xbl_config (1 ops) ... 2022/08/03 22:00:27 Done!
Extraction time on my machine: 5m 20s .
Pros:
Cons:
Slow (single-threaded and not optimized)
Can't extract individual images
Can't list images
Can be hard to build from sources
Can't specify an extraction directory
Comparison
Tool
Extraction duration
List images
Extract individual images
vm03/payload_dumper
5m 54s
✅
❌
LineageOS/scripts
3m 49s
✅
✅
ssut/payload-dumper-go
2m 6s
✅
✅
tobyxdd/android-ota-payload-extractor
5m 20s
❌
❌
Tool
Specify extraction directory
Parallel extraction
vm03/payload_dumper
✅
❌
LineageOS/scripts
✅
❌
ssut/payload-dumper-go
✅
✅
tobyxdd/android-ota-payload-extractor
❌
❌
ssut/payload-dumper-go seems the better choice so far by being both faster and having more features.