linuxandroid

Android OTA payload dumping / extraction: 4 tools review

noraj (Alexandre ZANNI)
,
🇬🇧

Introduction#

Anyone that already tried to root their phone using Magisk's Patching Images method would have already encountered this issue.

Magisk's Patching Images method ask you to copy boot.img or recovery.img (depending on your boot ramdisk availability), but when you extract the ZIP archive of your favorite ROM you see only something like this:

META-INF/com/android/metadata
META-INF/com/android/metadata.pb
apex_info.pb
care_map.pb
payload.bin
payload_properties.txt
META-INF/com/android/otacert

Newer version of Android OTA packages come with a payload.bin file that contains the individual image files of different firmware partitions.

payload.bin appeared when Google introduced the concept of A/B seamless system updates (Android Oreo (8)). A/B update mechanism introduced a new format for the Android OTA/firmware update files. Earlier, these update packages came with partition images, which could easily be extracted and used for manual flashing. Nowadays, these update packages come with a single payload.bin file that contains the individual partition images.

How to extract images#

Various open-source tools allow to extract the images from payload.bin.

For the sake of this examples I extracted a MIUI ROM (miui_ALIOTHGlobal_V13.0.3.0.SKHMIXM_07afcb74cd_12.0.zip) to a folder named miui.

This article will cover only the case of full OTA (not incremental OTA).

My hardware (for extraction time comparison):

  • CPU: AMD Ryzen 5 1600X 6c/6t
  • Memory: 32 GB
  • Disk: Medium SATA SSD

vm03/payload_dumper#

vm03/payload_dumper is a python script.

The requirements, outside of Python 3 itself, are:

  • protobuf
  • six
  • bsdiff4

To install them on ArchLinux:

$ sudo pacman -S python-protobuf python-six --needed --asdeps && pikaur -S python-bsdiff4 --asdeps

With pip (I advice using a virtual environment):

$ pip install --user protobuf six bsdiff4

There is also a docker image.

Beware of the size of storage available you need to be able to extract, for example with my MIUI ROM:

  • the unextracted ZIP weight 3.2 GB
  • the files once extracted have roughly the same size of 3.2 GB
  • the extracted images weight 7.6 GB

Launching the extraction is easy:

$ python payload_dumper/payload_dumper.py miui/payload.bin --out payload_dumper_test
Processing abl partition.Done
Processing aop partition.Done
Processing bluetooth partition.Done
Processing boot partition................................................................Done
Processing cmnlib partition.Done
Processing cmnlib64 partition.Done
Processing devcfg partition.Done
Processing dsp partition................................Done
Processing dtbo partition................Done
Processing featenabler partition.Done
Processing hyp partition.Done
Processing imagefv partition.Done
Processing keymaster partition.Done
Processing modem partition.........................................................................................................................................Done
Processing odm partition................................................................Done
Processing product partition........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................Done
Processing qupfw partition.Done
Processing system partition....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................Done
Processing system_ext partition..........................................................................................................................................................................................................................................................................................Done
Processing tz partition..Done
Processing uefisecapp partition.Done
Processing vbmeta partition.Done
Processing vbmeta_system partition.Done
Processing vendor partition.........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
..................................................................................................................................................Done
Processing vendor_boot partition................................................Done
Processing xbl partition..Done
Processing xbl_config partition.Done

Extraction time on my machine: 5m 54s.

If you are interested in a few images only, you can extract only those to save a lot of time and space:

$ python payload_dumper/payload_dumper.py miui/payload.bin --out payload_dumper_test --images boot,vendor_boot
Processing boot partition................................................................Done
Processing vendor_boot partition................................................Done

Pros:

  • Works
  • Can extract individual images

Cons:

  • Slow (single-threaded)
  • Can't list images

LineageOS/scripts#

LineageOS/scripts is a collection of Android python scripts from LineageOS.

This time we only requires protobuf as a dpendency.

Before extracting you can list the available images:

$ python scripts/update-payload-extractor/extract.py miui/payload.bin --list_partitions
abl (208896 bytes)
aop (204800 bytes)
bluetooth (421888 bytes)
boot (134217728 bytes)
cmnlib (397312 bytes)
cmnlib64 (516096 bytes)
devcfg (57344 bytes)
dsp (67108864 bytes)
dtbo (33554432 bytes)
featenabler (90112 bytes)
hyp (446464 bytes)
imagefv (2097152 bytes)
keymaster (282624 bytes)
modem (285831168 bytes)
odm (134217728 bytes)
product (1693982720 bytes)
qupfw (57344 bytes)
system (3363110912 bytes)
system_ext (590237696 bytes)
tz (3190784 bytes)
uefisecapp (126976 bytes)
vbmeta (8192 bytes)
vbmeta_system (4096 bytes)
vendor (1733591040 bytes)
vendor_boot (100663296 bytes)
xbl (3563520 bytes)
xbl_config (102400 bytes)

Launching the extraction is similar to the previous solution:

$ python scripts/update-payload-extractor/extract.py miui/payload.bin --output_dir ./lineageos_extractor
Extracting abl
Extracting aop
Extracting bluetooth
Extracting boot
Extracting cmnlib
Extracting cmnlib64
Extracting devcfg
Extracting dsp
Extracting dtbo
Extracting featenabler
Extracting hyp
Extracting imagefv
Extracting keymaster
Extracting modem
Extracting odm
Extracting product
Extracting qupfw
Extracting system
Extracting system_ext
Extracting tz
Extracting uefisecapp
Extracting vbmeta
Extracting vbmeta_system
Extracting vendor
Extracting vendor_boot
Extracting xbl
Extracting xbl_config

Extraction time on my machine: 3m 49s.

If you are interested in a few images only, you can extract only those to save a lot of time and space:

$ python scripts/update-payload-extractor/extract.py miui/payload.bin --output_dir ./lineageos_extractor --partitions boot vendor_boot
Extracting boot
Extracting vendor_boot

Pros:

  • Works
  • Can extract individual images
  • Can list images

Cons:

  • Slow (single-threaded)

ssut/payload-dumper-go#

ssut/payload-dumper-go is a tool written in go.

The only system dependency is xz (LZMA lib).

To install it on ArchLinux:

$ sudo pacman -Syu xz --needed --asdeps

For Ubuntu:

$ sudo apt-get install liblzma-dev

There is also a Docker image.

If you get trouble compiling with go you can either use the Docker image or download the prebuild binary.

Before extracting you can list the available images:

$ ./payload-dumper-go -l miui/payload.bin
payload.bin: miui/payload.bin
Payload Version: 2
Payload Manifest Length: 223511
Payload Manifest Signature Length: 267
Found partitions:
abl (209 kB), aop (205 kB), bluetooth (422 kB), boot (134 MB), cmnlib (397 kB), cmnlib64 (516 kB), devcfg (57 kB), dsp (67 MB), dtbo (34 MB), featenabler (90 kB), hyp (446 kB), imagefv (2.1 MB), keymaster (283 kB), modem (286 MB), odm (134 MB), product (1.7 GB), qupfw (57 kB), system (3.4 GB), system_ext (590 MB), tz (3.2 MB), uefisecapp (127 kB), vbmeta (8.2 kB), vbmeta_system (4.1 kB), vendor (1.7 GB), vendor_boot (101 MB), xbl (3.6 MB), xbl_config (102 kB)

Launching the extraction is similar to the previous solution, but payload-dumper-go is multi-threaded and will use 4 parallel workers by default (that can be tweaked with the -c option).

$ ./payload-dumper-go -o payload-dumper-go_test miui/payload.bin
payload.bin: miui/payload.bin
Payload Version: 2
Payload Manifest Length: 223511
Payload Manifest Signature Length: 267
Found partitions:
abl (209 kB), aop (205 kB), bluetooth (422 kB), boot (134 MB), cmnlib (397 kB), cmnlib64 (516 kB), devcfg (57 kB), dsp (67 MB), dtbo (34 MB), featenabler (90 kB), hyp (446 kB), imagefv (2.1 MB), keymaster (283 kB), modem (286 MB), odm (
134 MB), product (1.7 GB), qupfw (57 kB), system (3.4 GB), system_ext (590 MB), tz (3.2 MB), uefisecapp (127 kB), vbmeta (8.2 kB), vbmeta_system (4.1 kB), vendor (1.7 GB), vendor_boot (101 MB), xbl (3.6 MB), xbl_config (102 kB)
Number of workers: 4
aop (205 kB)            [============================================================================================================================================================================================================] 100 %
bluetooth (422 kB)      [============================================================================================================================================================================================================] 100 %
abl (209 kB)            [============================================================================================================================================================================================================] 100 %
boot (134 MB)           [============================================================================================================================================================================================================] 100 %
cmnlib (397 kB)         [============================================================================================================================================================================================================] 100 %
cmnlib64 (516 kB)       [============================================================================================================================================================================================================] 100 %
devcfg (57 kB)          [============================================================================================================================================================================================================] 100 %
dsp (67 MB)             [============================================================================================================================================================================================================] 100 %
dtbo (34 MB)            [============================================================================================================================================================================================================] 100 %
featenabler (90 kB)     [============================================================================================================================================================================================================] 100 %
hyp (446 kB)            [============================================================================================================================================================================================================] 100 %
imagefv (2.1 MB)        [============================================================================================================================================================================================================] 100 %
keymaster (283 kB)      [============================================================================================================================================================================================================] 100 %
modem (286 MB)          [============================================================================================================================================================================================================] 100 %
odm (134 MB)            [============================================================================================================================================================================================================] 100 %
product (1.7 GB)        [============================================================================================================================================================================================================] 100 %
qupfw (57 kB)           [============================================================================================================================================================================================================] 100 %
system (3.4 GB)         [============================================================================================================================================================================================================] 100 %
system_ext (590 MB)     [============================================================================================================================================================================================================] 100 %
tz (3.2 MB)             [============================================================================================================================================================================================================] 100 %
uefisecapp (127 kB)     [============================================================================================================================================================================================================] 100 %
vbmeta (8.2 kB)         [============================================================================================================================================================================================================] 100 %
vbmeta_system (4.1 kB)  [============================================================================================================================================================================================================] 100 %
vendor (1.7 GB)         [============================================================================================================================================================================================================] 100 %
vendor_boot (101 MB)    [============================================================================================================================================================================================================] 100 %
xbl (3.6 MB)            [============================================================================================================================================================================================================] 100 %
xbl_config (102 kB)     [============================================================================================================================================================================================================] 100

Extraction time on my machine: 2m 6s.

If you are interested in a few images only, you can extract only those to save a lot of time and space:

$ ./payload-dumper-go -o payload-dumper-go_test -p boot,vendor_boot miui/payload.bin
payload.bin: miui/payload.bin
Payload Version: 2
Payload Manifest Length: 223511
Payload Manifest Signature Length: 267
Found partitions:
abl (209 kB), aop (205 kB), bluetooth (422 kB), boot (134 MB), cmnlib (397 kB), cmnlib64 (516 kB), devcfg (57 kB), dsp (67 MB), dtbo (34 MB), featenabler (90 kB), hyp (446 kB), imagefv (2.1 MB), keymaster (283 kB), modem (286 MB), odm (134 MB), product (1.7 GB), qupfw (57 kB), system (3.4 GB), system_ext (590 MB), tz (3.2 MB), uefisecapp (127 kB), vbmeta (8.2 kB), vbmeta_system (4.1 kB), vendor (1.7 GB), vendor_boot (101 MB), xbl (3.6 MB), xbl_config (102 kB)
Number of workers: 4
vendor_boot (101 MB)  [==============================================================================================================================================================================================================] 100 %
boot (134 MB)         [==============================================================================================================================================================================================================] 100

Pros:

  • Works
  • Can extract individual images
  • Can list images
  • Fast (multi-threaded)

Cons:

  • Can be hard to build from sources

tobyxdd/android-ota-payload-extractor#

tobyxdd/android-ota-payload-extractor is a tool written in go.

If you get trouble compiling with go you can download the prebuild binary.

Launching the extraction is easy:

$ ./android-ota-extractor miui/payload.bin 
2022/08/03 21:55:07 Parsing payload...
2022/08/03 21:55:07 Block size: 4096, Partition count: 27
2022/08/03 21:55:07 Extracting abl (1 ops) ...
2022/08/03 21:55:07 Extracting aop (1 ops) ...
2022/08/03 21:55:07 Extracting bluetooth (1 ops) ...
2022/08/03 21:55:07 Extracting boot (64 ops) ...
2022/08/03 21:55:09 Extracting cmnlib (1 ops) ...
2022/08/03 21:55:10 Extracting cmnlib64 (1 ops) ...
2022/08/03 21:55:10 Extracting devcfg (1 ops) ...
2022/08/03 21:55:10 Extracting dsp (32 ops) ...
2022/08/03 21:55:11 Extracting dtbo (16 ops) ...
2022/08/03 21:55:11 Extracting featenabler (1 ops) ...
2022/08/03 21:55:11 Extracting hyp (1 ops) ...
2022/08/03 21:55:11 Extracting imagefv (1 ops) ...
2022/08/03 21:55:11 Extracting keymaster (1 ops) ...
2022/08/03 21:55:11 Extracting modem (137 ops) ...
2022/08/03 21:55:22 Extracting odm (64 ops) ...
2022/08/03 21:55:23 Extracting product (808 ops) ...
2022/08/03 21:56:24 Extracting qupfw (1 ops) ...
2022/08/03 21:56:24 Extracting system (1604 ops) ...
2022/08/03 21:58:54 Extracting system_ext (282 ops) ...
2022/08/03 21:59:13 Extracting tz (2 ops) ...
2022/08/03 21:59:13 Extracting uefisecapp (1 ops) ...
2022/08/03 21:59:13 Extracting vbmeta (1 ops) ...
2022/08/03 21:59:13 Extracting vbmeta_system (1 ops) ...
2022/08/03 21:59:13 Extracting vendor (827 ops) ...
2022/08/03 22:00:26 Extracting vendor_boot (48 ops) ...
2022/08/03 22:00:27 Extracting xbl (2 ops) ...
2022/08/03 22:00:27 Extracting xbl_config (1 ops) ...
2022/08/03 22:00:27 Done!

Extraction time on my machine: 5m 20s.

Pros:

  • Works

Cons:

  • Slow (single-threaded and not optimized)
  • Can't extract individual images
  • Can't list images
  • Can be hard to build from sources
  • Can't specify an extraction directory

Comparison#

Tool Extraction duration List images Extract individual images
vm03/payload_dumper 5m 54s ✅ ❌
LineageOS/scripts 3m 49s ✅ ✅
ssut/payload-dumper-go 2m 6s ✅ ✅
tobyxdd/android-ota-payload-extractor 5m 20s ❌ ❌
Tool Specify extraction directory Parallel extraction
vm03/payload_dumper ✅ ❌
LineageOS/scripts ✅ ❌
ssut/payload-dumper-go ✅ ✅
tobyxdd/android-ota-payload-extractor ❌ ❌

ssut/payload-dumper-go seems the better choice so far by being both faster and having more features.

Share