TLDR, just gimme the scripts

So on every other device you can just set up rsync or rsnapshot, but android is developed by a bunch of corporate-bastards. So here we are with a 1500 word guide on how to give the OEMs the middle finger with their proprietary, partial and/or paid backup crap.

There are multiple potential ways of going about this all with pros and cons, depending on what you want. This will mostly focus on the methods I use to end up with the setup I’m currently running and touch on alternatives, marked with [alt.], I also explored.

Prerequisites

  • an Android phone with debugging in the developer-options enabled

  • Some sort of linux server to back up to with a usb-port accessible to plug in the phone. I’m using a raspi 4B.

    Failing that you can also set adb up via wifi (or even ethernet). You can also have it controlled by the phone itself, but I have trust issues with android’s powersave. If you want to go that route look into Automate or Syncthing.

  • unless you want to just use my setup, a bit of bash knowledge

Getting the data from Android

Before you create your own script, run adb devices to get your ANDROID_SERIAL number.

Get the app data via adb backup

This will open a dialogue on your phone that you have to confirm, so this is most useful with the “on plug in” running (see next point on how to bypass this).

Also be warned that adb restore is not the most reliable utility, sometimes getting stuck on apps or just crashing for no good reason. But you can just extract the data from the backup (see below).

success=1

adb -s ${ANDROID_SERIAL} backup -all -keyvalue -f "${BACKUP_DEST}/backup.ab.tmp" || success=0
# adb just writes a very small file when you click "abort"
[[ $(find "${BACKUP_DEST}/backup.ab.tmp" -size +1M) == "" ]] && success=0

[alt.] unattended adb backup

I’ll just show some tricks to get adb backup to run unattended, you’ll have to combine as needed whatever works for you. Just keep in mind it has to work with locked and off screen.

  • start by running adb backup --all and see what happens, if you’re really lucky this just works and you can just go to the automation section.

  • Failing that try to run adb root before the backup (this usually requires a rooted phone, but supposedly it works without on some roms).

  • The last resort is adb shell input. This might require you go into the developer options of your phone and enabling “simulated input” or similar somewhere. If you’re lucky you can simulate all inputs needed to
    • turn on the screen
    • unlock the phone
    • click “backup” on the backup screen
    • lock the phone again

    But BE WARNED: anyone coming by during this will be able to mess with your phone, as the phone is unlocked for a bit.

  • If you’re reading this, I’m sorry. I don’t know any other tricks. At some later point I figured out that on my ROM adb shell input requires root otherwise you just get some java exception. And when you have a rooted phone there are better ways entirely.

Get the media

For the data on the (internal-) sdcard there are two options:

adb pull all the media

The internal data is usually stored in /storage/emulated/0/, the external in /sdcard/. We can just pull these folders. Note that this takes quite a while, because all media has to be copied.

# Backup the internal storage
mkdir "${BACKUP_DEST}/shared.tmp"
adb -s ${ANDROID_SERIAL} pull -a /storage/emulated/0/ "${BACKUP_DEST}/shared.tmp" > /dev/null || success=0

# Backup the sdcard
mkdir "${BACKUP_DEST}/sdcard.tmp"
adb -s ${ANDROID_SERIAL} pull -a /sdcard "${BACKUP_DEST}/backup/sdcard.tmp" > /dev/null || success=0

# check if device is still connected, otherwise backup probably failed
adb -s "${ANDROID_SERIAL}" get-serialno || success=0

[alt.] rsync from the phone as media device

WARNING: What follows creates a giant security hole, look up Juice Jacking.

First configure your phone to appear as media device for data transfer when plugged into a computer by default (!). This can be done in the Developer-Options under “Standard USB Configuration”.

Afterwards copying the data is as simple as mounting the storage and copying the files. Here rsync can be used to significantly speed up the operations by only copying changes, not all files like adb pull.

# create mountpoint
mkdir /mnt/phone

# mount phone (optionally give -d to specify which phone)
jmtpfs /mnt/phone || success=0

# rsync for the internal storage
rsync -az "/mnt/phone/internal shared storage/" "${BACKUP_DEST}/backup/shared" || success=0

# rsync for the sd-card
rsync -az "/mnt/phone/external storage/" "${BACKUP_DEST}/backup/sdcard" || success=0

# check if device is still connected, otherwise backup probably failed
adb -s "${ANDROID_SERIAL}" get-serialno || success=0

When to backup

Unless you managed to make it past the adb backup dialog a fully automated backup is sadly not possible. The next best thing I could think of is backing up whenever I charge my phone, that is plug it into said exposed USB port.

Whenever you plug the phone in

For this, plug the phone in and check lsusb for the vendor and product id:

lsusb
# Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
# Bus 001 Device 022: ID 152d:0578 JMicron Technology Corp. / JMicron USA Technology Corp. JMS567 SATA 6Gb/s bridge
# Bus 001 Device 003: ID 04e8:3268 Samsung Electronics Co., Ltd ML-1610 Mono Laser Printer
# Bus 001 Device 024: ID 18d1:4ee7 Google Inc. #####################This one: Vendor 18d1, Product 4ee7
# Bus 001 Device 002: ID 2109:3431 VIA Labs, Inc. Hub
# Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

With these infos you can create a udev-rule that requests a systemd-service whenever you plug your phone in:

# /etc/udev/rules.d/80-phone-backup.rules
ACTION=="add", SUBSYSTEM=="usb", ATTRS{idVendor}=="18d1", ATTRS{idProduct}=="4ee7", TAG+="systemd", ENV{SYSTEMD_WANTS}="phone_backup.service"

And the corresponding service that runs the backup script

#/lib/systemd/system/phone_backup.service
[Unit]
Description=Start backup of Android Phone

[Service]
ExecStart=/etc/backup_phone.sh

Don’t forget to enable the service using

sudo systemctl daemon-reload
sudo systemctl enable phone_backup

Additionally I recommend prefixing your script with something like this find trickery to prevent it from backing up too often:

do_backup=1
[[ $(find "${BACKUP_DEST}/backup.ab" -cmin -720) != "" ]] && do_backup=0
if [[ -n ((do_backup)) ]]
	exit
fi

And in case you’re wondering why not to just use RUN+="/etc/backup_phone.sh" in the udev rule: That will handily crash it, because udev runs those scripts on the main loop and therefore can’t handle any kind of hardware event while your backup is running.

[alt.] Cronjob

Open the crontab and enter a job running your script at some point at night, here I chose 2:05.

# crontab -e
5 2 * * * /etc/backup_phone.sh

(If this opens some sort of unresponsive text editor, it’s probably vim. Press ESC > : > q > ! > Enter, then run export EDITOR=nano and retry)

Telling the user what is going on

You can send notifications to the phone via adb and I’m using a small wrapper to make that easier:

send_status() {
	echo $1
	adb -s ${ANDROID_SERIAL} shell "cmd notification post -t Backup backup '$1'"
}

Here the first Backup is the title of the notification, the second is some kind of “group” the notification belongs to. Only the last Notification of some group is displayed.

Delta-compress the backups

In order to for any backup solution to be of some use, you want multiple snapshots of your data at different points in time. And in order to ba able to store them on an affordable harddrive, we can use delta compression. Essentially this is just keeping some initial version of the data and all changes to it.

For that we first need the files themself. Use abe to decompress the app-backups (the media data is already just files). Just drop the .jar file in /opt. Also it is recommended to install oracle’s java not openjre, as the compression used by adb backup is some java standard library and apparently reimplementations are prone to errors.

java -jar /opt/abe.jar unpack "${BACKUP_DEST}/backup.ab.tmp" - | tar -xC "${BACKUP_DEST}/backup"

Then use rsnapshot to snapshot the folder you store all the decompressed data in.

# in /etc/rsnapshot.conf
backup  /data/phone_backup/backup

Putting it all together

Here are the all the files I use (apart from the line in my rsnapshot.conf) containing a bunch of additional bash scripting to make everything more stable and convenient.

And how the files and folders it creates look like:

${BACKUP_DEST}/
├── backup        The decompressed data
│   ├── date.txt  The date the backup was run at
│   ├── apps/
│   ├── shared/
│   └── sdcard/
├── backup.ab     The most recent adb backup output
├── backup.ab.tmp The currently loading adb backup out
├── shared.tmp/   The currently loading adb pull out
└── sdcard.tmp/   The currently loading adb pull out

Known Issues

Debug authorization is lost

This happens from time to time, it seems the keys expire sometimes after a reboot.

adb backup fails every time

Try to add a bit of waiting beforehand. Android doesn’t like beeing commanded too quickly. Also click on the Backup button on your phone quite fast, sometimes the connection just dies.