zabpehely
/
PiShrink
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge branch 'master' into master

This commit is contained in:
bggardner 2020-05-29 22:13:18 -04:00 committed by GitHub
parent 4250d9a258 d38666fefd
commit 870649bca9
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 176 additions and 118 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

132
README.md
View File

@ -1,62 +1,70 @@
# PiShrink # # PiShrink #
PiShrink is a bash script that automatically shrink a pi image that will then resize to the max size of the SD card on boot. This will make putting the image back onto the SD card faster and the shrunk images will compress better. PiShrink is a bash script that automatically shrink a pi image that will then resize to the max size of the SD card on boot. This will make putting the image back onto the SD card faster and the shrunk images will compress better.
In addition the shrinked image can be compressed with gzip and xz to create an even smaller image. Parallel compression of the image
## Usage ## using multiple cores is supported.
```
sudo pishrink.sh [-sdrzh] imagefile.img [newimagefile.img] ## Usage ##
-s: Skip autoexpand ```
-d: Debug mode on Usage: $0 [-adhrspvzZ] imagefile.img [newimagefile.img]
-r: Use advanced repair options
-z: Gzip compress image after shrinking -s Don't expand filesystem when image is booted the first time
-h: display help text -v Be verbose
``` -r Use advanced filesystem repair option if the normal one fails
-z Compress image after shrinking with gzip
If you specify the `newimagefile.img` parameter, the script will make a copy of `imagefile.img` and work off that. You will need enough space to make a full copy of the image to use that option. -Z Compress image after shrinking with xz
-a Compress image in parallel using multiple cores
* `-s` will skip the autoexpanding part of the process. -p Remove logs, apt archives, dhcp leases and ssh hostkeys
* `-d` will create a logfile `pishrink.log` which may help for problem analysis. -d Write debug messages in a debug log file
* `-r` will attempt to repair the filesystem if regular repairs fail ```
* `-z` will Gzip compress the image after shrinking. The `.gz` extension will be added to the filename.
If you specify the `newimagefile.img` parameter, the script will make a copy of `imagefile.img` and work off that. You will need enough space to make a full copy of the image to use that option.
## Prerequisites ## * `-r` will attempt to repair the filesystem if regular repairs fail
If you are trying to shrink a [NOOBS](https://github.com/raspberrypi/noobs) image it will likely fail. This is due to [NOOBS partitioning](https://github.com/raspberrypi/noobs/wiki/NOOBS-partitioning-explained) being significantly different than Raspbian's. Hopefully PiShrink will be able to support NOOBS in the near future. * `-z` will compress the image after shrinking using gzip. `.gz` extension will be added to the filename.
* `-Z` will compress the image after shrinking using xz. `.xz` extension will be added to the filename.
If using Ubuntu, you will likely see an error about `e2fsck` being out of date and `metadata_csum`. The simplest fix for this is to use Ubuntu 16.10 and up, as it will save you a lot of hassle in the long run. * `-a` will use option -f9 for pigz and option -T0 for xz and compress in parallel.
* `-d` will create a logfile `pishrink.log` which may help for problem analysis.
## Installation ##
```bash Default options for compressors can be overwritten by defining PISHRINK_GZIP or PSHRINK_XZ environment variables for gzip and xz.
wget https://raw.githubusercontent.com/Drewsif/PiShrink/master/pishrink.sh
chmod +x pishrink.sh ## Prerequisites ##
sudo mv pishrink.sh /usr/local/bin If you are trying to shrink a [NOOBS](https://github.com/raspberrypi/noobs) image it will likely fail. This is due to [NOOBS partitioning](https://github.com/raspberrypi/noobs/wiki/NOOBS-partitioning-explained) being significantly different than Raspbian's. Hopefully PiShrink will be able to support NOOBS in the near future.
```
If using Ubuntu, you will likely see an error about `e2fsck` being out of date and `metadata_csum`. The simplest fix for this is to use Ubuntu 16.10 and up, as it will save you a lot of hassle in the long run.
## Example ##
```bash ## Installation ##
[user@localhost PiShrink]$ sudo pishrink.sh pi.img ```bash
e2fsck 1.42.9 (28-Dec-2013) wget https://raw.githubusercontent.com/Drewsif/PiShrink/master/pishrink.sh
Pass 1: Checking inodes, blocks, and sizes chmod +x pishrink.sh
Pass 2: Checking directory structure sudo mv pishrink.sh /usr/local/bin
Pass 3: Checking directory connectivity ```
Pass 4: Checking reference counts
Pass 5: Checking group summary information ## Example ##
/dev/loop1: 88262/1929536 files (0.2% non-contiguous), 842728/7717632 blocks ```bash
resize2fs 1.42.9 (28-Dec-2013) [user@localhost PiShrink]$ sudo pishrink.sh pi.img
resize2fs 1.42.9 (28-Dec-2013) e2fsck 1.42.9 (28-Dec-2013)
Resizing the filesystem on /dev/loop1 to 773603 (4k) blocks. Pass 1: Checking inodes, blocks, and sizes
Begin pass 2 (max = 100387) Pass 2: Checking directory structure
Relocating blocks XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX Pass 3: Checking directory connectivity
Begin pass 3 (max = 236) Pass 4: Checking reference counts
Scanning inode table XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX Pass 5: Checking group summary information
Begin pass 4 (max = 7348) /dev/loop1: 88262/1929536 files (0.2% non-contiguous), 842728/7717632 blocks
Updating inode references XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX resize2fs 1.42.9 (28-Dec-2013)
The filesystem on /dev/loop1 is now 773603 blocks long. resize2fs 1.42.9 (28-Dec-2013)
Resizing the filesystem on /dev/loop1 to 773603 (4k) blocks.
Shrunk pi.img from 30G to 3.1G Begin pass 2 (max = 100387)
``` Relocating blocks XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Begin pass 3 (max = 236)
## Contributing ## Scanning inode table XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
If you find a bug please create an issue for it. If you would like a new feature added, you can create an issue for it but I can't promise that I will get to it. Begin pass 4 (max = 7348)
Updating inode references XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Pull requests for new features and bug fixes are more than welcome! The filesystem on /dev/loop1 is now 773603 blocks long.
Shrunk pi.img from 30G to 3.1G
```
## Contributing ##
If you find a bug please create an issue for it. If you would like a new feature added, you can create an issue for it but I can't promise that I will get to it.
Pull requests for new features and bug fixes are more than welcome!

162
pishrink.sh
View File

@ -2,12 +2,18 @@
version="v0.1.3" version="v0.1.3"
CURRENT_DIR=$(pwd) CURRENT_DIR="$(pwd)"
SCRIPTNAME="${0##*/}" SCRIPTNAME="${0##*/}"
LOGFILE=${CURRENT_DIR}/${SCRIPTNAME%.*}.log MYNAME="${SCRIPTNAME%.*}"
LOGFILE="${CURRENT_DIR}/${SCRIPTNAME%.*}.log"
REQUIRED_TOOLS="parted losetup tune2fs md5sum e2fsck resize2fs"
ZIPTOOLS=("gzip xz")
declare -A ZIP_PARALLEL_TOOL=( [gzip]="pigz" [xz]="xz" ) # parallel zip tool to use in parallel mode
declare -A ZIP_PARALLEL_OPTIONS=( [gzip]="-f9" [xz]="-T0" ) # options for zip tools in parallel mode
declare -A ZIPEXTENSIONS=( [gzip]="gz" [xz]="xz" ) # extensions of zipped files
function info() { function info() {
echo "$SCRIPTNAME: $1..." echo "$SCRIPTNAME: $1 ..."
} }
function error() { function error() {
@ -63,45 +69,42 @@ fi
help() { help() {
local help local help
read -r -d '' help << EOM read -r -d '' help << EOM
Usage: $0 [-sdrpzh] imagefile.img [newimagefile.img] Usage: $0 [-adhrspvzZ] imagefile.img [newimagefile.img]
-s: Don't expand filesystem when image is booted the first time -s Don't expand filesystem when image is booted the first time
-d: Write debug messages in a debug log file -v Be verbose
-r: Use advanced filesystem repair option if the normal one fails -r Use advanced filesystem repair option if the normal one fails
-p: Remove logs, apt archives, dhcp leases and ssh hostkeys -z Compress image after shrinking with gzip
-z: Gzip compress image after shrinking -Z Compress image after shrinking with xz
-a Compress image in parallel using multiple cores
-p Remove logs, apt archives, dhcp leases and ssh hostkeys
-d Write debug messages in a debug log file
EOM EOM
echo "$help" echo "$help"
exit -1 exit -1
} }
usage() {
echo "Usage: $0 [-sdrpzh] imagefile.img [newimagefile.img]"
echo ""
echo " -s: Skip autoexpand"
echo " -d: Debug mode on"
echo " -r: Use advanced repair options"
echo " -p: Remove logs, apt archives, dhcp leases and ssh hostkeys"
echo " -z: Gzip compress image after shrinking"
echo " -h: display help text"
exit -1
}
should_skip_autoexpand=false should_skip_autoexpand=false
debug=false debug=false
repair=false repair=false
gzip_compress=false parallel=false
verbose=false
prep=false prep=false
ziptool=""
required_tools="$REQUIRED_TOOLS"
while getopts ":sdrpzh" opt; do while getopts ":adhprsvzZ" opt; do
case "${opt}" in case "${opt}" in
s) should_skip_autoexpand=true ;; a) parallel=true;;
d) debug=true;; d) debug=true;;
r) repair=true;;
p) prep=true;;
z) gzip_compress=true;;
h) help;; h) help;;
*) usage ;; p) prep=true;;
r) repair=true;;
s) should_skip_autoexpand=true ;;
v) verbose=true;;
z) ziptool="gzip";;
Z) ziptool="xz";;
*) help;;
esac esac
done done
shift $((OPTIND-1)) shift $((OPTIND-1))
@ -121,8 +124,9 @@ img="$1"
#Usage checks #Usage checks
if [[ -z "$img" ]]; then if [[ -z "$img" ]]; then
usage help
fi fi
if [[ ! -f "$img" ]]; then if [[ ! -f "$img" ]]; then
error $LINENO "$img is not a file..." error $LINENO "$img is not a file..."
exit -2 exit -2
@ -132,8 +136,22 @@ if (( EUID != 0 )); then
exit -3 exit -3
fi fi
# check selected compression tool is supported and installed
if [[ -n $ziptool ]]; then
if [[ ! " ${ZIPTOOLS[@]} " =~ " $ziptool " ]]; then
error $LINENO "$ziptool is an unsupported ziptool."
exit -17
else
if [[ $parallel == true && ziptool == "gzip" ]]; then
REQUIRED_TOOLS="$REQUIRED_TOOLS pigz"
else
REQUIRED_TOOLS="$REQUIRED_TOOLS $ziptool"
fi
fi
fi
#Check that what we need is installed #Check that what we need is installed
for command in parted losetup tune2fs md5sum e2fsck resize2fs; do for command in $REQUIRED_TOOLS; do
command -v $command >/dev/null 2>&1 command -v $command >/dev/null 2>&1
if (( $? != 0 )); then if (( $? != 0 )); then
error $LINENO "$command is not installed." error $LINENO "$command is not installed."
@ -159,16 +177,22 @@ trap cleanup ERR EXIT
#Gather info #Gather info
info "Gathering data" info "Gathering data"
beforesize=$(ls -lh "$img" | cut -d ' ' -f 5) beforesize="$(ls -lh "$img" | cut -d ' ' -f 5)"
parted_output=$(parted -ms "$img" unit B print | tail -n 1) parted_output="$(parted -ms "$img" unit B print)"
partnum=$(echo "$parted_output" | cut -d ':' -f 1) rc=$?
partstart=$(echo "$parted_output" | cut -d ':' -f 2 | tr -d 'B') if (( $rc )); then
loopback=$(losetup -f --show -o "$partstart" "$img") error $LINENO "parted failed with rc $rc"
tune2fs_output=$(tune2fs -l "$loopback") info "Possibly invalid image. Run 'parted $img unit B print' manually to investigate"
currentsize=$(echo "$tune2fs_output" | grep '^Block count:' | tr -d ' ' | cut -d ':' -f 2) exit -6
blocksize=$(echo "$tune2fs_output" | grep '^Block size:' | tr -d ' ' | cut -d ':' -f 2) fi
partnum="$(echo "$parted_output" | tail -n 1 | cut -d ':' -f 1)"
partstart="$(echo "$parted_output" | tail -n 1 | cut -d ':' -f 2 | tr -d 'B')"
loopback="$(losetup -f --show -o "$partstart" "$img")"
tune2fs_output="$(tune2fs -l "$loopback")"
currentsize="$(echo "$tune2fs_output" | grep '^Block count:' | tr -d ' ' | cut -d ':' -f 2)"
blocksize="$(echo "$tune2fs_output" | grep '^Block size:' | tr -d ' ' | cut -d ':' -f 2)"
logVariables $LINENO tune2fs_output currentsize blocksize logVariables $LINENO beforesize parted_output partnum partstart tune2fs_output currentsize blocksize
#Check if we should make pi expand rootfs on next boot #Check if we should make pi expand rootfs on next boot
if [ "$should_skip_autoexpand" = false ]; then if [ "$should_skip_autoexpand" = false ]; then
@ -214,7 +238,7 @@ if [[ $prep == true ]]; then
info "Syspreping: Removing logs, apt archives, dhcp leases and ssh hostkeys" info "Syspreping: Removing logs, apt archives, dhcp leases and ssh hostkeys"
mountdir=$(mktemp -d) mountdir=$(mktemp -d)
mount "$loopback" "$mountdir" mount "$loopback" "$mountdir"
rm -rf "$mountdir/var/cache/apt/archives/*" "$mountdir/var/lib/dhcpcd5/*" "$mountdir/var/log/*" "$mountdir/var/tmp/*" "$mountdir/tmp/*" "$mountdir/etc/ssh/*_host_*" rm -rf "$mountdir/var/cache/apt/archives/*" "$mountdir/var/lib/dhcpcd5/*" "$mountdir/var/log/*" "$mountdir/var/tmp/*" "$mountdir/tmp/*" "$mountdir/etc/ssh/*_host_*"
umount "$mountdir" umount "$mountdir"
fi fi
@ -228,7 +252,7 @@ if ! minsize=$(resize2fs -P "$loopback"); then
exit -10 exit -10
fi fi
minsize=$(cut -d ':' -f 2 <<< "$minsize" | tr -d ' ') minsize=$(cut -d ':' -f 2 <<< "$minsize" | tr -d ' ')
logVariables $LINENO minsize logVariables $LINENO currentsize minsize
if [[ $currentsize -eq $minsize ]]; then if [[ $currentsize -eq $minsize ]]; then
error $LINENO "Image already shrunk to smallest size" error $LINENO "Image already shrunk to smallest size"
exit -11 exit -11
@ -248,8 +272,9 @@ logVariables $LINENO minsize
#Shrink filesystem #Shrink filesystem
info "Shrinking filesystem" info "Shrinking filesystem"
resize2fs -p "$loopback" $minsize resize2fs -p "$loopback" $minsize
if [[ $? != 0 ]]; then rc=$?
error $LINENO "resize2fs failed" if (( $rc )); then
error $LINENO "resize2fs failed with rc $rc"
mount "$loopback" "$mountdir" mount "$loopback" "$mountdir"
mv "$mountdir/etc/rc.local.bak" "$mountdir/etc/rc.local" mv "$mountdir/etc/rc.local.bak" "$mountdir/etc/rc.local"
umount "$mountdir" umount "$mountdir"
@ -262,39 +287,64 @@ sleep 1
partnewsize=$(($minsize * $blocksize)) partnewsize=$(($minsize * $blocksize))
newpartend=$(($partstart + $partnewsize)) newpartend=$(($partstart + $partnewsize))
logVariables $LINENO partnewsize newpartend logVariables $LINENO partnewsize newpartend
if ! parted -s -a minimal "$img" rm "$partnum"; then parted -s -a minimal "$img" rm "$partnum"
rc=$? rc=$?
if (( $rc )); then
error $LINENO "parted failed with rc $rc" error $LINENO "parted failed with rc $rc"
exit -13 exit -13
fi fi
if ! parted -s "$img" unit B mkpart primary "$partstart" "$newpartend"; then parted -s "$img" unit B mkpart primary "$partstart" "$newpartend"
rc=$? rc=$?
if (( $rc )); then
error $LINENO "parted failed with rc $rc" error $LINENO "parted failed with rc $rc"
exit -14 exit -14
fi fi
#Truncate the file #Truncate the file
info "Shrinking image" info "Shrinking image"
if ! endresult=$(parted -ms "$img" unit B print free); then endresult=$(parted -ms "$img" unit B print free)
rc=$? rc=$?
if (( $rc )); then
error $LINENO "parted failed with rc $rc" error $LINENO "parted failed with rc $rc"
exit -15 exit -15
fi fi
endresult=$(tail -1 <<< "$endresult" | cut -d ':' -f 2 | tr -d 'B') endresult=$(tail -1 <<< "$endresult" | cut -d ':' -f 2 | tr -d 'B')
logVariables $LINENO endresult logVariables $LINENO endresult
if ! truncate -s "$endresult" "$img"; then truncate -s "$endresult" "$img"
rc=$? rc=$?
if (( $rc )); then
error $LINENO "trunate failed with rc $rc" error $LINENO "trunate failed with rc $rc"
exit -16 exit -16
fi fi
if [[ $gzip_compress == true ]]; then # handle compression
info "Gzipping the shrunk image" if [[ -n $ziptool ]]; then
if [[ ! $(gzip -f9 "$img") ]]; then options=""
img=$img.gz envVarname="${MYNAME^^}_${ziptool^^}" # PISHRINK_GZIP or PISHRINK_XZ environment variables allow to override all options for gzip or xz
fi [[ $parallel == true ]] && options="${ZIP_PARALLEL_OPTIONS[$ziptool]}"
[[ -v $envVarname ]] && options="${!envVarname}" # if environment variable defined use these options
[[ $verbose == true ]] && options="$options -v" # add verbose flag if requested
if [[ $parallel == true ]]; then
parallel_tool="${ZIP_PARALLEL_TOOL[$ziptool]}"
info "Using $parallel_tool on the shrunk image"
if ! $parallel_tool ${options} "$img"; then
rc=$?
error $LINENO "$parallel_tool failed with rc $rc"
exit -18
fi
else # sequential
info "Using $ziptool on the shrunk image"
if ! $ziptool ${options} $img; then
rc=$?
error $LINENO "$ziptool failed with rc $rc"
exit -19
fi
fi
img=$img.${ZIPEXTENSIONS[$ziptool]}
fi fi
aftersize=$(ls -lh "$img" | cut -d ' ' -f 5) aftersize=$(ls -lh "$img" | cut -d ' ' -f 5)