zabpehely
/
PiShrink
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Updated to original copy

This commit is contained in:
nhkhai 2020-08-09 13:53:01 +08:00
parent c6ffbb0294
commit 4959190ae9
2 changed files with 315 additions and 233 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

39
README.md
View File

@ -1,31 +1,48 @@
# PiShrink # # PiShrink #
PiShrink is a bash script that automatically shrink a pi image that will then resize to the max size of the SD card on boot. This will make putting the image back onto the SD card faster and the shrunk images will compress better. PiShrink is a bash script that automatically shrink a pi image that will then resize to the max size of the SD card on boot. This will make putting the image back onto the SD card faster and the shrunk images will compress better.
In addition the shrinked image can be compressed with gzip and xz to create an even smaller image. Parallel compression of the image
using multiple cores is supported.
## Usage ## ## Usage ##
``` ```
sudo pishrink.sh [-sdrzh] imagefile.img [newimagefile.img] Usage: $0 [-adhrspvzZ] imagefile.img [newimagefile.img]
-s: Skip autoexpand
-d: Debug mode on -s Don't expand filesystem when image is booted the first time
-r: Use advanced repair options -v Be verbose
-z: Gzip compress image after shrinking -r Use advanced filesystem repair option if the normal one fails
-h: display help text -z Compress image after shrinking with gzip
-Z Compress image after shrinking with xz
-a Compress image in parallel using multiple cores
-p Remove logs, apt archives, dhcp leases and ssh hostkeys
-d Write debug messages in a debug log file
``` ```
If you specify the `newimagefile.img` parameter, the script will make a copy of `imagefile.img` and work off that. You will need enough space to make a full copy of the image to use that option. If you specify the `newimagefile.img` parameter, the script will make a copy of `imagefile.img` and work off that. You will need enough space to make a full copy of the image to use that option.
* `-s` will skip the autoexpanding part of the process. * `-s` prevents automatic filesystem expantion on the images next boot
* `-v` enables more verbose output
* `-r` will attempt to repair the filesystem using aditional options if the normal repair fails
* `-z` will compress the image after shrinking using gzip. `.gz` extension will be added to the filename.
* `-Z` will compress the image after shrinking using xz. `.xz` extension will be added to the filename.
* `-a` will use option -f9 for pigz and option -T0 for xz and compress in parallel.
* `-d` will create a logfile `pishrink.log` which may help for problem analysis. * `-d` will create a logfile `pishrink.log` which may help for problem analysis.
* `-r` will attempt to repair the filesystem if regular repairs fail
* `-z` will Gzip compress the image after shrinking. The `.gz` extension will be added to the filename.
Default options for compressors can be overwritten by defining PISHRINK_GZIP or PSHRINK_XZ environment variables for gzip and xz.
## Prerequisites ## ## Prerequisites ##
If you are trying to shrink a [NOOBS](https://github.com/raspberrypi/noobs) image it will likely fail. This is due to [NOOBS partitioning](https://github.com/raspberrypi/noobs/wiki/NOOBS-partitioning-explained) being significantly different than Raspbian's. Hopefully PiShrink will be able to support NOOBS in the near future.
If you are running PiShrink in VirtualBox you will likely encounter an error if you
attempt to use VirtualBox's "Shared Folder" feature. You can copy the image you wish to
shrink on to the VM from a Shared Folder, but shrinking directctly from the Shared Folder
is know to cause issues.
If using Ubuntu, you will likely see an error about `e2fsck` being out of date and `metadata_csum`. The simplest fix for this is to use Ubuntu 16.10 and up, as it will save you a lot of hassle in the long run. If using Ubuntu, you will likely see an error about `e2fsck` being out of date and `metadata_csum`. The simplest fix for this is to use Ubuntu 16.10 and up, as it will save you a lot of hassle in the long run.
## Installation ## ## Installation ##
```bash ```bash
wget https://raw.githubusercontent.com/Drewsif/PiShrink/master/pishrink.sh wget https://raw.githubusercontent.com/Drewsif/PiShrink/master/pishrink.sh
chmod +x pishrink.sh chmod +x pishrink.sh
@ -33,6 +50,7 @@ sudo mv pishrink.sh /usr/local/bin
``` ```
## Example ## ## Example ##
```bash ```bash
[user@localhost PiShrink]$ sudo pishrink.sh pi.img [user@localhost PiShrink]$ sudo pishrink.sh pi.img
e2fsck 1.42.9 (28-Dec-2013) e2fsck 1.42.9 (28-Dec-2013)
@ -57,6 +75,7 @@ Shrunk pi.img from 30G to 3.1G
``` ```
## Contributing ## ## Contributing ##
If you find a bug please create an issue for it. If you would like a new feature added, you can create an issue for it but I can't promise that I will get to it. If you find a bug please create an issue for it. If you would like a new feature added, you can create an issue for it but I can't promise that I will get to it.
Pull requests for new features and bug fixes are more than welcome! Pull requests for new features and bug fixes are more than welcome!

371
pishrink.sh
View File

@ -2,9 +2,15 @@
version="v0.1.2" version="v0.1.2"
CURRENT_DIR=$(pwd) CURRENT_DIR="$(pwd)"
SCRIPTNAME="${0##*/}" SCRIPTNAME="${0##*/}"
LOGFILE=${CURRENT_DIR}/${SCRIPTNAME%.*}.log MYNAME="${SCRIPTNAME%.*}"
LOGFILE="${CURRENT_DIR}/${SCRIPTNAME%.*}.log"
REQUIRED_TOOLS="parted losetup tune2fs md5sum e2fsck resize2fs"
ZIPTOOLS=("gzip xz")
declare -A ZIP_PARALLEL_TOOL=( [gzip]="pigz" [xz]="xz" ) # parallel zip tool to use in parallel mode
declare -A ZIP_PARALLEL_OPTIONS=( [gzip]="-f9" [xz]="-T0" ) # options for zip tools in parallel mode
declare -A ZIPEXTENSIONS=( [gzip]="gz" [xz]="xz" ) # extensions of zipped files
function info() { function info() {
echo "$SCRIPTNAME: $1 ..." echo "$SCRIPTNAME: $1 ..."
@ -56,137 +62,28 @@ if [[ $repair == true ]]; then
(( $? < 4 )) && return (( $? < 4 )) && return
fi fi
error $LINENO "Filesystem recoveries failed. Giving up..." error $LINENO "Filesystem recoveries failed. Giving up..."
exit -9 exit 9
} }
help() { function set_autoexpand() {
local help
read -r -d '' help << EOM
Usage: $0 [-sdrpzh] imagefile.img [newimagefile.img]
-s: Don't expand filesystem when image is booted the first time
-d: Write debug messages in a debug log file
-r: Use advanced filesystem repair option if the normal one fails
-p: Remove logs, apt archives, dhcp leases and ssh hostkeys
-x: XZ compress image after shrinking
-e: XZ extreme compress image after shrinking
-z: Gzip compress image after shrinking
EOM
echo "$help"
exit -1
}
usage() {
echo "Usage: $0 [-sdrpzh] imagefile.img [newimagefile.img]"
echo ""
echo " -s: Skip autoexpand"
echo " -d: Debug mode on"
echo " -r: Use advanced repair options"
echo " -p: Remove logs, apt archives, dhcp leases and ssh hostkeys"
echo " -x: XZ compress image after shrinking"
echo " -e: XZ extreme compress image after shrinking"
echo " -z: Gzip compress image after shrinking"
echo " -h: display help text"
exit -1
}
should_skip_autoexpand=false
debug=false
repair=false
xzip_compress=false
exzip_compress=false
gzip_compress=false
prep=false
while getopts ":sdrpxezh" opt; do
case "${opt}" in
s) should_skip_autoexpand=true ;;
d) debug=true;;
r) repair=true;;
p) prep=true;;
x) xzip_compress=true;;
e) exzip_compress=true;;
z) gzip_compress=true;;
h) help;;
*) usage ;;
esac
done
shift $((OPTIND-1))
if [ "$debug" = true ]; then
info "Creating log file $LOGFILE"
rm "$LOGFILE" &>/dev/null
exec 1> >(stdbuf -i0 -o0 -e0 tee -a "$LOGFILE" >&1)
exec 2> >(stdbuf -i0 -o0 -e0 tee -a "$LOGFILE" >&2)
fi
echo "${0##*/} $version"
# Args
src="$1"
img="$1"
# Usage checks
if [[ -z "$img" ]]; then
usage
fi
if [[ ! -f "$img" ]]; then
error $LINENO "$img is not a file..."
exit -2
fi
if (( EUID != 0 )); then
error $LINENO "You need to be running as root."
exit -3
fi
# Check that what we need is installed
for command in parted losetup tune2fs md5sum e2fsck resize2fs; do
command -v $command >/dev/null 2>&1
if (( $? != 0 )); then
error $LINENO "$command is not installed."
exit -4
fi
done
# Copy to new file if requested
if [ -n "$2" ]; then
info "Copying $1 to $2..."
cp --reflink=auto --sparse=always "$1" "$2"
if (( $? != 0 )); then
error $LINENO "Could not copy file..."
exit -5
fi
old_owner=$(stat -c %u:%g "$1")
chown "$old_owner" "$2"
img="$2"
fi
# Cleanup at script exit
trap cleanup ERR EXIT
#Gather info
info "Gathering data"
beforesize=$(ls -lh "$img" | cut -d ' ' -f 5)
parted_output=$(parted -ms "$img" unit B print | tail -n 1)
partnum=$(echo "$parted_output" | cut -d ':' -f 1)
partstart=$(echo "$parted_output" | cut -d ':' -f 2 | tr -d 'B')
loopback=$(losetup -f --show -o "$partstart" "$img")
tune2fs_output=$(tune2fs -l "$loopback")
currentsize=$(echo "$tune2fs_output" | grep '^Block count:' | tr -d ' ' | cut -d ':' -f 2)
blocksize=$(echo "$tune2fs_output" | grep '^Block size:' | tr -d ' ' | cut -d ':' -f 2)
logVariables $LINENO tune2fs_output currentsize blocksize
# Check if we should make pi expand rootfs on next boot
if [ "$should_skip_autoexpand" = false ]; then
#Make pi expand rootfs on next boot #Make pi expand rootfs on next boot
mountdir=$(mktemp -d) mountdir=$(mktemp -d)
partprobe "$loopback"
mount "$loopback" "$mountdir" mount "$loopback" "$mountdir"
if [ "$(md5sum "$mountdir/etc/rc.local" | cut -d ' ' -f 1)" != "0542054e9ff2d2e0507ea1ffe7d4fc87" ]; then if [ ! -d "$mountdir/etc" ]; then
info "/etc not found, autoexpand will not be enabled"
umount "$mountdir"
return
fi
if [[ -f "$mountdir/etc/rc.local" ]] && [[ "$(md5sum "$mountdir/etc/rc.local" | cut -d ' ' -f 1)" != "1c579c7d5b4292fd948399b6ece39009" ]]; then
echo "Creating new /etc/rc.local" echo "Creating new /etc/rc.local"
if [ -f "$mountdir/etc/rc.local" ]; then
mv "$mountdir/etc/rc.local" "$mountdir/etc/rc.local.bak" mv "$mountdir/etc/rc.local" "$mountdir/etc/rc.local.bak"
fi
#####Do not touch the following lines##### #####Do not touch the following lines#####
cat <<\EOF1 > "$mountdir/etc/rc.local" cat <<\EOF1 > "$mountdir/etc/rc.local"
#!/bin/bash #!/bin/bash
@ -243,13 +140,166 @@ sleep 5
do_expand_rootfs do_expand_rootfs
echo "ERROR: Expanding failed..." echo "ERROR: Expanding failed..."
sleep 5 sleep 5
rm -f /etc/rc.local; cp -f /etc/rc.local.bak /etc/rc.local; /etc/rc.local if [[ -f /etc/rc.local.bak ]]; then
cp -f /etc/rc.local.bak /etc/rc.local
/etc/rc.local
fi
exit 0 exit 0
EOF1 EOF1
#####End no touch zone##### #####End no touch zone#####
chmod +x "$mountdir/etc/rc.local" chmod +x "$mountdir/etc/rc.local"
fi fi
umount "$mountdir" umount "$mountdir"
}
help() {
local help
read -r -d '' help << EOM
Usage: $0 [-adhrspvzZ] imagefile.img [newimagefile.img]
-s Don't expand filesystem when image is booted the first time
-v Be verbose
-r Use advanced filesystem repair option if the normal one fails
-z Compress image after shrinking with gzip
-Z Compress image after shrinking with xz
-a Compress image in parallel using multiple cores
-p Remove logs, apt archives, dhcp leases and ssh hostkeys
-d Write debug messages in a debug log file
EOM
echo "$help"
exit 1
}
should_skip_autoexpand=false
debug=false
repair=false
parallel=false
verbose=false
prep=false
ziptool=""
while getopts ":adhprsvzZ" opt; do
case "${opt}" in
a) parallel=true;;
d) debug=true;;
h) help;;
p) prep=true;;
r) repair=true;;
s) should_skip_autoexpand=true ;;
v) verbose=true;;
z) ziptool="gzip";;
Z) ziptool="xz";;
*) help;;
esac
done
shift $((OPTIND-1))
if [ "$debug" = true ]; then
info "Creating log file $LOGFILE"
rm "$LOGFILE" &>/dev/null
exec 1> >(stdbuf -i0 -o0 -e0 tee -a "$LOGFILE" >&1)
exec 2> >(stdbuf -i0 -o0 -e0 tee -a "$LOGFILE" >&2)
fi
echo "${0##*/} $version"
#Args
src="$1"
img="$1"
#Usage checks
if [[ -z "$img" ]]; then
help
fi
if [[ ! -f "$img" ]]; then
error $LINENO "$img is not a file..."
exit 2
fi
if (( EUID != 0 )); then
error $LINENO "You need to be running as root."
exit 3
fi
# check selected compression tool is supported and installed
if [[ -n $ziptool ]]; then
if [[ ! " ${ZIPTOOLS[@]} " =~ $ziptool ]]; then
error $LINENO "$ziptool is an unsupported ziptool."
exit 17
else
if [[ $parallel == true && $ziptool == "gzip" ]]; then
REQUIRED_TOOLS="$REQUIRED_TOOLS pigz"
else
REQUIRED_TOOLS="$REQUIRED_TOOLS $ziptool"
fi
fi
fi
#Check that what we need is installed
for command in $REQUIRED_TOOLS; do
command -v $command >/dev/null 2>&1
if (( $? != 0 )); then
error $LINENO "$command is not installed."
exit 4
fi
done
#Copy to new file if requested
if [ -n "$2" ]; then
f="$2"
if [[ -n $ziptool && "${f##*.}" == "${ZIPEXTENSIONS[$ziptool]}" ]]; then # remove zip extension if zip requested because zip tool will complain about extension
f="${f%.*}"
fi
info "Copying $1 to $f..."
cp --reflink=auto --sparse=always "$1" "$f"
if (( $? != 0 )); then
error $LINENO "Could not copy file..."
exit 5
fi
old_owner=$(stat -c %u:%g "$1")
chown "$old_owner" "$f"
img="$f"
fi
# cleanup at script exit
trap cleanup EXIT
#Gather info
info "Gathering data"
beforesize="$(ls -lh "$img" | cut -d ' ' -f 5)"
parted_output="$(parted -ms "$img" unit B print)"
rc=$?
if (( $rc )); then
error $LINENO "parted failed with rc $rc"
info "Possibly invalid image. Run 'parted $img unit B print' manually to investigate"
exit 6
fi
partnum="$(echo "$parted_output" | tail -n 1 | cut -d ':' -f 1)"
partstart="$(echo "$parted_output" | tail -n 1 | cut -d ':' -f 2 | tr -d 'B')"
if [ -z "$(parted -s "$img" unit B print | grep "$partstart" | grep logical)" ]; then
parttype="primary"
else
parttype="logical"
fi
loopback="$(losetup -f --show -o "$partstart" "$img")"
tune2fs_output="$(tune2fs -l "$loopback")"
rc=$?
if (( $rc )); then
echo "$tune2fs_output"
error $LINENO "tune2fs failed. Unable to shrink this type of image"
exit 7
fi
currentsize="$(echo "$tune2fs_output" | grep '^Block count:' | tr -d ' ' | cut -d ':' -f 2)"
blocksize="$(echo "$tune2fs_output" | grep '^Block size:' | tr -d ' ' | cut -d ':' -f 2)"
logVariables $LINENO beforesize parted_output partnum partstart parttype tune2fs_output currentsize blocksize
#Check if we should make pi expand rootfs on next boot
if [ "$parttype" == "logical" ]; then
echo "WARNING: PiShrink does not yet support autoexpanding of this type of image"
elif [ "$should_skip_autoexpand" = false ]; then
set_autoexpand
else else
echo "Skipping autoexpanding process..." echo "Skipping autoexpanding process..."
fi fi
@ -262,19 +312,20 @@ if [[ $prep == true ]]; then
umount "$mountdir" umount "$mountdir"
fi fi
#Make sure filesystem is ok #Make sure filesystem is ok
checkFilesystem checkFilesystem
if ! minsize=$(resize2fs -P "$loopback"); then if ! minsize=$(resize2fs -P "$loopback"); then
rc=$? rc=$?
error $LINENO "resize2fs failed with rc $rc" error $LINENO "resize2fs failed with rc $rc"
exit -10 exit 10
fi fi
minsize=$(cut -d ':' -f 2 <<< "$minsize" | tr -d ' ') minsize=$(cut -d ':' -f 2 <<< "$minsize" | tr -d ' ')
logVariables $LINENO minsize logVariables $LINENO currentsize minsize
if [[ $currentsize -eq $minsize ]]; then if [[ $currentsize -eq $minsize ]]; then
error $LINENO "Image already shrunk to smallest size" error $LINENO "Image already shrunk to smallest size"
exit -11 exit 11
fi fi
#Add some free space to the end of the filesystem #Add some free space to the end of the filesystem
@ -291,13 +342,14 @@ logVariables $LINENO minsize
#Shrink filesystem #Shrink filesystem
info "Shrinking filesystem" info "Shrinking filesystem"
resize2fs -p "$loopback" $minsize resize2fs -p "$loopback" $minsize
if [[ $? != 0 ]]; then rc=$?
error $LINENO "resize2fs failed" if (( $rc )); then
error $LINENO "resize2fs failed with rc $rc"
mount "$loopback" "$mountdir" mount "$loopback" "$mountdir"
mv "$mountdir/etc/rc.local.bak" "$mountdir/etc/rc.local" mv "$mountdir/etc/rc.local.bak" "$mountdir/etc/rc.local"
umount "$mountdir" umount "$mountdir"
losetup -d "$loopback" losetup -d "$loopback"
exit -12 exit 12
fi fi
sleep 1 sleep 1
@ -305,53 +357,64 @@ sleep 1
partnewsize=$(($minsize * $blocksize)) partnewsize=$(($minsize * $blocksize))
newpartend=$(($partstart + $partnewsize)) newpartend=$(($partstart + $partnewsize))
logVariables $LINENO partnewsize newpartend logVariables $LINENO partnewsize newpartend
if ! parted -s -a minimal "$img" rm "$partnum"; then parted -s -a minimal "$img" rm "$partnum"
rc=$? rc=$?
if (( $rc )); then
error $LINENO "parted failed with rc $rc" error $LINENO "parted failed with rc $rc"
exit -13 exit 13
fi fi
if ! parted -s "$img" unit B mkpart primary "$partstart" "$newpartend"; then parted -s "$img" unit B mkpart "$parttype" "$partstart" "$newpartend"
rc=$? rc=$?
if (( $rc )); then
error $LINENO "parted failed with rc $rc" error $LINENO "parted failed with rc $rc"
exit -14 exit 14
fi fi
#Truncate the file #Truncate the file
info "Shrinking image" info "Shrinking image"
if ! endresult=$(parted -ms "$img" unit B print free); then endresult=$(parted -ms "$img" unit B print free)
rc=$? rc=$?
if (( $rc )); then
error $LINENO "parted failed with rc $rc" error $LINENO "parted failed with rc $rc"
exit -15 exit 15
fi fi
endresult=$(tail -1 <<< "$endresult" | cut -d ':' -f 2 | tr -d 'B') endresult=$(tail -1 <<< "$endresult" | cut -d ':' -f 2 | tr -d 'B')
logVariables $LINENO endresult logVariables $LINENO endresult
if ! truncate -s "$endresult" "$img"; then truncate -s "$endresult" "$img"
rc=$? rc=$?
if (( $rc )); then
error $LINENO "trunate failed with rc $rc" error $LINENO "trunate failed with rc $rc"
exit -16 exit 16
fi fi
if [[ $xzip_compress == true ]]; then # handle compression
info "XZipping the shrunk image" if [[ -n $ziptool ]]; then
if [[ ! $(xz -9v "$img") ]]; then options=""
img=$img.xz envVarname="${MYNAME^^}_${ziptool^^}" # PISHRINK_GZIP or PISHRINK_XZ environment variables allow to override all options for gzip or xz
fi [[ $parallel == true ]] && options="${ZIP_PARALLEL_OPTIONS[$ziptool]}"
[[ -v $envVarname ]] && options="${!envVarname}" # if environment variable defined use these options
[[ $verbose == true ]] && options="$options -v" # add verbose flag if requested
if [[ $parallel == true ]]; then
parallel_tool="${ZIP_PARALLEL_TOOL[$ziptool]}"
info "Using $parallel_tool on the shrunk image"
if ! $parallel_tool ${options} "$img"; then
rc=$?
error $LINENO "$parallel_tool failed with rc $rc"
exit 18
fi fi
if [[ $exzip_compress == true ]]; then else # sequential
info "Extreme XZipping the shrunk image" info "Using $ziptool on the shrunk image"
if [[ ! $(xz -9ve "$img") ]]; then if ! $ziptool ${options} "$img"; then
img=$img.xz rc=$?
error $LINENO "$ziptool failed with rc $rc"
exit 19
fi fi
fi fi
img=$img.${ZIPEXTENSIONS[$ziptool]}
if [[ $gzip_compress == true ]]; then
info "Gzipping the shrunk image"
if [[ ! $(gzip -f9 "$img") ]]; then
img=$img.gz
fi
fi fi
aftersize=$(ls -lh "$img" | cut -d ' ' -f 5) aftersize=$(ls -lh "$img" | cut -d ' ' -f 5)