25 Commits
Author SHA1 Message Date
Hartmut Bauer a2d9b6a74e exclude container overlays from backup 2025-12-18 14:46:12 +01:00
Hartmut Bauer c89cfe66e9 30 min delay for mail.sh 2025-08-16 13:13:53 +02:00
Hartmut Bauer eaab960133 fixup! fix check-services, fix export disc detection 2025-07-30 12:29:15 +02:00
Hartmut Bauer a8bccb8cb2 fix check-services, fix export disc detection 2025-07-30 12:15:29 +02:00
Hartmut Bauer 03df70a244 podman migration done, auto-update fixes 2025-05-16 21:23:25 +02:00
Hartmut Bauer bf8c226bbe prevent ssh errors 2025-05-11 17:38:29 +02:00
Hartmut Bauer 5696e01322 fix remote backup 2025-05-11 15:19:04 +02:00
Hartmut Bauer f96447d0e7 split daily/cron daily 2025-05-11 15:14:48 +02:00
Hartmut Bauer 8eccb0ed59 sort tar archive 2025-05-11 14:56:45 +02:00
Hartmut Bauer 6efd48d3c6 fix log overwrite 2025-05-11 13:48:47 +02:00
Hartmut Bauer 16c8f1afce fix rc handling 2025-05-11 13:48:06 +02:00
Hartmut Bauer 21be8f3864 fix mail.sh, log results of auto update 2025-05-10 14:42:05 +02:00
Hartmut Bauer e7b9cc4e15 all docker references cleared, fixes 2025-05-09 18:57:23 +02:00
Hartmut Bauer 27e0889bd1 mail.sh an neuen Container angepasst 2025-05-08 16:59:25 +02:00
Hartmut Bauer 912d4b9434 Fix DB-Dump 2025-05-05 13:07:14 +02:00
Hartmut Bauer a8a00a9f46 Weitere podman-Anpassungen, DB-Dump geändert 2025-05-05 02:09:02 +02:00
Hartmut Bauer 15f77b6bf5 1st step to podman 2025-04-29 22:56:18 +02:00
Hartmut Bauer 42af16928f weekly SA->SO 2025-04-29 15:28:51 +02:00
Hartmut Bauer 39bdfbd021 bugfix 2025-04-15 22:49:29 +02:00
Hartmut Bauer 0cb2748447 exclude lists from export 2025-04-15 12:15:42 +02:00
Hartmut Bauer 02f82e8147 run weekly parallel to daily 2025-04-14 16:22:30 +02:00
Hartmut Bauer d26f7700bd Small fixes 2025-04-13 19:53:54 +02:00
Hartmut Bauer 328936748e Restore, List Archives, Content 2025-04-13 02:36:47 +02:00
Hartmut Bauer ab61e0e916 Fix log+backup acls 2025-04-13 00:10:00 +02:00
Hartmut Bauer 80c5bb0ff0 Release 2.1.0 2025-04-11 23:03:47 +02:00
16 changed files with 650 additions and 303 deletions

No files matched your search

+1 -1
View File
@@ -1,6 +1,6 @@
MIT License MIT License
Copyright (c) 2025 hb Copyright (c) 2025 Hartmut Bauer
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
-1
View File
@@ -24,5 +24,4 @@ daily/nathan/backup.1 20GB/ 0:40h
... ...
backup.7 backup.7
weekly/synology 320GB/ 9:30h weekly/synology 320GB/ 9:30h
logs/<hostname>/
``` ```
+35 -35
View File
@@ -1,26 +1,26 @@
### TODOs ### TODOs
- code cleanup (-> export!)
- test export
- Doku - Doku
- (x) code cleanup (-> export!)
- (x) test export
DB-Dump DB-Dump
(x) new for postgres - (x) new for postgres
(x) jede db einzeln? - (x) jede db einzeln?
(x) +mariadb - (x) +mariadb
(-) hooks für weitere - (-) hooks für weitere
nextcloud: nextcloud:
(x) vol/nextcloud/data/data/hb zeigt auf NAS-share - (x) vol/nextcloud/data/data/hb zeigt auf NAS-share
(x) -> alles auf Server - (x) -> alles auf Server
backup MAC backup MAC
(x) ccc 2x 2TB intern (weekly) - (x) ccc 2x 2TB intern (weekly)
(x) ccc 2x 2TB extern - (x) ccc 2x 2TB extern
(x) TODO: SSDs auf Verschlüsselung umstellen - (x) TODO: SSDs auf Verschlüsselung umstellen
(x) TODO: Kalendereintrag für wöchentlichen backup - (x) TODO: Kalendereintrag für wöchentlichen backup
nas-backup daily nas-backup daily
- (x) tar, gpg to nas - (x) tar, gpg to nas
@@ -33,36 +33,36 @@ nas-backup daily
- (x) TODO: script (python?) - (x) TODO: script (python?)
nas-backup weekly 1 nas-backup weekly 1
(x) media/books, media/music/_all + _hoerspiel -> tar, gpg nach //backup - (x) media/books, media/music/_all + _hoerspiel -> tar, gpg nach //backup
(x) TODO: s. nas-backup daily - (x) TODO: s. nas-backup daily
logging logging
(x) für auto-fns mail() - (x) für auto-fns mail()
(x) mail() loggt in extra Datei und ruft log(), für std log - (x) mail() loggt in extra Datei und ruft log(), für std log
(x) Name std log in mail() ausgeben - (x) Name std log in mail() ausgeben
(-) mail log lokal anlegen (wo?) - (-) mail log lokal anlegen (wo?)
(-) am Ende von auto mail log in mail und file löschen -> py-script - (-) am Ende von auto mail log in mail und file löschen -> py-script
(x) alle Programme müssen stdout/stderr auf std log umleiten - (x) alle Programme müssen stdout/stderr auf std log umleiten
(x) für jeden Lauf neue log datei (+date) - (x) für jeden Lauf neue log datei (+date)
(-) Name der log Datei bei !auto auf stderr ausgeben (damit stdout ggf. geparst werden kann, alt.: param --machine) - (-) Name der log Datei bei !auto auf stderr ausgeben (damit stdout ggf. geparst werden kann, alt.: param --machine)
(x) log file in nfs/backups/log/$HOST_NAME-$(date '+%Y-%m-%d_%H%M').log - (x) log file in nfs/backups/log/$HOST_NAME-$(date '+%Y-%m-%d_%H%M').log
(x) bei jedem Lauf alte logs löschen (7d?) -> wie? - (x) bei jedem Lauf alte logs löschen (7d?) -> wie?
(-) evtl. logs lokal ablegen und am Ende auf NAS kopieren - (-) evtl. logs lokal ablegen und am Ende auf NAS kopieren
backup server+nas extern backup server+nas extern
(x) rsync nas-backup 2x HDD intern (weekly) (5TB?) - (x) rsync nas-backup 2x HDD intern (weekly) (5TB?)
(x) TODO: script (python?) mit HD-Erkennung - (x) TODO: script (python?) mit HD-Erkennung
(x) TODO: zusätzliche HDD - (x) TODO: zusätzliche HDD
(x) TODO: Kalendereintrag, s. backup MAC - (x) TODO: Kalendereintrag, s. backup MAC
export backups export backups
(x) rsync nas-backup 2x 2TB HDD extern - (x) rsync nas-backup 2x 2TB HDD extern
(x) TODO: script, s. backup server+nas - (x) TODO: script, s. backup server+nas
export serien export serien
(x) rsync dir 1x 5TB HDD extern - (x) rsync dir 1x 5TB HDD extern
(x) TODO: vorhandenes script in neue Routine einbauen - (x) TODO: vorhandenes script in neue Routine einbauen
export video export video
(x) rsync dir 1x 5TB HDD extern - (x) rsync dir 1x 5TB HDD extern
(x) TODO: s. export serien - (x) TODO: s. export serien
+1
View File
@@ -1,3 +1,4 @@
#!/usr/bin/env bash #!/usr/bin/env bash
cp -v *.sh ../../bin/backup cp -v *.sh ../../bin/backup
cp -v save-conf ../../bin
Executable
+94
View File
@@ -0,0 +1,94 @@
#!/usr/bin/eval bash
PROTON_TARGET="/Users/hb/mnt/nas/backups/div"
PROTON_TARGET_FILE="proton_mail_pass.$EXT"
PROTON_ROOT="/Users/hb/Downloads"
PROTON_BACKUP_DIR="Proton_Backup"
PROTON_BACKUP="$PROTON_ROOT/$PROTON_BACKUP_DIR"
PROTON_MAIL_EXPORT="$PROTON_ROOT/proton-mail-export-cli"
PROTON_PASS_EXPORT="$PROTON_ROOT/Proton Pass_export"
check_proton() {
if [ ! -d "$PROTON_MAIL_EXPORT" ]; then
mail_log $ERROR "$ERR there is no Proton Mail export."
return 1
fi
# tr ist nötig, weil bei macOS führene blanks entfernt werden müssen
if [ "$(ls "$PROTON_PASS_EXPORT"*|wc -l|tr -d ' ')" != "1" ]; then
mail_log $ERROR "$ERR there is no Proton Pass export."
return 1
fi
if [ ! -d "$PROTON_TARGET" ]; then
mail_log $ERROR "$ERR could not find targe dir $PROTON_TARGET!"
return 1
fi
if [ -f "$PROTON_TARGET/$PROTON_TARGET_FILE" ]; then
log $DEBUG "$DBG > rm -f \"$PROTON_TARGET/$PROTON_TARGET_FILE\""
rm -f "$PROTON_TARGET/$PROTON_TARGET_FILE" >>$LOG_FILE 2>&1
fi
}
delete_proton_backup_dir() {
log $DEBUG "$DBG delete_proton_backup_dir()"
if [ -d "$PROTON_BACKUP" ]; then
log $DEBUG "$DBG > rm -rf \"$PROTON_BACKUP\""
rm -rf "$PROTON_BACKUP" >>$LOG_FILE 2>&1 && return 0
fi
return 1
}
create_proton_backup_dir() {
log $DEBUG "$DBG create_proton_backup_dir()"
delete_proton_backup_dir
log $DEBUG "$DBG > mkdir -p \"$PROTON_BACKUP\""
mkdir -p "$PROTON_BACKUP" >>$LOG_FILE 2>&1 && return 0
return 1
}
move_proton_exports() {
log $DEBUG "$DBG move_proton_exports()"
log $DEBUG "$DBG > mv \"$PROTON_MAIL_EXPORT\" \"$PROTON_BACKUP\""
mv "$PROTON_MAIL_EXPORT" "$PROTON_BACKUP" >>$LOG_FILE 2>&1
if [ $? -ne 0 ]; then return 1; fi
log $DEBUG "$DBG > mv \"$PROTON_PASS_EXPORT\"* \"$PROTON_BACKUP\""
mv "$PROTON_PASS_EXPORT"* "$PROTON_BACKUP" >>$LOG_FILE 2>&1
if [ $? -ne 0 ]; then return 1; fi
}
do_proton_backup() {
log $DEBUG "$DBG do_proton_backup()"
mail_log $INFO "$INF checking environment"
check_proton
is_error $? && return 1
mail_log $INFO "$INF create $PROTON_BACKUP"
create_proton_backup_dir
is_error $? && return 1
mail_log $INFO "$INF move Proton exports to source directory"
move_proton_exports
is_error $? && return 1
mail_log $INFO "$INF backing up Proton exports to $PROTON_TARGET/$PROTON_TARGET_FILE"
backup_directory "$PROTON_ROOT" "$PROTON_TARGET/$PROTON_TARGET_FILE" "$PROTON_BACKUP_DIR"
is_error $? && return 1
}
proton_backup() {
mail_log $INFO ">>>>>>>>>> Proton backup $VER at $HOST_NAME <<<<<<<<<<"
do_proton_backup
mail_log $INFO "$INF deleting $PROTON_BACKUP"
delete_proton_backup_dir
is_error $?
mail_log $DEBUG "$DBG rm -f \"$PROTON_PASS_EXPORT\"*"
mail_log $DEBUG "$(rm -f "$PROTON_PASS_EXPORT"* 2>&1)"
mail_log $INFO ">>>>>>>>>> end of Proton backup"
send_mail_log "Proton Backup"
}
Executable
+41
View File
@@ -0,0 +1,41 @@
#!/bin/bash
#
# check if container services are running
UNITS="/var/containers/systemd"
FAILED=""
check_service() {
local service=$1
systemctl is-active --user --machine=container@ $service >/dev/null
if [ $? -ne 0 ]; then
FAILED="$FAILED $service"
fi
}
check() {
local type=$1
for i in $UNITS/$type/*.socket; do
check_service $(basename $i)
done
for i in $UNITS/$type/*.container; do
check_service $(basename $i .container).service
done
}
check user
check system
if [ -z "$FAILED" ]; then
echo "OK All services are running"
exit 0
else
echo "#" >&2
echo "# ERROR some services are not running:" >&2
echo "#" >&2
echo "# $FAILED" >&2
echo "#" >&2
exit 1
fi
+1 -1
View File
@@ -3,4 +3,4 @@
HERE=$(dirname "$0") HERE=$(dirname "$0")
# local start # local start
sudo sh -c "nohup $HERE/main.sh -d </dev/null &" sudo sh -c "nohup $HERE/main.sh --dx </dev/null &"
Executable
+19
View File
@@ -0,0 +1,19 @@
#!/bin/bash
# falls das nochmal gebraucht wird
HERE=$(dirname $0)
DOCKER=/usr/bin/docker
ORACLEDUMP="$DOCKER exec -i oracle /opt/oracle/product/18c/dbhomeXE/bin/expdp"
ORACLECONNECT="sbsdb_master/sbsdbpwd@//localhost:1521/xepdb1"
ORACLEDATAPUMP=/var/docker/vol/oracle/dump
function dump_oracle() {
mkdir -p $HERE/oracle > /dev/null 2>&1
echo "$(date) - Oracle dump"
$ORACLEDUMP $ORACLECONNECT dumpfile=sbsdb_dump:sbsdb.dmp logfile=sbsdb_dump:sbsdb.log reuse_dumpfiles=yes
mv $ORACLEDATAPUMP/* $HERE/oracle
}
dump_oracle
+13 -12
View File
@@ -1,32 +1,33 @@
#!/usr/bin/eval bash #!/usr/bin/eval bash
USER_CONTAINERS=
ROOT_CONTAINERS="pihole"
NAS_ROOT=/var/srv/nas/backups BACKUP_TARGET=/var/containers/tmp
BACKUP_TARGET=/var/docker/tmp
NAS_TARGET="$NAS_ROOT/daily/$HOST_NAME/backup.$DAY" NAS_TARGET="$NAS_ROOT/daily/$HOST_NAME/backup.$DAY"
backup_daily1() { backup_daily1() {
BACKUP_SOURCE="/var" BACKUP_SOURCE="/var"
BACKUP_EXCLUDE_DIRS=(docker) BACKUP_EXCLUDE_DIRS=(containers)
} }
backup_daily2() { backup_daily2() {
BACKUP_SOURCE="/var/docker" BACKUP_SOURCE="/var/containers"
BACKUP_EXCLUDE_DIRS=(tmp) BACKUP_EXCLUDE_DIRS=(tmp)
} }
before_backup() { before_backup() {
stop_docker stop_containers
return $? return $?
} }
after_backup() { after_backup() {
start_docker let local rc=0
local dock=$? start_containers
let rc=rc+$?
copy_to_nas "$BACKUP_TARGET" "$NAS_TARGET" copy_to_nas "$BACKUP_TARGET" "$NAS_TARGET"
local copy=$? let rc=rc+$?
if [ $dock -eq 0 ] && [ $copy -eq 0 ]; then #check_services
return 0 #let rc=rc+$?
fi return $rc
return 1
} }
+24 -6
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Mail via GMX-Relay versenden (Docker-Image "hb42/smtp") # Mail via GMX-Relay versenden (Podman-Image "smtp-relay")
# #
# Klappt nur direkt im Container, daher ggf. via ssh aufrufen (Annahme: remote user == local user) # Klappt nur direkt im Container, daher ggf. via ssh aufrufen (Annahme: remote user == local user)
# #
@@ -12,10 +12,15 @@
# > mail.sh hb242@pm.me "log whatever" ../log/some.log # > mail.sh hb242@pm.me "log whatever" ../log/some.log
# #
DOCKER=/usr/bin/docker PODMAN=/usr/bin/podman
MAILCONTAINER=smtp #
# -i ist entscheident, damit die restliche Kommandozeile sauber ausgewertet wird
# das ersetzt den expliziten Aufruf mit "sh -c "..." "
SUDO="sudo -u container -i"
MAILCONTAINER=smtp-relay
MAILSERVER_IP=10.4.2.30 MAILSERVER_IP=10.4.2.30
SSH="" REMOTE=""
which ip >/dev/null 2>&1 which ip >/dev/null 2>&1
if [ $? -eq 0 ]; then if [ $? -eq 0 ]; then
@@ -27,7 +32,7 @@ else
fi fi
if [ "$myip" != "$MAILSERVER_IP" ]; then if [ "$myip" != "$MAILSERVER_IP" ]; then
SSH="ssh $MAILSERVER_IP" REMOTE=yes
fi fi
TO=$1 TO=$1
@@ -41,11 +46,24 @@ if [ -z "$TO" ] || [ -z "$SUBJECT" ]; then
exit exit
fi fi
PODMANCMD="$SUDO $PODMAN"
if [ ! -z "$REMOTE" ]; then
PODMANCMD="ssh $MAILSERVER_IP $PODMANCMD"
fi
# der Mail-Container sollte laufen, falls nicht, dann ist wahrscheinlich der Backup
# noch nicht durch => 30 Minuten warten
$PODMANCMD ps | grep $MAILCONTAINER >/dev/null
if [ $? -ne 0 ]; then
sleep 30m
fi
# Mail body aus Datei oder STDIN holen # Mail body aus Datei oder STDIN holen
CONTENT=$(cat ${1:--}) CONTENT=$(cat ${1:--})
MAILCMD="$PODMANCMD exec -i $MAILCONTAINER /usr/sbin/sendmail -i -t"
# construct mail header + content and pipe to sendmail # construct mail header + content and pipe to sendmail
cat <<EOMB | $SSH $DOCKER exec -i $MAILCONTAINER /usr/sbin/sendmail -i -t cat <<EOMB | $MAILCMD
FROM: hb42@gmx.de FROM: hb42@gmx.de
TO: ${TO} TO: ${TO}
DATE: $(date -R) DATE: $(date -R)
+299 -95
View File
@@ -1,20 +1,38 @@
HOST_NAME=$(hostname -s) HOST_NAME=$(hostname -s)
DAY=$(date +%u) # MO == 1 DAY=$(date +%u) # 1-7, MO is 1
WEEKLY_DAY=7 # SA->SO
BACKUP_MASTER_CHECK="/var/docker/vol/proxy" NAS_MOUNT_POINT="$(realpath $HERE/../../..)" # Server/bin/backup
NAS_ROOT="$NAS_MOUNT_POINT/backups"
LOG_FILE_PATH="$NAS_MOUNT_POINT/Server/log/backup"
log_file_name="$LOG_FILE_PATH/$HOST_NAME-$(date '+%Y-%m-%d_%H%M%S')"
LOG_FILE="${log_file_name}.log"
LOG_KEEP_DAYS=14
RESTORE_PATH="$NAS_MOUNT_POINT/Server/restore"
BACKUP_MASTER_CHECK="/var/containers/vol/caddy"
NAS_HOSTNAME=synology NAS_HOSTNAME=synology
NAS_IP=10.4.2.20 NAS_IP=10.4.2.20
NAS_SCRIPT=/volume1/nfs/Server/bin/backup/main.sh NAS_SCRIPT=/volume1/nfs/Server/bin/backup/main.sh
WEEKLY_DAY=1 # SO->MO
PASSPHRASE=L1rfCXu8PHMQsGL0HJWwzQYpCT7f5VboislOTHMaEV PASSPHRASE=L1rfCXu8PHMQsGL0HJWwzQYpCT7f5VboislOTHMaEV
EXT=tgz.gpg EXT=tgz.gpg
LIST_EXT=txt LIST_EXT=list
DOCKER="/usr/bin/docker compose -f /var/docker/run/docker-compose.yml --ansi never" #DOCKER="/usr/bin/docker compose -f /var/docker/run/docker-compose.yml --ansi never"
SYNO_USB=/usr/syno/bin/synousbdisk SYNO_USB=/usr/syno/bin/synousbdisk
SYSTEMCTL_USER="/usr/bin/systemctl --user --machine=container@"
SYSTEMCTL="/usr/bin/systemctl"
PODMAN=" sudo -u container -i podman"
CONTAINER_EXEC_USER=" $PODMAN exec"
CONTAINER_SVC_UPDATE_USER=" $PODMAN auto-update"
CONTAINER_EXEC="podman exec"
tagP1=".backup#"
tagP2=".tag"
LOG_RECIPIENT=hb242@pm.me LOG_RECIPIENT=hb242@pm.me
INFO=INFO INFO=INFO
@@ -71,11 +89,12 @@ send_mail_log() {
# Alte Logs löschen # Alte Logs löschen
# #
# Dateien älter als $LOG_KEEP_DAYS Tage löschen
#
prune_logs() { prune_logs() {
log $DEBUG "$DBG prune_logs()" log $DEBUG "$DBG prune_logs()"
# Dateien älter als 14 Tage löschen log $DEBUG "$DBG > find \"$LOG_FILE_PATH\" -type f -mtime +$LOG_KEEP_DAYS -delete"
log $DEBUG "$DBG > find \"$LOG_FILE_PATH\" -type f -mtime +14 -delete" find "$LOG_FILE_PATH" -type f -mtime +$LOG_KEEP_DAYS -delete >>$LOG_FILE 2>&1
find "$LOG_FILE_PATH" -type f -mtime +14 -delete >>$LOG_FILE 2>&1
if [ $? -ne 0 ]; then if [ $? -ne 0 ]; then
# Fehler nur protokollieren, kein Grund abzubrechen # Fehler nur protokollieren, kein Grund abzubrechen
mail_log $ERROR "$ERR error deleting old log files." mail_log $ERROR "$ERR error deleting old log files."
@@ -84,32 +103,36 @@ prune_logs() {
# ----- db dump ----- # ----- db dump -----
dump_mariadb() { #dump_mariadb() {
log $DEBUG "$DBG dump_mariadb( $@ )" # return 0
local dump_dir=$1 # log $DEBUG "$DBG dump_mariadb( $@ )"
local mysqlconnect="--user=root --password=snafu" # local dump_dir=$1
# local mysqlconnect="--user=root --password=snafu"
log $DEBUG "$DBG > \$DOCKER exec mysql /usr/bin/mariadb --silent --user=xxx --password=xxx -e \"show databases\" -B" #
for db in $($DOCKER exec mysql /usr/bin/mariadb --silent $mysqlconnect -e "show databases" -B 2>>$LOG_FILE | grep -v information_schema 2>>$LOG_FILE | grep -v performance_schema 2>>$LOG_FILE | sed -e 's/\r//' 2>>$LOG_FILE); do # log $DEBUG "$DBG > \$DOCKER exec mysql /usr/bin/mariadb --silent --user=xxx --password=xxx -e \"show databases\" -B"
log $DEBUG "$DBG > \$DOCKER exec mysql /usr/bin/mariadb-dump --hex-blob --user=xxx --password=xxx --databases $db" # for db in $($DOCKER exec mysql /usr/bin/mariadb --silent $mysqlconnect -e "show databases" -B 2>>$LOG_FILE | grep -v information_schema 2>>$LOG_FILE | grep -v performance_schema 2>>$LOG_FILE | sed -e 's/\r//' 2>>$LOG_FILE); do
$DOCKER exec mysql /usr/bin/mariadb-dump --hex-blob $mysqlconnect --databases $db >$dump_dir/$db.sql 2>>$LOG_FILE # log $DEBUG "$DBG > \$DOCKER exec mysql /usr/bin/mariadb-dump --hex-blob --user=xxx --password=xxx --databases $db"
is_error $? false && return 1 # $DOCKER exec mysql /usr/bin/mariadb-dump --hex-blob $mysqlconnect --databases $db >$dump_dir/$db.sql 2>>$LOG_FILE
done # is_error $? false && return 1
return 0 # done
} # return 0
#}
dump_postgresql() { dump_postgresql() {
local pg_container=$1
local dump_dir=$2
log $DEBUG "$DBG dump_postgresql( $@ )" log $DEBUG "$DBG dump_postgresql( $@ )"
local dump_dir=$1 delete_dir $dump_dir
check_directory $dump_dir
# users and roles # users and roles
log $DEBUG "$DBG > \$DOCKER exec -u=postgres postgres pg_dumpall --globals-only" log $DEBUG "$DBG > \$CONTAINER_EXEC_USER -u=postgres $pg_container pg_dumpall --globals-only"
$DOCKER exec -u=postgres postgres pg_dumpall --globals-only > $dump_dir/_globals.sql 2>>$LOG_FILE $CONTAINER_EXEC_USER -u=postgres $pg_container pg_dumpall --globals-only > $dump_dir/_globals.sql 2>>$LOG_FILE
log $DEBUG "$DBG \$DOCKER exec -u=postgres postgres psql -t -c \"select datname from pg_database where not datistemplate\"" log $DEBUG "$DBG \$CONTAINER_EXEC_USER -u=postgres $pg_container psql -t -c \"select datname from pg_database where not datistemplate\""
for db in $($DOCKER exec -u=postgres postgres psql -t -c "select datname from pg_database where not datistemplate" 2>>$LOG_FILE); do for db in $($CONTAINER_EXEC_USER -u=postgres $pg_container psql -t -c "select datname from pg_database where not datistemplate" 2>>$LOG_FILE); do
log $DEBUG "$DBG > \$DOCKER exec -u=postgres postgres pg_dump --create $db" log $DEBUG "$DBG > \$CONTAINER_EXEC_USER -u=postgres $pg_container pg_dump --create $db"
$DOCKER exec -u=postgres postgres pg_dump --create $db > $dump_dir/$db.sql 2>>$LOG_FILE $CONTAINER_EXEC_USER -u=postgres $pg_container pg_dump --create $db > $dump_dir/$db.sql 2>>$LOG_FILE
is_error $? false && return 1 is_error $? false && return 1
done done
return 0 return 0
@@ -117,18 +140,126 @@ dump_postgresql() {
# ----- helper functions ----- # ----- helper functions -----
start_docker() { # Für Backup und Export muss das Script mit root-Rechten
log $DEBUG "$DBG start_docker()" # ausgeführt werden.
log $DEBUG "$DBG > \$DOCKER up -d" #
$DOCKER up -d >>$LOG_FILE 2>&1 check_root() {
if [ "$(id -u)" -ne 0 ]; then
log $ERROR "$ERR script must be run as superuser."
echo "****"
echo "***** script must be run as superuser! *****"
echo "****"
exit 100
fi
}
# Ausführung auf macOS begrenzen
#
check_mac() {
if [[ "$(uname)" != "Darwin" ]]; then
log $ERROR "$ERR script works only on MAC."
echo "****"
echo "***** script works only on MAC! *****"
echo "****"
exit 110
fi
}
start_containers() {
local let rc=0
log $DEBUG "$DBG start_containers()"
# if [ -f /var/docker/run/docker-compose.yml ]; then
# log $DEBUG "$DBG > \$DOCKER up -d"
# $DOCKER up -d >>$LOG_FILE 2>&1
# let rc=rc+$?
# fi
if [ -d /var/containers ]; then
do_start_stop_containers start
let rc=rc+$?
# warten bis alle wieder da sind
sleep 2m
fi
return $rc
}
stop_containers() {
local let rc=0
log $DEBUG "$DBG stop_containers()"
# if [ -f /var/docker/run/docker-compose.yml ]; then
# log $DEBUG "$DBG > \$DOCKER down"
# $DOCKER down >>$LOG_FILE 2>&1
# let rc=rc+$?
# fi
if [ -d /var/containers ]; then
do_start_stop_containers stop
let rc=rc+$?
fi
return $rc
}
do_start_stop_containers() {
local cmd=$1
local let rc=0
log $DEBUG "$DBG do_start_stop_containers() $cmd"
if [ ! -z "$ROOT_CONTAINERS" ]; then
log $DEBUG "$DBG > $SYSTEMCTL $cmd $ROOT_CONTAINERS"
$SYSTEMCTL $cmd $ROOT_CONTAINERS >>$LOG_FILE 2>&1
let rc=rc+$?
fi
if [ ! -z "$USER_CONTAINERS" ]; then
log $DEBUG "$DBG > $SYSTEMCTL_USER $cmd $USER_CONTAINERS"
$SYSTEMCTL_USER $cmd $USER_CONTAINERS >>$LOG_FILE 2>&1
let rc=rc+$?
fi
return $rc
}
# podman auto-update nach dem Backup starten
#
# Die fn schreibt sehr viel ins Log, damit der Update ggf. nachvollzogen werden kann,
# wenn etwas daneben geht.
#
update_services() {
log $DEBUG "$DBG update_services()"
mail_log $INFO "$INF update services"
local old_img=$($PODMAN ps --filter "label=io.containers.autoupdate=registry" --format "table {{.ImageID}} {{.Names}} {{.Image}}" --sort names)
echo "$old_img" >>$LOG_FILE 2>&1
local plan=$($CONTAINER_SVC_UPDATE_USER --dry-run --format "{{.Image}} {{.Updated}}" 2>&1 | grep pending | sed 's/^\(.*\) pending/\1/')
for i in $plan; do
mail_log $INFO "$INF pending: $i"
done
local upd=$($CONTAINER_SVC_UPDATE_USER --format "table {{.Image}} {{.Updated}}" 2>&1 | tee -a $LOG_FILE | grep true | sed 's/^\(.*\) true/\1/')
check_pipe_rc ${PIPESTATUS[@]}
local rc=$?
for i in $upd; do
mail_log $INFO "$INF updated: $i"
done
# warten bis alle wieder da sind (wg. ps)
sleep 2m >>$LOG_FILE 2>&1
local new_img=$($PODMAN ps --filter "label=io.containers.autoupdate=registry" --format "table {{.ImageID}} {{.Names}} {{.Image}}" --sort names)
echo "$new_img" >>$LOG_FILE 2>&1
# diff output bring zu wenig
#log $DEBUG "$DBG diff before after"
#local diff_img=$(diff <(echo "$old_img") <(echo "$new_img"))
#echo "$diff_img" >> $LOG_FILE
return $rc
}
check_services() {
log $DEBUG "$DBG check_services()"
mail_log $INFO "$INF check services"
$HERE/check.sh >>$LOG_FILE 2>&1
return $? return $?
} }
stop_docker() { # Zugriffsrechte für Logs und Backups setzen
log $DEBUG "$DBG stop_docker()" #
log $DEBUG "$DBG > \$DOCKER down" fix_acls() {
$DOCKER down >>$LOG_FILE 2>&1 log $DEBUG "$DBG fix_acls()"
return $?
log $DEBUG "$DBG > chmod -R a+r $NAS_ROOT"
chmod -R a+r $NAS_ROOT >>$LOG_FILE 2>&1
log $DEBUG "$DBG > chmod -R a+r $LOG_FILE_PATH"
chmod -R a+r $LOG_FILE_PATH >>$LOG_FILE 2>&1
} }
# Script per ssh auf Server $1 im Hintergrund starten # Script per ssh auf Server $1 im Hintergrund starten
@@ -314,33 +445,58 @@ backup_all_dirs() {
for dir in $(iterate_dirs_in "$root" "$@"); do for dir in $(iterate_dirs_in "$root" "$@"); do
tar_file="$target/$prefix-$dir.$EXT" tar_file="$target/$prefix-$dir.$EXT"
backup_dir "$root" "$tar_file" "$dir" backup_directory "$root" "$tar_file" "$dir"
if [ $? -ne 0 ]; then return 1; fi if [ $? -ne 0 ]; then return 1; fi
create_list "$tar_file" "$dir"
if [ $? -ne 0 ]; then return 1; fi
test_integrity "$tar_file"
if [ $? -ne 0 ]; then return 1; fi
done done
return 0 return 0
} }
# Einzelnes Verzeichnis sichern. # Einzelnes Verzeichnis sichern und Backup prüfen
# #
# $1 - Pfad um Verzeichnis, das gesichert werden soll # $1 - Pfad zum Verzeichnis, das gesichert werden soll
# $2 - Zielpfad und -name für das Archiv # $2 - Zielpfad und -name für das Archiv
# $2 - zu sicherndes Verzeichnis # $2 - zu sicherndes Verzeichnis
# #
# returns 0 for success, 1 otherwise # returns 0 for success, 1 otherwise
# #
backup_dir() {
log $DEBUG "$DBG backup_dir( $@ )" backup_directory() {
log $DEBUG "$DBG backup_directory( $@ )"
local root=$1
local tar_file=$2
local dir=$3
tar_dir "$root" "$tar_file" "$dir"
if [ $? -ne 0 ]; then return 1; fi
create_list "$tar_file" "$dir"
if [ $? -ne 0 ]; then return 1; fi
test_integrity "$tar_file"
if [ $? -ne 0 ]; then return 1; fi
}
# Einzelnes Verzeichnis sichern.
#
# $1 - Pfad zum Verzeichnis, das gesichert werden soll
# $2 - Zielpfad und -name für das Archiv
# $2 - zu sicherndes Verzeichnis
#
# returns 0 for success, 1 otherwise
#
tar_dir() {
log $DEBUG "$DBG tar_dir( $@ )"
local root=$1 local root=$1
local archive=$2 local archive=$2
local dir=$3 local dir=$3
log $DEBUG "$DBG > tar -c -z --numeric-owner --one-file-system --exclude \"@eaDir\" -C \"$root\" \"$dir\" | gpg --batch -c --passphrase xxxx -o \"$archive\"" local sort=--sort=name
tar -c -z --numeric-owner --one-file-system --exclude "@eaDir" -C "$root" "$dir" 2>>$LOG_FILE | gpg --batch -c --passphrase $PASSPHRASE -o "$archive" 2>>$LOG_FILE if [[ "$(uname)" == "Darwin" ]]; then
sort=""
fi
log $DEBUG "$DBG > tar -c -z --warning=no-file-ignored --numeric-owner --one-file-system $sort --exclude \"@eaDir\" -C \"$root\" \"$dir\" | gpg --batch -c --passphrase xxxx -o \"$archive\""
tar -c -z--warning=no-file-ignored --numeric-owner --one-file-system $sort --exclude "@eaDir" -C "$root" "$dir" 2>>$LOG_FILE | gpg --batch -c --passphrase $PASSPHRASE -o "$archive" 2>>$LOG_FILE
check_pipe_rc ${PIPESTATUS[@]} check_pipe_rc ${PIPESTATUS[@]}
return $? return $?
} }
@@ -362,7 +518,7 @@ do_auto_backup() {
local rc local rc
if ! $func$count 2>/dev/null ; then # Seiteneffekt: $func$count wird ausgeführt if ! $func$count 2>/dev/null ; then # Seiteneffekt: $func$count wird ausgeführt
mail_log $ERROR "$ERR keine Backup-Function $func$count() für $HOST_NAME." log $ERROR "$ERR keine Backup-Function $func$count() für $HOST_NAME."
exit 1 # kein Mail-Benachrichtigung! exit 1 # kein Mail-Benachrichtigung!
fi fi
set_log_info set_log_info
@@ -399,16 +555,27 @@ set_log_info() {
mail_log $INFO ">>> " mail_log $INFO ">>> "
} }
backup_daily() { cron_backup_daily() {
# "master"-Server startet Backup auf NAS # "master"-Server startet Backup auf NAS
if [ -d $BACKUP_MASTER_CHECK ]; then if [ -d $BACKUP_MASTER_CHECK ]; then
log $DEBUG "$DBG starting daily backup on $NAS_HOSTNAME" log $DEBUG "$DBG starting daily backup on $NAS_HOSTNAME"
run_script_remote $NAS_IP "$NAS_SCRIPT -d" run_script_remote $NAS_IP "$NAS_SCRIPT --dx"
# alte Logs löschen (nur auf einem Server ausführen) # alte Logs löschen (nur auf einem Server ausführen)
prune_logs prune_logs
fi fi
# ggf. weekly backup parallel starten (z.ZT. nur auf NAS)
if [ $HAS_WEEKLY -a $WEEKLY_DAY -eq $DAY ]; then
log $DEBUG "$DBG starting weekly backup"
sleep 2 # wg. log name
nohup $THIS_SCRIPT -w >>$LOG_FILE 2>&1 </dev/null &
fi
backup_daily
}
backup_daily() {
mail_log $INFO ">>>>>>>>>> daily auto backup $VER for $HOST_NAME <<<<<<<<<<" mail_log $INFO ">>>>>>>>>> daily auto backup $VER for $HOST_NAME <<<<<<<<<<"
do_auto_backup backup_daily do_auto_backup backup_daily
@@ -419,13 +586,9 @@ backup_daily() {
is_error $? is_error $?
mail_log $INFO ">>>>>>>>>> end of auto backup" mail_log $INFO ">>>>>>>>>> end of auto backup"
fix_acls
send_mail_log "Daily Backup" send_mail_log "Daily Backup"
# ggf. weekly backup starten (z.ZT. nur auf NAS)
if [ $HAS_WEEKLY -a $WEEKLY_DAY -eq $DAY ]; then
log $DEBUG "$DBG starting weekly backup"
nohup $THIS_SCRIPT -w >>$LOG_FILE 2>&1 </dev/null &
fi
} }
backup_weekly() { backup_weekly() {
mail_log $INFO ">>>>>>>>>> weekly auto backup $VER for $HOST_NAME <<<<<<<<<<" mail_log $INFO ">>>>>>>>>> weekly auto backup $VER for $HOST_NAME <<<<<<<<<<"
@@ -437,6 +600,7 @@ backup_weekly() {
is_error $? is_error $?
mail_log $INFO ">>>>>>>>>> end of auto backup" mail_log $INFO ">>>>>>>>>> end of auto backup"
fix_acls
send_mail_log "Weekly Backup" send_mail_log "Weekly Backup"
} }
@@ -462,21 +626,50 @@ test_integrity() {
# Restore aus Archiv # Restore aus Archiv
# #
# Wird nach pwd ausgegeben! # Wird nach $RESTORE_PATH ausgegeben und mit root-Rechten
# auf der NAS ausgeführt.
# #
# $1 - Archiv, ggf. mit Pfad # $1 - Archiv, mit Pfad ab $NAS_ROOT
# $2 ff. - zurückzusichernde Dateien/Verzeichnisse # $2 ff. - zurückzusichernde Dateien/Verzeichnisse
# #
# returns 0 for success, 1 otherwise
#
restore() { restore() {
log $DEBUG "$DBG restore( $@ )" log $DEBUG "$DBG restore( $@ )"
local archive=$1 local archive=$1
shift 1 shift
log $DEBUG "$DBG > gpg --batch --decrypt --passphrase xxxx \"$archive\" | tar -xzv $@" echo "--- Restore ---"
gpg --batch --decrypt --passphrase $PASSPHRASE "$archive" 2>>$LOG_FILE | tar -xzv $@ >/dev/null 2>>$LOG_FILE echo "--- archive: $archive"
echo "--- files: $@"
echo "--- to: $RESTORE_PATH"
echo
echo "Restore started on NAS, see e-mail"
echo
run_script_remote $NAS_IP "$NAS_SCRIPT --rx $archive $@"
}
# Restore
#
# Beim Restore einzelner Dateien wird nur die ownership der Datei
# restored, nicht die des Pfades zur Datei!
#
do_restore() {
if [ "$HOST_NAME" != "$NAS_HOSTNAME" ]; then
restore $@
exit
fi
local archive=$1
shift
mail_log $INFO ">>>>>>>>>> restore $VER for $HOST_NAME <<<<<<<<<<"
mail_log $INFO ">>> restore from: $archive"
mail_log $INFO ">>> files : $@"
mail_log $INFO ">>> restore to : $RESTORE_PATH"
mail_log $INFO ">>> log file : $LOG_FILE"
log $DEBUG "$DBG > gpg --batch --decrypt --passphrase xxxx \"$NAS_ROOT/${archive/#\//}\" | tar -xzv --directory $RESTORE_PATH $@"
gpg --batch --decrypt --passphrase $PASSPHRASE "$NAS_ROOT/${archive/#\//}" 2>>$LOG_FILE | tar -xzv --directory $RESTORE_PATH $@ >>$LOG_FILE 2>&1
check_pipe_rc ${PIPESTATUS[@]} check_pipe_rc ${PIPESTATUS[@]}
return $? is_error $?
mail_log $INFO ">>>>>>>>>> end of restore"
send_mail_log "Restore"
} }
# ----- archive content ------ # ----- archive content ------
@@ -498,41 +691,37 @@ create_list() {
# Liste der Archives auf stdout ausgeben # Liste der Archives auf stdout ausgeben
# #
# Zeigt alle Archive incl. Verzeichnis (ohne $1) an # Zeigt alle Archive incl. Verzeichnis (ohne $NAS_ROOT) an
# #
# z.B. # z.B.
# $1 = /volume1/nfs/backups # $NAS_ROOT = /volume1/nfs/backups
# ... # ...
# /daily/synology/backup.3/volume1-nfs-Server.tgz.gpg # /daily/synology/backup.3/volume1-nfs-Server.tgz.gpg
# /weekly/synology/volume1-media-books.tgz.gpg # /weekly/synology/volume1-media-books.tgz.gpg
# #
# $1 - Verzeichnis unter dem alle Backups liegen
#
# returns 0 for success, 1 otherwise # returns 0 for success, 1 otherwise
# #
list_archives() { list_archives() {
log $DEBUG "$DBG list_archives( $@ )" log $DEBUG "$DBG list_archives()"
local target=$1 local target=$1
local targ=${target//\//\\\/} # slashes escapen local targ=${NAS_ROOT//\//\\\/} # slashes escapen
log $DEBUG "$DBG > find \"$target\" -type f -name \"*.$EXT\" | grep \"${HOST_NAME}/\" | sed -E \"s/^$targ//\"" log $DEBUG "$DBG > find \"$NAS_ROOT\" -type f -name \"*.$EXT\" | sed -E \"s/^$targ//\""
find "$target" -type f -name "*.$EXT" 2>>$LOG_FILE | grep "${HOST_NAME}/" 2>>$LOG_FILE | sed -E "s/^$targ//" 2>>$LOG_FILE find "$NAS_ROOT" -type f -name "*.$EXT" 2>>$LOG_FILE | sed -E "s/^$targ//" 2>>$LOG_FILE | sort
check_pipe_rc ${PIPESTATUS[@]} check_pipe_rc ${PIPESTATUS[@]}
return $? return $?
} }
# Inhalt des Archivs $2 auf stdout ausgeben # Inhalt des Archivs $1 auf stdout ausgeben
# #
# $1 - Verzeichnis unter dem alle Backups liegen # $1 - Archiv-Datei mit Pfad ab $NAS_ROOT
# $2 - Archiv-Datei mit Pfad ab $1
# #
# returns 0 for success, 1 otherwise # returns 0 for success, 1 otherwise
# #
list_archive_content() { list_archive_content() {
log $DEBUG "$DBG list_archive_content( $@ )" log $DEBUG "$DBG list_archive_content( $@ )"
local root=$1 local archive=$1
local archive=$2 log $DEBUG "$DBG > cat \"$NAS_ROOT/${archive/#\//}.$LIST_EXT\""
log $DEBUG "$DBG > cat \"$root/${archive/#\//}.$LIST_EXT\"" cat "$NAS_ROOT/${archive/#\//}.$LIST_EXT" 2>>$LOG_FILE
cat "$root/${archive/#\//}.$LIST_EXT" 2>>$LOG_FILE
} }
# ----- export ----- # ----- export -----
@@ -609,8 +798,8 @@ mount_usb_device() {
fi fi
} }
# Check if auto mounted device is exfat and return mount point # Check if auto mounted device is exfat or vfat and return mount point
# (fn for consitence with linux fn) # (fn for consistence with linux fn)
# #
# $1 - device name # $1 - device name
# returns mount point # returns mount point
@@ -620,7 +809,7 @@ mount_usb_device_syno() {
local device=$1 local device=$1
log $DEBUG "$DBG > $SYNO_USB -info $device | grep -i \"file system\" | sed -E \"s/.*:\s*(.*)\s*/\1/\"" log $DEBUG "$DBG > $SYNO_USB -info $device | grep -i \"file system\" | sed -E \"s/.*:\s*(.*)\s*/\1/\""
local fs=$($SYNO_USB -info $device 2>>$LOG_FILE | grep -i "file system" 2>>$LOG_FILE | sed -E "s/.*:\s*(.*)\s*/\1/" 2>>$LOG_FILE) local fs=$($SYNO_USB -info $device 2>>$LOG_FILE | grep -i "file system" 2>>$LOG_FILE | sed -E "s/.*:\s*(.*)\s*/\1/" 2>>$LOG_FILE)
if [ "$fs" != "exfat" ]; then if [ "$fs" != "exfat" && "$fs" != "vfat" ]; then
mail_log $ERROR "$ERR device $device not exfat formatted." mail_log $ERROR "$ERR device $device not exfat formatted."
return 1 return 1
fi fi
@@ -631,6 +820,7 @@ mount_usb_device_syno() {
mail_log $ERROR "$ERR mount point \"$mount_point\" does not exist." mail_log $ERROR "$ERR mount point \"$mount_point\" does not exist."
return 1 return 1
fi fi
mount_point=$(check_syno_mount_point $mount_point)
echo $mount_point echo $mount_point
return 0 return 0
} }
@@ -695,6 +885,28 @@ umount_usb_device_linux() {
return $? return $?
} }
# Mit GUID-Partitontable meldet Synology die falsche Partition (/volumeUSB1/usbshare1-1)
# zusätzlich auf ggf. vorhandener 2. Partition (*1-2) suchen
#
# $1 - mount point
#
check_syno_mount_point() {
log $DEBUG "$DBG check_syno_mount_point( $@ )"
local mount_point=$1
log $DEBUG "$DBG > ls $mount_point/${tagP1}*${tagP2} | wc -l"
if [ "$(ls $mount_point/${tagP1}*${tagP2} 2>>$LOG_FILE | wc -l 2>>$LOG_FILE)" != "1" ]; then
log $DEBUG "No tag at mount point \"$mount_point\". Checking next partiton."
mount_point=$(echo $mount_point | sed -E "s/(.*1)-1/\1-2/")
if [ "$(ls $mount_point/${tagP1}*${tagP2} 2>>$LOG_FILE | wc -l 2>>$LOG_FILE)" != "1" ]; then
mail_log $ERROR "$ERR no tag found on device at $mount_point."
return 1
fi
fi
echo $mount_point
}
# USB-HDD suchen und Backup-Typ ermitteln # USB-HDD suchen und Backup-Typ ermitteln
# #
# Der Typ wird über eine Datei auf der USB-HDD festgelegt: # Der Typ wird über eine Datei auf der USB-HDD festgelegt:
@@ -705,19 +917,11 @@ umount_usb_device_linux() {
check_backup_device() { check_backup_device() {
log $DEBUG "$DBG check_backup_drive( $@ )" log $DEBUG "$DBG check_backup_drive( $@ )"
local mount_point=$1 local mount_point=$1
local tagP1=".backup#"
local tagP2=".tag"
log $DEBUG "$DBG > ls $mount_point/${tagP1}*${tagP2} | wc -l"
if [ "$(ls $mount_point/${tagP1}*${tagP2} 2>>$LOG_FILE | wc -l 2>>$LOG_FILE)" != "1" ]; then
mail_log $ERROR "$ERR no tag found on device at $mount_point."
return 1
fi
log $DEBUG "$DBG > ls $mount_point/${tagP1}*${tagP2} | sed -E \"s/.*$tagP1(.*)$tagP2/\1/\"" log $DEBUG "$DBG > ls $mount_point/${tagP1}*${tagP2} | sed -E \"s/.*$tagP1(.*)$tagP2/\1/\""
local tag=$(ls $mount_point/${tagP1}*${tagP2} 2>>$LOG_FILE | sed -E "s/.*$tagP1(.*)$tagP2/\1/" 2>>$LOG_FILE) local tag=$(ls $mount_point/${tagP1}*${tagP2} 2>>$LOG_FILE | sed -E "s/.*$tagP1(.*)$tagP2/\1/" 2>>$LOG_FILE)
if [ $? -ne 0 ]; then if [ $? -ne 0 ]; then
mail_log $ERROR "$ERR no tag found on device at $mount_point." mail_log $ERROR "$ERR no valid tag found on device at $mount_point."
return 1 return 1
fi fi
echo $tag echo $tag
@@ -791,8 +995,8 @@ do_export() {
mail_log $INFO ">>> " mail_log $INFO ">>> "
mail_log $INFO "$INF exporting $source_path -> $mount_point" mail_log $INFO "$INF exporting $source_path -> $mount_point"
log $DEBUG "$DBG rsync -rti --one-file-system --delete --stats --exclude=@eaDir $source_path $mount_point" log $DEBUG "$DBG rsync -rti --one-file-system --delete --stats --exclude=@eaDir --exclude='*'.$LIST_EXT $source_path $mount_point"
rsync -rti --one-file-system --delete --stats --exclude=@eaDir $source_path $mount_point >>$LOG_FILE 2>&1 rsync -rti --one-file-system --delete --stats --exclude=@eaDir --exclude='*'.$LIST_EXT $source_path $mount_point >>$LOG_FILE 2>&1
is_error $? is_error $?
mail_log $INFO "$INF unmounting USB device $device." mail_log $INFO "$INF unmounting USB device $device."
+81 -39
View File
@@ -3,37 +3,59 @@
# entrypoint for all actions # entrypoint for all actions
# #
VER="2.0.4" # 2.3.3 18.12.2025 - smtp-relay läuft durch, dadurch ändern sich ggf. Dateien im Container, was
# zu Fehlern beim tar führt. Deshalb wird das Verzeichnis
# /var/home/container/.local/share/containers vom Backup ausgenommen.
# Die Container-Overlays, die hier liegen können aus dem jeweiligen Image
# wiederhergestellt werden, es geht also nichts verloren.
# Außerdem kann tar über Sockets stolpern und unnötige Warnungen produzieren.
# Abhilfe schafft der Parameter "--warning=no-file-ignored".
# Den mail delay im synology backup wieder entfernt.
# 2.3.2 18.09.2025 - synology backup: wait 60 min before mailing
# 2.3.1 16.08.2025 - 30 min delay for mail.sh
# 2.3.0 12.08.2025 - new function for backup & encryption of Proton exports
# 2.2.4 30.07.2025 - inc delay (again) for container startup-check (2m)
# 2.2.3 24.06.2025 - Export-Disk kann exfat- oder vfat-formatiert sein
# Export-Disk mit mehreren Partitonen erkennen
# 2.2.2 19.06.2025 - inc delay for container startup check, fix error in log output
# 2.2.1 16.06.2025 - delay container startup check
# 2.2.0 16.05.2025 - fixes for auto-update, podman migration done
# 2.1.17 11.05.2025 - bugfixes, sort tar archive, split daily/cron daily
# 2.1.16 10.05.2025 - fix mail.sh, log results of auto-update
# 2.1.15 09.05.2025 - podman auto-update
# 2.1.14 09.05.2025 - Fix mail.sh, homeserver.target, cleared all docker references
# 2.1.13 08.05.2025 - check if services are running after backup
# 2.1.12 08.05.2025 - mail.sh an neuen Container angepasst
# 2.1.11 05.05.2025 - Fix DB-Dump
# 2.1.10 05.05.2025 - Postgres-Dump auf podman umgestellt
# 2.1.9 02.05.2025 - DB-Dump erweitert, damit Dumps für mehrere postgres möglich sind
# 2.1.8 30.04.2025 - systemctl start/stop eingebaut
# 2.1.7 29.04.2025 - 1. Schritt zur Umstellung auf podman
# 2.1.6 29.04.2025 - weekly backup Sa->SO
show_help() { VER="2.3.3"
echo "usage: $0 options"
echo " -d, --daily start daily backup on local machine"
echo " -w, --weekly start weekly backup on local machine"
echo " --list list archives"
echo ' --content <archive> '
echo " show contents of archive"
echo ' -r, --restore <archive> <file(s)> '
echo " restore files from archive to pwd"
echo " -h, -?. --help show this help"
echo " "
echo " Script muss auf dem jeweiligen Server ausgeführt werden (ggf. via ssh)."
echo " "
}
#
# Script muss unter root bzw. sudo laufen, sonst gibt es jede Menge Probleme
# mit Zugriff auf die diversen Verzeichnisse.
#
if [ "$(id -u)" -ne 0 ]; then
echo "**"
echo "** ERROR: script has to be run as superuser!"
echo "**"
exit 100
fi
HERE=$(dirname "$0") HERE=$(dirname "$0")
THIS_SCRIPT=$0 THIS_SCRIPT=$0
show_help() {
echo "Server Backup $VER"
echo "usage: $0 options"
echo " -d, --daily start daily backup on local machine"
echo " -w, --weekly start weekly backup on local machine"
echo " -e, --export export backups to USB-HDD"
echo " -l, --list list archives"
echo ' -c, --content <archive> '
echo " show contents of archive (path from $NAS_ROOT)"
echo ' -r, --restore <archive> <file(s)> '
echo " restore files from archive to $RESTORE_PATH"
echo " -p, --proton backup and encrypt Proton exports (MacBook only)"
echo " -h, -?. --help show this help"
echo " "
echo " -d, -w, -e must be run as superuser."
echo " "
}
if [ -f $HERE/main.inc.sh ]; then if [ -f $HERE/main.inc.sh ]; then
. $HERE/main.inc.sh . $HERE/main.inc.sh
else else
@@ -56,9 +78,6 @@ else
exit 102 exit 102
fi fi
LOG_FILE_PATH="$(realpath $HERE/../../../log/backup)"
log_file_name="$LOG_FILE_PATH/$HOST_NAME-$(date '+%Y-%m-%d_%H%M')"
LOG_FILE="${log_file_name}.log"
file= file=
archive= archive=
@@ -69,46 +88,69 @@ while :; do
show_help # Display a usage synopsis. show_help # Display a usage synopsis.
exit 0 exit 0
;; ;;
--dx)
MAIL_LOG_FILE="${log_file_name}.auto.log"
check_root
cron_backup_daily
break
;;
-d|--daily) -d|--daily)
MAIL_LOG_FILE="${log_file_name}.auto.log" MAIL_LOG_FILE="${log_file_name}.auto.log"
check_root
backup_daily backup_daily
break break
;; ;;
-w|--weekly) -w|--weekly)
MAIL_LOG_FILE="${log_file_name}.auto.log" MAIL_LOG_FILE="${log_file_name}.auto.log"
check_root
backup_weekly backup_weekly
break break
;; ;;
-e|--export) -e|--export)
MAIL_LOG_FILE="${log_file_name}.auto.log" MAIL_LOG_FILE="${log_file_name}.auto.log"
check_root
export_backups export_backups
break break
;; ;;
--list) -l|--list)
list_archives "$NAS_ROOT/daily/$HOST_NAME" list_archives
if [ -d "$NAS_ROOT/weekly/$HOST_NAME" ]; then
list_archives "$NAS_ROOT/weekly/$HOST_NAME"
fi
break; break;
;; ;;
--content) -c|--content)
if [ "$2" ]; then if [ "$2" ]; then
list_archive_content "$NAS_ROOT" "$2" list_archive_content "$2"
else else
echo 'ERROR: "--content" requires a non-empty option argument.' echo 'ERROR: "--content" requires the archive name as an argument.'
echo
show_help
fi fi
break break
;; ;;
-r|--restore) -r|--restore)
if [ "$2" ]; then if [ "$2" ]; then
archive=$2 archive=$2
shift shift 2
restore "$archive" "$@" restore "$archive" "$@"
else else
echo 'ERROR: "--restore" requires a non-empty option argument.' echo 'ERROR: "--restore" requires at least the archive name as an argument.'
echo
show_help
fi fi
break break
;; ;;
--rx)
MAIL_LOG_FILE="${log_file_name}.auto.log"
check_root
shift
do_restore $@
break
;;
-p|--proton)
MAIL_LOG_FILE="${log_file_name}.proton.log"
check_mac
proton_backup
break
;;
--) # End of all options. --) # End of all options.
shift shift
break break
+27 -19
View File
@@ -1,45 +1,53 @@
#!/usr/bin/env bash #!/usr/bin/env bash
NAS_ROOT=/var/srv/nas/backups USER_CONTAINERS="homeserver.target"
DOCKER_ROOT=/var/docker ROOT_CONTAINERS=
BACKUP_TARGET=$DOCKER_ROOT/tmp
#DOCKER_ROOT=/var/docker
CONTAINERS_ROOT=/var/containers
BACKUP_TARGET=$CONTAINERS_ROOT/tmp
NAS_TARGET="$NAS_ROOT/daily/$HOST_NAME/backup.$DAY" NAS_TARGET="$NAS_ROOT/daily/$HOST_NAME/backup.$DAY"
backup_daily1() { backup_daily1() {
BACKUP_SOURCE="/var" BACKUP_SOURCE="/var"
BACKUP_EXCLUDE_DIRS=(docker) BACKUP_EXCLUDE_DIRS=(containers home/container/.local/share/containers)
} }
backup_daily2() { backup_daily2() {
BACKUP_SOURCE="/var/docker" BACKUP_SOURCE="/var/containers"
BACKUP_EXCLUDE_DIRS=(vol tmp) BACKUP_EXCLUDE_DIRS=(vol tmp)
} }
backup_daily3() { backup_daily3() {
BACKUP_SOURCE="/var/docker/vol" BACKUP_SOURCE="/var/containers/vol"
BACKUP_EXCLUDE_DIRS=() BACKUP_EXCLUDE_DIRS=()
} }
before_backup() { before_backup() {
dump_mariadb $DOCKER_ROOT/dump/mysql let local rc=0
local mys=$? #dump_mariadb $DOCKER_ROOT/dump/mysql
dump_postgresql $DOCKER_ROOT/dump/postgresql #let rc=rc+$?
local pos=$? dump_postgresql ncd_pg $CONTAINERS_ROOT/dump/nextcloud
if [ $mys -gt 0 ] || [ $pos -gt 0 ]; then let rc=rc+$?
dump_postgresql gitea_pg $CONTAINERS_ROOT/dump/gitea
let rc=rc+$?
if [ $rc -gt 0 ]; then
return 1 return 1
fi fi
stop_docker stop_containers
return $? return $?
} }
after_backup() { after_backup() {
start_docker let local rc=0
local dock=$? start_containers
let rc=rc+$?
copy_to_nas "$BACKUP_TARGET" "$NAS_TARGET" copy_to_nas "$BACKUP_TARGET" "$NAS_TARGET"
local copy=$? let rc=rc+$?
if [ $dock -eq 0 ] && [ $copy -eq 0 ]; then update_services
return 0 let rc=rc+$?
fi check_services
return 1 let rc=rc+$?
return $rc
} }
Executable
+12
View File
@@ -0,0 +1,12 @@
#!/usr/bin/env bash
#
# Archive docker config
#
# copy this file to docker/run
SOURCE=/var/docker/run/docker-compose.yml
SAVE_DIR=/srv/nas/Server/CoreOS/docker-run
SAVE_FILE=$(hostname -s)-docker-compose-$(date '+%Y-%m-%d_%H%M').yml
cp $SOURCE $SAVE_DIR/$SAVE_FILE
+2 -2
View File
@@ -1,6 +1,5 @@
#!/usr/bin/eval bash #!/usr/bin/eval bash
NAS_ROOT=/volume1/nfs/backups
HAS_WEEKLY=true HAS_WEEKLY=true
backup_daily1() { backup_daily1() {
@@ -26,6 +25,7 @@ before_backup() {
} }
after_backup() { after_backup() {
# nothing to do # wait for mail server backup
## sollte sich erledigt haben ## sleep 60m
echo echo
} }
-92
View File
@@ -1,92 +0,0 @@
#!/usr/bin/env bash
if [ "$(id -u)" -ne 0 ]; then
echo "**"
echo "** ERROR: script has to be run as superuser!"
echo "**"
exit 100
fi
HERE=$(dirname "$0")
if [ -f $HERE/main.inc.sh ]; then
. $HERE/main.inc.sh
else
echo "**"
echo "** ERROR: Datei \"$HERE/main.inc.sh\" nicht gefunden. "
echo "** Script wird abgebrochen"
echo "**"
show_help
exit 101
fi
if [ -f $HERE/$HOST_NAME.sh ]; then
. $HERE/$HOST_NAME.sh
else
echo "**"
echo "** ERROR: Datei \"$HOST_NAME.sh\" nicht gefunden (falscher Rechner?). "
echo "** Script wird abgebrochen."
echo "**"
show_help
exit 102
fi
log_file_name="$NAS_ROOT/log/$HOST_NAME-$(date '+%Y-%m-%d_%H%M')"
LOG_FILE="${log_file_name}.log"
# nicht auf calvados
#. $HERE/$HOST_NAME
test() {
for i in "$@"; do echo "$i"; done
}
isFunction() { declare -Ff "$1" ; }
if [ "$HOST_NAME" != "$NAS_HOSTNAME" ]; then
run_script_remote $NAS_IP "/volume1/nfs/Server/bin/backup/test.sh -x"
#ssh 10.4.2.20 "sudo sh -c 'nohup /volume1/nfs/Server/bin/backup/main.sh -e &>/dev/null < /dev/null &'"
exit
else
echo "klappt $1" > /volume1/nfs/log/backup/remote_run.log
fi
exit
list1() {
EXCL=("eaDir" "Temporary Items" "lost+found" backups)
ONE="erster #1"
TWO="zweiter #1"
#for i in "${a[@]}"; do echo "$i"; done
}
list2() {
EXCL=("@eaDir" "Temporary Items" "lost+found")
ONE="eins2"
TWO="zwei2"
return 0
#for i in "${a[@]}"; do echo "$i"; done
}
fun0() {
return 0
}
if [ "$(id -u)" -ne 0 ]; then
echo "**"
echo "** ERROR: script has to be run as superuser!"
echo "**"
exit 100
fi
fun0
rc=$?
if [ $rc -eq 0 ]; then
echo "$rc -eq 0"
fi
#echo "result=$rc0"
exit