[devel] Re: sis mirroring script

[Michael Shigorin]

On Wed, Apr 28, 2004 at 04:37:37PM +0300, Nick S. Grechukh wrote:
> вот похвастаюсь :-) исходники: две кружки кофе и 20 минут. 

Ууу.... rsync(1) видели? ;-)

-- 
 ---- WBR, Michael Shigorin <mike на altlinux.ru>
  ------ Linux.Kiev http://www.linux.kiev.ua/
----------- следующая часть -----------
#!/bin/sh
# script by Michael Shigorin <mike на osdn.org.ua> for snapshotting
# ALT Linux Sisyphus locally; feel free to use and modify, would
# be glad to hear back about changes

# mirror/site-specific
SRC="ftp.altlinux.org.ua::ALTLinux/Sisyphus/"
BASEDIR="/var/ftp/pub/EMT/Sisyphus/"

# retry params
MAXCOUNT=5
TIMEOUT=60

# shouldn't need to be changed
DATE=`date +%Y%m%d`
LOGDIR="$BASEDIR/log/"
LOGFILE="$LOGDIR/$DATE.log"
DEST="$BASEDIR/.syncing/"

RSYNCARGS="-av --partial --delete"

RPMARGS="-qp --queryformat %{RELEASE}\n"
TAGFILE="$DEST/files/i586/RPMS/altlinux-release-Sisyphus-alt*.noarch.rpm"

unset OK

# init env
mkdir -p "$BASEDIR" "$LOGDIR"
cd "$BASEDIR"

# create hardlinked copy or init dir; order critical
LAST=`ls -d1 20?????? 2>/dev/null | tail -1`
rm -rf "$DEST"
[ -n "$LAST" ] && cp -al "$LAST" "$DEST" || mkdir -p "$DEST"

# try to sync up
for attempt in `seq 1 $MAXCOUNT`; do 
	rsync $RSYNCARGS "$SRC" "$DEST" 2>&1 >> "$LOGFILE" && {
		OK="yes"
		break
	}
	sleep "$TIMEOUT"
done

[ -z "$OK" ] && exit 1

# get timestamp from specific package and move to it
TIMESTAMP=`rpm $RPMARGS $TAGFILE | tail -1 | sed 's/alt//'`

[ ! -d "$TIMESTAMP" ] && {
	mv "$DEST" "$TIMESTAMP"
	touch -d "$DATE" "$TIMESTAMP"
	# fix symlink
	rm -f current
	ln -s "$TIMESTAMP" current
} || rm -rf "$DEST"

# that's all :)
echo "** $TIMESTAMP sync OK" >> "$LOGFILE"
----------- следующая часть -----------
#!/bin/sh
RSYNC_RSH=rsh
MAXATTEMPTS=5
TMOUT1=500
TMOUT2=15
SRCROOT=rsync://rsync.altlinux.ru/ALTLinux	# 2003-04-15
#SRCROOT=rsync://ibiblio.org/Linux/distributions/altlinux	# 2002-12-07
#SRCROOT=rsync://ftp.leo.org/ALTLinux	# 2002-12-09
DESTROOT=~ftp/pub/Linux/ALT
#LIST="updates Sisyphus Mozilla OpenOffice Daedalus"
LIST="updates Sisyphus Mozilla Daedalus"
#LIST="updates"
RSYNC=/usr/bin/rsync
RSYNCARGS="-rltgoDuz --partial --timeout=$TMOUT1 --delete --delete-after"
#RSYNCARGS="-rltgoDuz --partial --timeout=$TMOUT1 --delete "
RSHOME=~/rsync
MAINTAINER=mike на osdn.org.ua

for mirror in $LIST; do
	OK=
	attempt=1
	LOG=$RSHOME/rsync-$mirror.log
	LOCK=$RSHOME/lock.$mirror
	FLAG=$DESTROOT/$mirror/__SYNCING__
	date >> $LOG
	[ -f $LOCK ] && {
		ps auxww | grep rsync | mail -s "OSDN: $mirror: lockfile exists" $MAINTAINER
		echo "Lockfile for $mirror exists, quitting" >> $LOG
		continue
	}
	trap "rm -f $LOCK; exit 1" INT
	touch $LOCK $FLAG
	while [ -z "$OK" -a $attempt -lt $MAXATTEMPTS ]; do 
		$RSYNC $RSYNCARGS $* $SRCROOT/$mirror/ $DESTROOT/$mirror \
			>> $LOG 2>&1 && OK=1
		sleep $TMOUT2
		attempt=$(($attempt+1))
	done
	[ -z "$OK" ] && {
		tail $LOG | mail -s "OSDN: $mirror trouble" $MAINTAINER
	} || {
		rm -f $FLAG
	}
	rm -f $LOCK
	date >> $LOG
done

nice symlinks -dr $DESTROOT >/dev/null
----------- следующая часть -----------
Было удалено вложение не в текстовом формате...
Имя     : =?iso-8859-1?q?=CF=D4=D3=D5=D4=D3=D4=D7=D5=C5=D4?=
Тип     : application/pgp-signature
Размер  : 189 байтов
Описание: =?iso-8859-1?q?=CF=D4=D3=D5=D4=D3=D4=D7=D5=C5=D4?=
Url     : <http://lists.altlinux.org/pipermail/devel/attachments/20040428/97944014/attachment-0001.bin>