[make-initrd] [PATCH v6 12/22] bootchain-core: introduces extended debugging
Leonid Krivoshein
klark.devel at gmail.com
Tue Oct 26 22:17:36 MSK 2021
26.10.2021 14:55, Alexey Gladkov пишет:
> On Sun, Oct 24, 2021 at 08:21:50PM +0300, Leonid Krivoshein wrote:
>> Extends the bootchain-sh-functions API, adds the "bc_debug" parameter,
>> which will enable advanced debugging mode, adds a service step-script
>> "debug", which is executed before starting other steps-scripts in the
>> advanced debugging mode, which allows to monitor the data coming to
>> the <IN> of the next step. Also enables more verbose logging when the
>> daemon is running and ensures that the boot log is saved to the stage2.
>>
>> Signed-off-by: Leonid Krivoshein <klark.devel at gmail.com>
>> ---
>> features/bootchain-core/README.md | 31 ++++++-
>> .../data/bin/bootchain-sh-functions | 78 +++++++++++++++++-
>> .../data/etc/initrd/cmdline.d/bootchain-core | 1 +
>> .../bootchain-core/data/lib/bootchain/debug | 81 +++++++++++++++++++
>> features/bootchain-core/data/sbin/chaind | 73 ++++++++++++-----
>> 5 files changed, 242 insertions(+), 22 deletions(-)
>> create mode 100755 features/bootchain-core/data/lib/bootchain/debug
>>
>> diff --git a/features/bootchain-core/README.md b/features/bootchain-core/README.md
>> index 83897da..cea2d84 100644
>> --- a/features/bootchain-core/README.md
>> +++ b/features/bootchain-core/README.md
>> @@ -49,6 +49,8 @@ us to optimize fill in `initramfs` only which we are need.
>>
>> - Modularity: loading methods are initially separated from the common
>> code and daemon.
>> +- `bootchain-sh-functions` extends the API of the original `pipeline-sh-functions`,
>> + see the details in the corresponding section.
>> - Via resolve_target() supports not only forward, but also reverse addressing,
>> relative to the current step. For example, a record like `step-3/dir1/dev`
>> will process the result of `dir1/dev`, made in the third step from the current
>> @@ -60,6 +62,11 @@ us to optimize fill in `initramfs` only which we are need.
>> - The daemon can be configured when building initramfs via the included file
>> configurations of `/etc/sysconfig/bootchain`, and not only through boot
>> parameters, see the details in the corresponding section.
>> +- The `chaind` daemon offers visual and advanced debugging. By default, the log
>> + is kept in `/var/log/chaind.log`.
>> + Service step-script `debug` in advanced debugging mode is run before launching
>> + any other step-script and allows you to visually track the received values at
>> + each <IN>.
>>
>> Despite the differences, `chaind` is backward compatible with previously
>> written steps for the `pipelined` daemon and does not require changes for
>> @@ -71,6 +78,10 @@ The configuration is defined in the file `/etc/sysconfig/bootchain` when
>> building the initramfs image, is optional, and may contain the following
>> parameters:
>>
>> +- `BC_LOGFILE` - the full path to the log file or the name of a special device,
>> + to which debugging messages will be output. In NATIVE mode, the default value
>> + is `/var/log/chaind.log`, in COMPATIBILITY mode with `pipeline` the default
>> + value is `/var/log/pipelined.log`.
>> - `mntdir` - where to create subdirectories of the bootchain steps. In NATIVE
>> mode the default value is `/dev/bootchain`, in COMPATIBILITY mode with
>> `pipeline`, the default value is `/dev/pipeline`.
>> @@ -95,8 +106,10 @@ own needs.
>> - `pipeline=name1[,name2][,name3]` - alias for `bootchain=...`.
>> - `mountfs=target` - specifies the file or device to be mounted.
>> - `overlayfs=list` - defines the list of elements to combine.
>> +- `bc_debug` - a boolean parameter that enables advanced debugging and forces
>> + if the daemon completes successfully, copy boot log to the stage2.
>>
>> -## bootchain-sh-functions API
>> +## bootchain-sh-functions extended API
>>
>> - check_parameter() - checks that the required parameter is not empty, otherwise
>> it exits via fatal().
>> @@ -104,7 +117,21 @@ own needs.
>> the index $callnum.
>> - resolve_target() - output the path to a file, directory or device, depending
>> on from the parameter.
>> -- run() - run an external command.
>> +- resolve_devname() - output the path to a special device file at the specified
>> + directory. Usually the step directory contains a DEVNAME or dev file if the
>> + device was the result of a step, then the function will return a readable
>> + `/dev/node`.
>> +- debug() - text message output during extended debugging.
>> +- enter() - tracing during extended debugging: entering the specified function.
>> +- leave() - tracing during extended debugging: exit from the specified function.
>> +- run() - run an external command. With extended debugging, the executed command
>> + will be logged.
>> +- fdump() - output of the contents of the specified file during extended
>> + debugging.
>> +- assign() - assignment of the specified value to a variable that gets into
>> + the log with advanced debugging. The left-hand expression is also computable.
>> +- initrd_version() - output of the current version of make-initrd. It is proposed
>> + to move to make-initrd/data/bin/initrd-sh-functions after has_module().
>>
>> ## Examples
>>
>> diff --git a/features/bootchain-core/data/bin/bootchain-sh-functions b/features/bootchain-core/data/bin/bootchain-sh-functions
>> index 8167e29..dcc1ab2 100644
>> --- a/features/bootchain-core/data/bin/bootchain-sh-functions
>> +++ b/features/bootchain-core/data/bin/bootchain-sh-functions
>> @@ -3,15 +3,22 @@
>> if [ -z "${__bootchain_sh_functions-}" ]; then
>> __bootchain_sh_functions=1
>>
>> +BC_DEBUG=
>> +
>> [ ! -s /etc/sysconfig/bootchain ] ||
>> . /etc/sysconfig/bootchain
>>
>> +. initrd-sh-functions
>> . /.initrd/initenv
>> . shell-signal
>>
>> +message_time=1
>> +
>> if [ "${ROOT-}" = pipeline ]; then
>> + BC_LOGFILE="${BC_LOGFILE:-/var/log/pipelined.log}"
>> mntdir="${mntdir:-/dev/pipeline}"
>> else
>> + BC_LOGFILE="${BC_LOGFILE:-/var/log/chaind.log}"
>> mntdir="${mntdir:-/dev/bootchain}"
>> fi
>>
>> @@ -81,10 +88,77 @@ resolve_target()
>> printf '%s' "$target"
>> }
>>
>> +resolve_devname()
>> +{
>> + local dir="${1-}"
>> + local devname=
>> +
>> + [ -n "$dir" ] ||
>> + dir="${prevdir-}"
>> + [ -d "$dir" ] ||
>> + return 0
>> +
>> + if [ -s "$dir"/DEVNAME ]; then
>> + read -r devname <"$dir"/DEVNAME ||:
>> + elif [ -b "$dir"/dev ] || [ -c "$dir"/dev ]; then
>> + devname="$(get_majmin "$dir"/dev ||:)"
>> + devname="/sys/dev/block/$devname/uevent"
>> + [ -b "$dir"/dev ] ||
>> + devname="/sys/dev/char/$devname/uevent"
>> + devname="/dev/$(sed -E -n 's/^DEVNAME=(.*)/\1/p' "$devname" ||:)"
>> + fi
>> +
>> + [ ! -b "$devname" ] && [ ! -c "$devname" ] ||
>> + printf '%s' "$devname"
> Если мы ничего не нашли или devname не устройство, то функция молча ничего
> не выводит. Даже код возврата не покажет, что у нас явная ошибка.
>
>> +}
>> +
>> +debug()
>> +{
>> + [ -z "$BC_DEBUG" ] ||
>> + message "[$callnum] $*"
>> +}
>> +
>> +enter()
>> +{
>> + debug "ENTER: $*"
>> +}
>> +
>> +leave()
>> +{
>> + debug "LEAVE: $*"
>> +}
>> +
>> run()
>> {
>> - [ -z "${DEBUG-}" ] || message "[$callnum] RUN: $*"
>> - "$@"
>> + debug "RUN: $*"
>> + "$@" || return $?
>> +}
>> +
>> +fdump()
>> +{
>> + [ -n "$BC_DEBUG" ] && [ -f "$1" ] ||
>> + return 0
>> + { printf '============================================================\n'
>> + cat -- "$1" ||:
>> + printf '============================================================\n\n'
>> + } 1>&2
>> +}
>> +
>> +assign()
>> +{
>> + local _v="" _k="$1" _e="${2-}"
>> +
>> + eval "_v=\"${_e}\""
> eval "_v=\"\${_e}\""
> ^
>
> Эту переменную нужно экранировать, чтобы в _v попало содержимое _e а не
> eval содержимого. Иначе тут можно выполнить вообще произвольный код.
>
>> + eval "${_k}=\"${_e}\""
> тут тоже самое.
Отлично, это как раз то, на что я очень рассчитывал. Сделать код более
безопасным и более правильным с точки зрения проекта make-initrd. Поправлю.
>
>> + debug "LET: ${_k}=\"${_v}\""
>> +}
>> +
>> +initrd_version()
>> +{
>> + [ ! -s /etc/initrd-release ] ||
>> + . /etc/initrd-release
>> + local __version="${VERSION_ID-}"
>> + printf '%s' "INITRAMFS${__version:+ $__version}"
>> }
>>
>> fi # __bootchain_sh_functions
>> diff --git a/features/bootchain-core/data/etc/initrd/cmdline.d/bootchain-core b/features/bootchain-core/data/etc/initrd/cmdline.d/bootchain-core
>> index 1b1198c..4151a8d 100644
>> --- a/features/bootchain-core/data/etc/initrd/cmdline.d/bootchain-core
>> +++ b/features/bootchain-core/data/etc/initrd/cmdline.d/bootchain-core
>> @@ -1,4 +1,5 @@
>> register_parameter string BOOTCHAIN
>> register_alias BOOTCHAIN PIPELINE
>> +register_parameter bool BC_DEBUG
>> register_array string MOUNTFS
>> register_array string OVERLAYFS
>> diff --git a/features/bootchain-core/data/lib/bootchain/debug b/features/bootchain-core/data/lib/bootchain/debug
>> new file mode 100755
>> index 0000000..9ad4e96
>> --- /dev/null
>> +++ b/features/bootchain-core/data/lib/bootchain/debug
>> @@ -0,0 +1,81 @@
>> +#!/bin/bash -efu
>> +
>> +. bootchain-sh-functions
>> +
>> +
>> +listpvdir()
>> +{
>> + # shellcheck disable=SC2012
>> + if [ -z "${1-}" ]; then
>> + ls -1F "$prevdir/" |sort |head -n20
>> + else
>> + ls -1F "$prevdir/" |sort |grep -svE "$1" |head -n20
> в обоих случаях sort лишний. ls отсортирует сам.
Понял, уберу.
>
>> + fi
>> +}
>> +
>> +
>> +# Entry point
>> +message "$PROG for $name [$callnum] started"
>> +
>> +message "PREV DIR: $prevdir"
>> +message "Data DIR: $datadir"
>> +message "DEST DIR: $destdir"
>> +
>> +{ printf "##############################"
>> +
>> + if [ -d "$prevdir" ]; then
>> + printf "\nPrevious step results (%s):\n" "$prevdir"
>> +
>> + if mountpoint -q -- "$prevdir"; then
>> + listpvdir
>> + elif [ ! -b "$prevdir/dev" ] &&
>> + [ ! -c "$prevdir/dev" ] &&
>> + [ ! -s "$prevdir/DEVNAME" ]
>> + then
>> + listpvdir
>> + else
<<<---
>> + if [ -b "$prevdir/dev" ] || [ -c "$prevdir/dev" ]; then
>> + devname="$(resolve_devname "$prevdir/dev")"
> Эта функция проверяет:
>
> elif [ -b "$dir"/dev ] || [ -c "$dir"/dev ]; then
>
> где dir это аргумент. То есть получается, тут проверяется
> "$prevdir/dev"/dev ?
Весь этот фрагмент ошибочный, виноват. Похоже, переделал второпях,
запихнув и сюда в последний момент resolve_devname(). Именно в шаге
debug необходимо раздельно проверять dev и DEVNAME, в отдельном случае
можно использовать resolve_devname("$prevdir") без лишнего if...fi.
>
>> + devno="$(get_majmin "$prevdir/dev" ||:)"
>> + [ -z "$devno" ] || [ -z "$devname" ] ||
>> + printf 'dev (%s -> %s)\n' "$devno" "$devname"
>> + fi
>> +
>> + if [ -s "$prevdir/DEVNAME" ]; then
>> + read -r devname <"$prevdir/DEVNAME" 2>/dev/null ||
>> + devname=
> Этот же read есть в resolve_devname. Зачем вообще нужно вводить
> resolve_devname если она нигде по сути не используется ?
>
> Почему весь этот дебаг не в resolve_devname ?
Потому что сначала был только debug. Хорошая мысль ввести
resolve_devname() пришла слишком поздно. Переделаю здесь. И ещё много
где надо будет более внимательней добавить вызов этой функции.
>
>> + if [ -z "$devname" ]; then
>> + printf 'DEVNAME\n'
>> + else
>> + devno="$(get_majmin "$devname")" ||
>> + devno="ABSENT"
>> + printf 'DEVNAME (%s -> %s)\n' "$devno" "$devname"
>> + fi
>> + fi
>> +
>> + if [ -s "$prevdir/FILESIZE" ]; then
>> + read -r fsize <"$prevdir/FILESIZE" 2>/dev/null ||
>> + fsize="NOT READABLE"
>> + printf 'FILESIZE (%s)\n' "$fsize"
>> + fi
>> +
>> + listpvdir "^(dev|DEVNAME|FILESIZE)$"
>> + fi
>> + fi
>> +
>> + # FIXME: make this better to specify interested mount points only
>> + [ -z "${OEM_CDROOT-}" ] && regex="$rootmnt" ||
>> + regex="$rootmnt $OEM_CDROOT"
>> + regex="$regex $mntdir"
>> + regex="${regex//\//\\/}"
>> + regex=" (${regex// /\|})\/"
>> +
>> + if mount |grep -qsE "$regex"; then
>> + printf "\nMount points and devices:\n"
>> + mount |grep -sE "$regex"
>> + fi
>> +
>> + printf "##############################\n"
>> +} 1>&2
>> +
>> +message "$PROG for $name [$callnum] finished"
>> diff --git a/features/bootchain-core/data/sbin/chaind b/features/bootchain-core/data/sbin/chaind
>> index 08e5e58..ec4458e 100755
>> --- a/features/bootchain-core/data/sbin/chaind
>> +++ b/features/bootchain-core/data/sbin/chaind
>> @@ -1,11 +1,8 @@
>> #!/bin/bash -efu
>>
>> -. shell-error
>> -. shell-signal
>> +. bootchain-sh-functions
>>
>> -message_time=1
>> pidfile="/var/run/$PROG.pid"
>> -logfile="/var/log/$PROG.log"
>>
>>
>> exit_handler()
>> @@ -16,26 +13,31 @@ exit_handler()
>> exit $rc
>> }
>>
>> +debug()
>> +{
>> + [ -z "$BC_DEBUG" ] ||
>> + message "$*"
>> +}
>> +
>>
>> [ ! -f "$pidfile" ] ||
>> fatal "already running"
>> -
>> set_cleanup_handler exit_handler
>> echo "$$" >"$pidfile"
>>
>> -. bootchain-sh-functions
>> -
>> -
>> [ "${RDLOG-}" != 'console' ] ||
>> - logfile=/dev/console
>> + BC_LOGFILE=/dev/console
>> +
>> +exec >"$BC_LOGFILE" 2>&1
>>
>> -exec >"$logfile" 2>&1
>> -message "Starting server ..."
>> +message "Starting server [$(initrd_version)]..."
>> +debug "Booting with /proc/cmdline:"
>> +fdump /proc/cmdline
>>
>> -mkdir -p -- "$mntdir"
>> +run mkdir -p -- "$mntdir"
>>
>> -mountpoint -q "$mntdir" ||
>> - mount -t tmpfs tmpfs "$mntdir" ||:
>> +mountpoint -q -- "$mntdir" ||
>> + run mount -t tmpfs tmpfs "$mntdir" ||:
>>
>> stepnum=0
>> chainsteps="$BOOTCHAIN"
>> @@ -47,23 +49,25 @@ while [ -n "$chainsteps" ]; do
>> exe="$handlerdir/$name"
>>
>> if [ -x "$exe" ]; then
>> - eval "callnum=\"\${callnum_$name:-0}\""
>> -
>> + assign "callnum" "\${callnum_$name:-0}"
>> datadir="$mntdir/src/step$stepnum"
>> destdir="$mntdir/dst/step$stepnum"
>>
>> [ "$stepnum" != 0 ] ||
>> prevdir=""
>>
>> - mkdir -p -- "$datadir" "$destdir"
>> + run mkdir -p -- "$datadir" "$destdir"
>>
>> if ! mountpoint -q "$destdir"; then
>> message "[$callnum] Handler: $exe"
>>
>> export name callnum datadir destdir prevdir
>>
>> + [ -z "$BC_DEBUG" ] ||
>> + run "$handlerdir/debug" ||:
>> rc=0
>> - "$exe" || rc=$?
>> + run "$exe" ||
>> + rc=$?
>>
>> if [ "$rc" != 0 ]; then
>> [ "$rc" != 2 ] ||
>> @@ -87,3 +91,36 @@ while [ -n "$chainsteps" ]; do
>>
>> stepnum=$(($stepnum + 1))
>> done
>> +
>> +[ -z "$chainsteps" ] ||
>> + message "remaining steps after breaking loop: $chainsteps"
>> +
>> +if [ -z "$BC_DEBUG" ]; then
>> + grep -qs " $mntdir/" /proc/mounts ||
>> + run umount -- "$mntdir" &&
>> + run rm -rf -- "$mntdir" ||:
>> +else
>> + prevdir="$(readlink-e "$destdir" 2>/dev/null ||:)"
>> + [ "$prevdir" = "$rootmnt" ] && datadir="" ||
>> + datadir="$rootmnt"
>> + destdir=
>> + callnum=0
>> + name=STAGE2
>> +
>> + message "[$callnum] Handler: $handlerdir/debug"
>> +
>> + export name callnum datadir destdir prevdir
>> +
>> + run "$handlerdir/debug" ||:
>> + debug "last step finished with exit code $rc"
>> +fi
>> +
>> +if [ -f "$BC_LOGFILE" ] && [ -n "$BC_DEBUG" ]; then
>> + if [ -d "$rootmnt/var/log" ]; then
>> + destdir="$rootmnt/var/log"
>> + else
>> + run mkdir -p -- "$mntdir"
>> + destdir="$mntdir"
>> + fi
>> + run cp -Lf -- "$BC_LOGFILE" "$destdir/"
>> +fi
>> --
>> 2.24.1
>>
>> _______________________________________________
>> Make-initrd mailing list
>> Make-initrd at lists.altlinux.org
>> https://lists.altlinux.org/mailman/listinfo/make-initrd
>>
--
Best regards,
Leonid Krivoshein.
More information about the Make-initrd
mailing list