[devel] Fwd: Native upstart scripts
Michael Shigorin
mike на osdn.org.ua
Ср Май 12 12:58:02 UTC 2010
Здравствуйте.
Вдруг пригодится... из открытых источников... и всё такое.
----- Forwarded message from Jacek Konieczny <jajcus/jajcus.net> -----
Date: Tue, 4 May 2010 11:17:41 +0200
From: Jacek Konieczny <jajcus/jajcus.net>
To: PLD Devel <pld-devel-en/lists.pld-linux.org>
Subject: Native upstart scripts
Hello,
In PLD Th, we have upstart as the /sbin/init daemon for some time, but
it is only used to start the old 'SysVinit' scripts from
/etc/rc.d/init.d. To make full use of Upstart features, like process
supervising, parallel start, etc. we should find a way to include
Upstart support in our packages. Though, we should not replace the
current, working solution.
I think Upstart support should be implemented as an option, coexisting
with current solution, so the administrator may choose what he prefers
and even use init.d for some services and upstart for other.
My proposition:
- packages will provide upstart configuration files in /etc/rc.d/upstart
- those could be linked or copied to /etc/init/subsys when needed
- chkconfig would link/unlink the files when requested (global
configuration option and command line option to prefer upstart)
- user could copy them manually and modify them if needed
- rc-scripts won't start/stop services via /etc/rc.d/init.d scripts when
/etc/init/subsys/$name exists
- 'service' script would use 'initctl' instead of /etc/rc.d/init.d
when /etc/init/subsys/$name exists
- scripts in /etc/rc.d/init.d would emit 'started' and 'stopped'
events when necessary, so upstart services can rely on that
What do you think? Should I try to prepare a proof-of-concept
implementation?
P.S. Does anybody read those 'long' posts I send to pld-devel-en? ;)
_______________________________________________
pld-devel-en mailing list
pld-devel-en/lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en
----- End forwarded message -----
----- Forwarded message from Jacek Konieczny <jajcus/jajcus.net> -----
Date: Tue, 4 May 2010 12:53:40 +0200
From: Jacek Konieczny <jajcus/jajcus.net>
To: pld-devel-en/lists.pld-linux.org
Subject: Re: Native upstart scripts
On Tue, May 04, 2010 at 12:04:39PM +0200, Patryk Zawadzki wrote:
> > - scripts in /etc/rc.d/init.d would emit 'started' and 'stopped'
> > events when necessary, so upstart services can rely on that
>
> I'd opt for having 2 separate -init subpackages, one with the current
> rc.d contents and one with an upstart job description and a simple
> rc.d wrapper that runs "start $foo", "stop $foo" etc.
There is one more thing why I would prefer to keep both old-style init.d
script and the new upstart configuration together: we often have
something more than 'start' 'stop' and 'status' implemented in the init
scripts. Some of these actions would have to be copied to upstart
script somehow (initialization before first run), other probably cannot
be handled by upstart means.
sshd init script builds host keys the first time it is started. This
functionality is available also via '/etc/rc.d/init.d/sshd init' and
would have to be copied to the upstart configuration to and maintained
in the two places.
As starting and stopping of services is quite different in LSB init
scripts and upstart and reusing code does not make sens, in other cases,
like 'init' the code may be reused and upstart script could just call
'/etc/rc.d/init.d/sshd init'.
Also, there are things like '/etc/rc.d/init.d/lighttpd configtest' which
are not part of normal job control and thus not applicable to upstart
config, but still useful.
Greets,
Jacek
----- End forwarded message -----
----- Forwarded message from Jacek Konieczny <jajcus/jajcus.net> -----
Date: Tue, 4 May 2010 19:51:49 +0200
From: Jacek Konieczny <jajcus/jajcus.net>
To: pld-devel-en/lists.pld-linux.org
Subject: Re: Native upstart scripts
On Tue, May 04, 2010 at 07:40:43PM +0300, Elan Ruusam??e wrote:
> On Tuesday 04 May 2010 12:17:41 Jacek Konieczny wrote:
> > My proposition:
> > - packages will provide upstart configuration files in /etc/rc.d/upstart
> > - those could be linked or copied to /etc/init/subsys when needed
> > - chkconfig would link/unlink the files when requested (global
> > configuration option and command line option to prefer upstart)
> > - user could copy them manually and modify them if needed
>
> i don't like the idea that the links are managed via some script, i'd like
> easily to boot to upstart-mode or sysvinit-mode with a kernel commandline, or
> something in /etc/sysconfig/system the both solutions should be available and
> configured at the same time.
I'll take this into account. Though keeping both configured could cause
mistakes like starting a service again the different way. I guess this
can be solved somehow anyway
> also i'd not invent new paths, but use /etc/init for upstart scripts.
Recent upstart allows hierarchical names. I thought a separate namespace
(subsys/* or rc/*) for the rc-scripts equivalents would be a good idea.
Now I think it will be probably better to use subdirs for local stuff
(/etc/init/local -> local/* jobs) and sub-tasks (e.g.
/etc/init/postgresql.conf ??? 'postgresql' task to start/stop whole
postgresql and /etc/init/postgresql/instance.conf ???
'postgresql/instance' describing each instance) instead.
Greets,
Jacek
----- End forwarded message -----
----- Forwarded message from Jacek Konieczny <jajcus/jajcus.net> -----
Date: Wed, 5 May 2010 12:34:34 +0200
From: Jacek Konieczny <jajcus/jajcus.net>
To: pld-devel-en/lists.pld-linux.org
Subject: Re: Native upstart scripts
On Tue, May 04, 2010 at 07:40:43PM +0300, Elan Ruusam?e wrote:
> i don't like the idea that the links are managed via some script, i'd like
> easily to boot to upstart-mode or sysvinit-mode with a kernel commandline, or
> something in /etc/sysconfig/system the both solutions should be available and
> configured at the same time.
I thought about that and cannot see how it could be implemented. When
there are upstart configs for some services then Upstart will start them
by the dependencies inside and I cannot see a way to stop this from a
kernel command line and still have the flexibility of which services
should be handled by Upstart and which the old way.
Greets,
Jacek
----- End forwarded message -----
----- Forwarded message from Jacek Konieczny <jajcus/jajcus.net> -----
Date: Wed, 5 May 2010 12:31:55 +0200
From: Jacek Konieczny <jajcus/jajcus.net>
To: pld-devel-en/lists.pld-linux.org
Subject: Re: Native upstart scripts
On Wed, May 05, 2010 at 11:20:30AM +0200, Patryk Zawadzki wrote:
> On Tue, May 4, 2010 at 11:17 AM, Jacek Konieczny <jajcus/jajcus.net> wrote:
> > In PLD Th, we have upstart as the /sbin/init daemon for some time, but
> > it is only used to start the old 'SysVinit' scripts from
> > /etc/rc.d/init.d. To make full use of Upstart features, like process
> > supervising, parallel start, etc. we should find a way to include
> > Upstart support in our packages. Though, we should not replace the
> > current, working solution.
>
> BTW: http://0pointer.de/blog/projects/systemd.html
Great. This fixes some problems of Upstart??? but???
This would look great to me if I haven't seen several other ???great
SysVinit replacements???. All of them were much better than SysVinit and
even had some ???working implementations???. Often long before the Upstart
came up. And everybody have been still using SysVinit.
Upstart is the only one which caught on. It is terrible, with its
big incompatibilities between each version??? poor documentation (at least
on the web), impractical event system. Nevertheless it is still much
better than SysVinit with the pile of shell scripts starting daemons
doing anything not to be managed (double-forking, etc.).
Unfortunately /sbin/init is so critical, that we cannot even experiment
much, especially with something no one else uses. And it is hard to have
multiple /sbin/init implementation side by side in one distribution.
They all differ too much not only in the configuration file formats but
in the whole philosophy of the configuration (not only ???how to describe
a service configuration??? but even ???what is a service configuration???).
One could think of making some kind of an abstraction layer, like our
rc-inetd, but that is a very bad idea. We would probably lose most of
the advantages of each init implementation then.
We should rather think how can we provided a few very different
configuration items, each for a different init implementation, with one
package. I am losing my enthusiasm about implementing the ???native
upstart support??? in PLD when I think systemd may eventually mature
enough and we would need to start from the beginning???
And I need a good init system now. Currently I use SysVinit +
daemontools in my PLD-based project. It generally works, but is not
elegant nor optimal. Even upstart, when properly deployed, would be
better and systemd could be much better, but I guess it is a matter of
future.
Pozdrowienia,
Jacek
----- End forwarded message -----
----- Forwarded message from Jacek Konieczny <jajcus/jajcus.net> -----
Date: Wed, 5 May 2010 14:32:19 +0200
From: Jacek Konieczny <jajcus/jajcus.net>
To: pld-devel-en/lists.pld-linux.org
Subject: Re: Native upstart scripts
On Wed, May 05, 2010 at 02:13:51PM +0200, Patryk Zawadzki wrote:
> On Wed, May 5, 2010 at 2:03 PM, ??ukasz Jerna?? <deejay1/srem.org> wrote:
> > Well, after seeing how many people have weird problems with PulseAudio
> > I would be cautious about using another thing thought out by
> > Lennart...
>
> To be fair you have to admit that a vast number of the problems
> experienced while using pulseaudio are actually bugs in kernel audio
> modules and/or alsa API abuse (think opal, ekiga etc.).
I would rather say it is because of putting another audio server where
it is not needed at all. ALSA alone does its job well enough nowadays
(and if it does not, it should be fixed not wrapped with another layer).
Fortunately things have not gone too far yet and a system without
PulseAudio can still be set up (there was no such freedom in the dark
ages of ESD and ARTS).
This /sbin/init things are not that easy. We will always need some init
daemon??? though I still won't chose an implementation only basing on the
fact that the idea is great. It must work and be maintained (or at least
stable as SysVinit) too.
Greets,
Jacek
----- End forwarded message -----
----- Forwarded message from Jacek Konieczny <jajcus/jajcus.net> -----
Date: Fri, 7 May 2010 15:33:21 +0200
From: Jacek Konieczny <jajcus/jajcus.net>
To: PLD Devel <pld-devel-en/lists.pld-linux.org>
Subject: Native upstart scripts ??? implemented
Hello,
Your volunteer has done his job :)
I have made the proof-of-concept implementation of native upstart job
control. I did it a bit different way than I initially suggested ???
taking into account your comments and improving my own ideas by trial
and errors.
My main requirements were fulfilled:
- this doesn't break current /etc/rc.d/init.d/* scripts
- this can coexist with the current /etc/rc.d/init.d/* script based
job control
- /etc/init.d/* scripts are quite LSB compatible even when the jobs
are controlled by upstart.
Currently all the changes are applied to 'upstart_native' branch, so the
solution may be tested before merging into the distribution. As it
doesn't seem to break anything I guess it can be merged very soon (after
the weekend?) unless someone shows me some big problem with the code or
has a good idea on how some things could be done better.
I have modified:
- rc-scripts
to provide facilities needed for upstart-controlled services
- syslog-ng
as an example of a generic service implementation other services
rely on ??? how to make job depend on a 'syslog' service not
a specific 'syslog-ng' implementation
- openssh-server
as an example of some system service relying on network and
syslog
- postgresql
as an example of multi-instance service. It also shows how
to run a service with non-root uid (it is strange upstart
doesn't provide this function directly)
To play with that build rc-scripts, syslog-ng, openssh and postgresql
from the 'upstart_native' branch, upgrade to these builds (this should
change nothing in how your system works) and then install
syslog-ng-upstart, openssh-upstart and postgresql-upstart.
When the *-upstart packages are installed you can still control the
services with /sbin/service or /etc/rc.d/init.d/$service scripts
or directly with the 'initctl' tool.
The way it is implemented gives two options of booting the PLD system:
1. the traditional ??? serialized, slow but verbose and colorful way
??? just do not install the *-upstart packaged or boot with
'pld.no-upstart'
2. the new event-based, parallelized, auto-respawning way, but less
verbose
- instal the *-upstart packages and let them do their job
Some documentation for the rc-scripts+upstart usage is here:
http://svn.pld-linux.org/cgi-bin/viewsvn/rc-scripts/branches/upstart_native/doc/upstart.txt?rev=11395&view=markup
Patryk Zawadzki <patrys/pld-linux.org> wrote:
> I'd opt for having 2 separate -init subpackages, one with the current
> rc.d contents and one with an upstart job description and a simple
> rc.d wrapper that runs "start $foo", "stop $foo" etc.
-upstart subpackages done. Addin "-init" makes no sense, as current
upstart job handling implementaion relies on the init.d scripts for LSB
compatibility and doing things not doable with bare Upstart
(non-SIGHUP-reloading, 'checkconfig', advanced status monitoring).
Elan Ruusam??e <glen/delfi.ee> wrote:
> i don't like the idea that the links are managed via some script, i'd like
> easily to boot to upstart-mode or sysvinit-mode with a kernel commandline, or
> something in /etc/sysconfig/system the both solutions should be available and
> configured at the same time.
Done. No chkconfig patching, no script-managed-links, only a minor
update do /sbin/service added.
The new, upstart event-based boot may be disabled by a kernel command
line option: "pld.no-upstart".
> also i'd not invent new paths, but use /etc/init for upstart scripts.
Done. But this has some consequences:
- The package-provided /etc/init/*.conf files are config files. Expect
user to modify them in any way. Do not put things, that may change on
every upgrade, there.
- Package-provided job description could conflict with used locally or
provided by third-party packages. I suggest using
/etc/init/local/*.conf for local jobs and keeping /etc/init/*.conf
files for /etc/rc.d/init.d/* equivalents (put extra jobs needed for
the functionality into a subdirectory)
Greets,
Jacek
----- End forwarded message -----
----- Forwarded message from Jacek Konieczny <jajcus/jajcus.net> -----
Date: Sun, 9 May 2010 08:57:57 +0200
From: Jacek Konieczny <jajcus/jajcus.net>
To: pld-devel-en/lists.pld-linux.org
Subject: Re: Native upstart scripts ??? implemented
On Sun, May 09, 2010 at 02:52:57AM +0300, Elan Ruusam?e wrote:
> i'd rather avoid completely the new subpackage here, if needed move
> the /etc/init dir to filesystem package to avoid dirdeps pulling upstart, and
> use conflicts tag for the current requires tag.
Patrys asked for subpackages and it makes sense - the easiest to choose
between upstart-way and the-old-way per package.
> syslog-ng/syslog-ng.init:
> ...
> upstart_controlled --except configtest
>
> i don't really understand how flush-logs is handled,
It is not. 'service syslog-ng flush-logs' will tell you that. On the
other hand - with upstart 'reload' does the same.
Of course, flush-logs can be implemented in the init script. I guess it
should if it is used by anything else (logrotate?)
> also how do you specify multiple excludes remains unclear.
When '--except' is used, then all what follows are the excludes.
> also, configtest before reload/restart action would be really important to
> have in upstart as well, considering that we restart services on rpm
> upgrades. does upstart have such concept after all as restart/reload in
> scripts?
No. 'reload' in upstart is 'kill -HUP', anything else must be
re-implemented in the init script. Restart is 'stop; start'. Hmmm...
Now I can see 'restart with configtest' may be easily implemented.
Whenever '--except configtest' is used, 'upstart_controlled' can call
configtest before trying restart.
configtest on start makes little sense with upstart IMHO.
> ps: would be nice to know where's source of documentation, for example i did
> not find myself description of job file directives.
>From the rc-scripts init.txt:
The syntax of the ``/etc/init/*.conf`` files is described in the
init(5) man page.
And yes, looking for a current Upstart documentation in the web gives
no good results.
Greets,
Jacek
----- End forwarded message -----
----- Forwarded message from Jacek Konieczny <jajcus/jajcus.net> -----
Date: Sun, 9 May 2010 11:00:57 +0200
From: Jacek Konieczny <jajcus/jajcus.net>
To: pld-devel-en/lists.pld-linux.org
Subject: Re: Native upstart scripts ??? implemented
On Sun, May 09, 2010 at 11:39:14AM +0300, Elan Ruusam??e wrote:
> On Friday 07 May 2010 16:33:21 Jacek Konieczny wrote:
> > Hello,
> >
> > Your volunteer has done his job :)
>
> seems there's some deadlock with initctl emiting
Isn't that another result of https://bugs.launchpad.net/upstart/+bug/406397
> also seems the "nice service name" is lost there (see sshd part).
How is that supposed to work and where is that 'sshd part'?
> also seems there's no our typical restart service after package upgrade,
> if you've upgrading upstart-enabled service.
> %define upstart_post() \
> if [ -f /var/lock/subsys/"%1" ] ; then \
> /sbin/service --no-upstart "%1" stop \
> /sbin/service "%1" start \
> fi
The service will be restarted after main package upgrade. Though we have
the real problem of -upstart subpackage here: the restart should
probably be done when both main and -upstart packages are upgraded.
> anyway, ps of stall:
> root 25974 0.4 0.5 15596 6064 pts/3 S+ 11:23 0:00 \_ poldek -u upstart --up --sn carme openssh-server-upstart syslog-ng-upstart
> root 27078 0.4 0.5 12516 5804 pts/3 S+ 11:24 0:00 \_
> rpm --upgrade -vh --root / /var/cache/poldek/http_carme.pld-linux.org..glen.th.i686/syslog-ng-3.0.5-2.1.i
> root 27181 0.0 0.0 1872 584 pts/3 S+ 11:24 0:00 \_ /bin/sh /home/users/glen/tmp/rpm-tmp.59975 2
> root 27184 0.0 0.0 1872 604 pts/3 S+ 11:24 0:00 \_ /bin/sh /sbin/service syslog-ng restart
> root 27187 0.0 0.0 2000 808 pts/3 S+ 11:24 0:00 \_ /bin/sh /etc/rc.d/init.d/syslog-ng restart
> root 27321 0.0 0.0 5716 972 pts/3 S+ 11:24 0:00 \_ /sbin/initctl emit started JOB=syslog-ng SERVICE=syslog
I guess --no-wait for emit should be used here. Though there is
something blocking ??? emit waits for some action on the 'started' event
to finish and some service starting on this even has probably locked up
(most probably due to https://bugs.launchpad.net/upstart/+bug/406397 and
broken /etc/init/*.conf file (I have recently fixed those for openssh
and syslog-ng)).
Greets,
Jacek
----- End forwarded message -----
----- Forwarded message from Jacek Konieczny <jajcus/jajcus.net> -----
Date: Sat, 8 May 2010 22:25:27 +0200
From: Jacek Konieczny <jajcus/jajcus.net>
To: pld-devel-en/lists.pld-linux.org
Subject: Re: Native upstart scripts
On Sat, May 08, 2010 at 10:41:07PM +0300, Elan Ruusam?e wrote:
> On Tuesday 04 May 2010 12:17:41 Jacek Konieczny wrote:
> > What do you think? Should I try to prepare a proof-of-concept
> > implementation?
>
> any plans of moving rc.sysinit also to upstart?
No. I don't think we would gain a lot. Reimplementing all these stuff
via Upstart keeping all this working may be quite difficult and
maintaining both will be a hell.
And no, switching fully to Upstart at this point is not a good idea.
Have anybody recently upgraded upstart 0.5 to 0.6 on a production
machine? Have anybody used 'expect daemon' instead of 'expect fork' or
made a similar mistake? Any of these would break such system so it won't
even shut down or reboot properly.
Upstart is great in many points much much better than SysVinit. But it
is not ready to be the only option for a distribution. Let's add upstart
support to more and more packages, but do not make it a critical
component of a PLD system.
Greets,
Jacek
_______________________________________________
pld-devel-en mailing list
pld-devel-en/lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en
----- End forwarded message -----
--
---- WBR, Michael Shigorin <mike на altlinux.ru>
------ Linux.Kiev http://www.linux.kiev.ua/
Подробная информация о списке рассылки Devel