udev does not start under OpenVZ gentoo container

[Alexander <piavka@cs.bgu.ac.il>]

Hi, I've upgraded one of my gentoo openvz containers from
baselayout-2.0.0_rc6-r1 to baselayout-2.0.0 and openrc-0.2.5

I've ported /etc/conf.d/rc to /etc/rc.conf ,in particular i have
rc_devices="udev"
rc_device_tarball="YES"
The VE starts ok , but udev is not started.

I think that running udev in openvz contrainer is legitimate

and has it's advantages. So please consider the following patch
which allows udev to start in openvz container.

---

# diff -Nru /lib/rc/sh/init.sh.orig /lib/rc/sh/init.sh
--- /lib/rc/sh/init.sh.orig 2008-07-23 01:09:31.000000000 +0300
+++ /lib/rc/sh/init.sh 2008-07-23 01:10:44.000000000 +0300
@@ -129,7 +129,6 @@

# Try to figure out how the user wants /dev handled
if [ "${rc_devices}" = "static" \

-o "${RC_SYS}" = "VSERVER" \

• -o "${RC_SYS}" = "OPENVZ" \

-o "${RC_UNAME}" = "GNU/kFreeBSD" ]

then

ebegin "Using existing device nodes in /dev"

---

Also /lib/rc/sh/init.sh skips mounting /sys
in openvz container, I think it may be useful visually usefull
then using tools like udevinfo to know sysfs device path.
What do you think?

PS.
Also I've rc_logger="YES" but /var/log/rc.log is not created.
This is especially useful then starting openvz container since
is does not display any output then container is booting.
AFAI understand rc_logger is embedded into /sbin/r, is
there something that does not like openvz in the rc.c code
regarding rc_logger or i just have some missconfiguration?

Thanks
Alex

[Alexander <piavka@cs.bgu.ac.il>]

bump

[roy]

Sorry, I'm trying to get hold of Gentoo dev Hollow to comment on this as he provided the logic for this.

[Alexander <piavka@cs.bgu.ac.il>]

Ok, but if udev and /sys mounting can be controlled from /etc/rc.conf

why not to leave this choice to the admin, while the default could be
not to use udev and /sys for openvz and vserver.

I'm using both udev and /sys for a couple of weeks with openrc-0.2.5

and did not have any issues with such setup.

Also could you comment regarding of: why then i have rc_logger="YES"

nothing is logged in /var/log/rc.log?

Thanks
Alex

[Alexander <piavka@cs.bgu.ac.il>]

Any new from Hollow commenting on running udev inside a container?

Also it would be great to have some response on the rc_logger="YES"

issue, maybe you want me to open a separate bug for it?

[roy]

(In reply to comment #4)

Any new from Hollow commenting on running udev inside a container?

Also it would be great to have some response on the rc_logger="YES"

issue, maybe you want me to open a separate bug for it?

Still trying to get some comment on this.

rc_logger also requires PTY support in the kernel. Does OpenVZ support this?

[Daniel Robbins <drobbins@funtoo.org>]

The proper way to do this is to set rc_devices="static" and then to put a skeleton set of device nodes in your VE in /dev that do not change. You do not need to run udev inside your container. At least for any reason I know of. If there is a good reason to run udev inside a VE, then post it here. As far as I know, udev doesn't do anything for you in a VE since VE's generally use a minimal static set of device nodes and do not access hardware directly so there is no need to dynamically add/remove device nodes.

[Alexander <piavka@cs.bgu.ac.il>]

(In reply to comment #6)

The proper way to do this is to set rc_devices="static" and then to put a
skeleton set of device nodes in your VE in /dev that do not change. You do not
need to run udev inside your container. At least for any reason I know of. If
there is a good reason to run udev inside a VE, then post it here.

I think there is a reason, pls read below

As far as I
know, udev doesn't do anything for you in a VE since VE's generally use a
minimal static set of device nodes and do not access hardware directly

BTW there is 'vzctl --devnodes ...' which gives VE access to devices it otherwise would not see.

so there is no need to dynamically add/remove device nodes.

Actually udev behavior inside same VE differs depending on kernel base,

more details below.

What i currently do is set rc_devices="udev" & rc_device_tarball="YES"

And i have same minimal device tarball distributed for all different VEs

---

# tar jtvf /vz/private/1000/lib64/udev/state/devices.tar.bz2
crw-r--r-- root/root 1,8 2008-02-06 12:01 random
crw-r--r-- root/root 1,9 2008-02-06 12:01 urandom

---

/dev/{u,}random are the only needed devices that udev did not create.
This is all i have to do while udev takes care of the rest.
For example from withing running VE i have

---

# ls -la /dev/
total 56
drwxr-xr-x 5 root root 380 2008-08-17 10:43 .
drwxr-xr-x 21 root root 4096 2008-08-12 01:20 ..
crw-r--r-- 1 root root 5, 1 2008-02-06 12:00 console
lrwxrwxrwx 1 root root 13 2008-08-17 10:43 fd -> /proc/self/fd
prw------- 1 root root 0 2008-08-17 10:43 initctl
srw-rw-rw- 1 root root 0 2008-08-17 10:43 log
crwxrwxrwx 1 root root 1, 3 2008-02-06 12:00 null
crw-rw-rw- 1 root tty 5, 2 2008-08-19 10:29 ptmx
drwxr-xr-x 2 root root 0 2008-08-17 10:43 pts
crw-r--r-- 1 root root 1, 8 2008-02-06 12:01 random
drwxrwxrwt 2 root root 40 2008-08-17 10:43 shm
lrwxrwxrwx 1 root root 4 2008-08-17 10:43 stderr -> fd/2
lrwxrwxrwx 1 root root 4 2008-08-17 10:43 stdin -> fd/0
lrwxrwxrwx 1 root root 4 2008-08-17 10:43 stdout -> fd/1
crw-r--r-- 1 root root 4, 1 2008-02-06 12:00 tty1
drwxr-xr-x 6 root root 140 2008-08-17 10:43 .udev
crw-r--r-- 1 root root 1, 9 2008-02-06 12:01 urandom

---

Today instead of vanilla 2.6.18 kernel with openvz patch i've compiled

and booted RHEL 2.6.18 kernel with openvz patch.
And i've changed to udev only without tarball:
rc_devices="udev" & rc_device_tarball="YES"
and it turn out that with RHEL udev created the /dev/{u,}random devices also
and several others too:

---

# ls -la /dev
total 4
drwxr-xr-x 5 root root 400 2008-08-19 13:35 .
drwxr-xr-x 17 root root 4096 2008-07-02 01:02 ..
crw-r--r-- 1 root root 5, 1 2008-08-19 13:35 console
lrwxrwxrwx 1 root root 13 2008-08-19 13:35 fd -> /proc/self/fd
crw-rw-rw- 1 root root 1, 7 2008-08-19 13:35 full
prw------- 1 root root 0 2008-08-19 13:35 initctl
srw-rw-rw- 1 root root 0 2008-08-19 13:35 log
crw-rw-rw- 1 root root 1, 3 2008-08-19 13:35 null
crw-rw-rw- 1 root tty 5, 2 2008-08-19 14:50 ptmx
drwxr-xr-x 2 root root 0 2008-08-19 13:35 pts
crw-rw-rw- 1 root root 1, 8 2008-08-19 13:35 random
drwxrwxrwt 2 root root 40 2008-08-19 13:35 shm
lrwxrwxrwx 1 root root 4 2008-08-19 13:35 stderr -> fd/2
lrwxrwxrwx 1 root root 4 2008-08-19 13:35 stdin -> fd/0
lrwxrwxrwx 1 root root 4 2008-08-19 13:35 stdout -> fd/1
crw-r--r-- 1 root root 4, 1 2008-08-19 13:35 tty1
drwxr-xr-x 6 root root 140 2008-08-19 13:35 .udev
crw-rw-rw- 1 root root 1, 9 2008-08-19 13:35 urandom
lrwxrwxrwx 1 root root 4 2008-08-19 13:35 XOR -> null
crw-rw-rw- 1 root root 1, 5 2008-08-19 13:35 zero

---

So with RHEL openvz kernel i can just use pure udev.

Then i've also tried with static only /dev: the devpts under /dev/pts and /dev/shm were not mounted (i guess i could simply explicitly add these to /etc/fstab)

Also i don't like creating a bunch of /dev/{p,t}ty* statically,
so i just use the old style BSD PTYs with dynamic ptmx and /dev/pts, but
i think if i disable BSD PTYs in kernel udev might create /dev/{p,t}ty*
on the fly.

IMHO udev with device tarball is more elegant and convenient solution for VE than static udev only.

Unless running udev is really harmful to VE

i think that udev should be left as choice for sysadmin.
What do you think?

One of the things i've noticed with udev inside VE is that

/dev/console and /dev/tty1 are created with 644 perms
in spite of that udev rules supposed to change it to 600
50-udev-default.rules:KERNEL=="console", MODE="0600", OPTIONS="last_rule"
It would be nice to investigate why but i'm not udev expert.

So to be safe i've included /dev/console and /dev/tty1 with 600 perms to device tarball.

[Alexander <piavka@cs.bgu.ac.il>]

(In reply to comment #5)

rc_logger also requires PTY support in the kernel. Does OpenVZ support this?

Yes

---

# zgrep -i pty /proc/config.gz
CONFIG_UNIX98_PTYS=y
CONFIG_LEGACY_PTYS=y
CONFIG_LEGACY_PTY_COUNT=256

---

Also i've booted VE with /dev/{p,t}ty* precreated and still it does not work.
Other ideas?

[Daniel Robbins <drobbins@funtoo.org>]

I did check on wiki.openvz.org and udev is supported as a possible configuration. It is not common or useful for most people, but it can be done. There are some udev tips spread between these two pages:

 http://wiki.openvz.org/Container_enter_failed (how to do device nodes, etc.)
 http://wiki.openvz.org/Installing_udev_in_VE_based_on_Debian-4.0 (how to enable sysfs)

I have not personally tried using udev inside a VE as it is easier to disable it and everything works fine. If you want to see what static device nodes to use under gentoo, grab one of my Gentoo OpenVZ templates from  http://www.funtoo.org/linux/openvz and cp -a /dev to your VE template and rc_devices="static" should then work fine.

Regards,

Daniel

[roy]

Allow OpenVZ to mount /sys and /dev

[roy]

Created an attachment (id=47)
Allow OpenVZ to mount /sys and /dev

Does this patch fit the bill?
We default to using static device nodes on OpenVZ, but will allow it to be overriden by using rc_devices.

[Alexander <piavka@cs.bgu.ac.il>]

(In reply to comment #10)

Does this patch fit the bill?

Yes it looks fine, thanks!

Do you have any ideas regarding rc_logger log creation?

[roy]

Fixed in  http://git.overlays.gentoo.org/gitweb/?p=proj/openrc.git;a=commitdiff;h=a72649062717502240d547530ee5b219e40758d1

Thanks to all :)

(In reply to comment #11)

Do you have any ideas regarding rc_logger log creation?

No, please open a new bug for that.