Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 4 Sep 2017 08:51:10 +0200
From:      Giulio Ferro <auryn@zirakzigil.org>
To:        freebsd-hackers@freebsd.org
Subject:   Re: devd in jail
Message-ID:  <4a1a99a5-35ea-19c9-7ac8-77875ac6f71f@zirakzigil.org>
In-Reply-To: <20170810225439.Horde.1s8Qi_dlNtxgEigsNKbdrer@webmail.leidinger.net>
References:  <e03a6040-1322-c82c-0e96-49c474188d5c@zirakzigil.org> <CAOtMX2g7PR9S7v%2BrUXwBeQUPDFfJy2zZwAvg7q5ze99jDF4hmA@mail.gmail.com> <20170810225439.Horde.1s8Qi_dlNtxgEigsNKbdrer@webmail.leidinger.net>

next in thread | previous in thread | raw e-mail | index | archive | help
On 10/08/2017 22:54, Alexander Leidinger wrote:
>
> Quoting Alan Somers <asomers@freebsd.org> (from Wed, 9 Aug 2017 
> 13:14:20 -0600):
>
>> On Wed, Aug 9, 2017 at 12:47 PM, Giulio Ferro <auryn@zirakzigil.org> 
>> wrote:
>>> Hello all,
>>>
>>> Setup :  11.1-STABLE FreeBSD 11.1-STABLE #0 r321925M amd64
>>>
>>>
>>> I'm trying to create a fully virtualized desktop enviroment in a 
>>> jail by
>>> means of installing there
>>> a xrdp-devel server + Xorg installation (xorg + xorgrdp).
>>>
>>> Everything seems to work until the moment when the X server actually 
>>> tries
>>> to come up (after I choose session=xorg, username + password)
>>> In the X logs in the jail, in fact, I have this error:
>>>
>>> ...
>>> [  9768.824] (EE) config/devd: fail to connect to devd
>>> [  9768.824] [config] failed to initialise devd
>>>
>>>
>>> I've checked on the host machine, and I don't have that error as 
>>> everything
>>> works fine there...
>>>
>>> In my jail, I've setup the devfs like this (/etc/jail.conf in the 
>>> host):
>>>
>>> ---
>>> exec.start="/bin/sh /etc/rc";
>>> exec.stop="/bin/sh /etc/rc.shutdown";
>>> exec.clean;
>>> mount.devfs;
>>> devfs_ruleset=1;
>>>
>>> path="/usr/home/jail/$name";
>>>
>>> myjail {
>>>         host.hostname="myjail.me.com";
>>>         vnet;
>>>         vnet.interface = epair0b, epair1b;
>>>         persist;
>>> }
>>> ---
>>>
>>>
>>> and in the /etc/devfs.conf everythink is commented out.
>>>
>>> In the /dev directory in the jail, I get both the devctl and devctl2
>>> devices.
>>>
>>> As the devd demon is not running in the jail, I've tried adding
>>> devd_enable="YES"
>>>
>>> in the rc.conf (jail), but when I try to start it, I get:
>>>
>>> # /etc/rc.d/devd start
>>> Starting devd.
>>> devd: Can't open devctl device /dev/devctl: Device busy
>>> /etc/rc.d/devd: WARNING: failed to start devd
>>>
>>>
>>> Do you know if I'm doing something wrong, or there's a proper way to 
>>> have
>>> devd running in the jail?
>>>
>>> I've thought that maybe I should use the devtcl2 device, as the 
>>> devctl is
>>> used by the host,
>>> but I don't know how to specify that to devd...
>>>
>>> Thanks in advance for your help.
>>>
>>> Giulio
>>
>> Unfortunately, you're not going to be able to run devd(8) in the jail.
>> /dev/devctl can be opened by only one reader at a time, and that
>> reader is always devd(8).  /dev/devctl2 is actually a completely
>> different device with a totally different interface.  Apologies for
>> the confusing names.  But you may not need to run a totally separate
>> instance of devd.  The X server is probably trying to open either
>> /var/run/devd.pipe or /var/run/devd.seqpacket.pipe.  ktrace would tell
>> you which.  If you can bridge those sockets into the jail, then X
>> would probably run.
>
> Apart from using an explicit config of devices instead of HAL / devd, 
> if this is a X server connecting to a graphics card (instead of just a 
> remote accessible framebuffer), the X-in-a-jail patches are needed, as 
> the X server needs access to /dev/(k)mem and /dev/io (and /dev/drm).
>

Hi Alexander,

as I said, I don't actually need to connect to any device, as this Xorg 
server in the jail only uses xrdp as backend.




> ATTENTION: doing this compromises the complete security of the entire 
> machine. The jail with the X-server can then access the entire memory 
> of the machine, this means circumventing any kernel-level security 
> protection / jail restrictions.

Can you confirm that the following patch is everything I need to run 
Xorg in a jail in a 11.1 stable system?

>
> Using the PR_ALLOW_* flags from current in 11.1 is ok (sys/sys/jail.h):
> ---snip---
> /* Flags for pr_allow */
> #define PR_ALLOW_SET_HOSTNAME           0x00000001
> #define PR_ALLOW_SYSVIPC                0x00000002
> #define PR_ALLOW_RAW_SOCKETS            0x00000004
> #define PR_ALLOW_CHFLAGS                0x00000008
> #define PR_ALLOW_MOUNT                  0x00000010
> #define PR_ALLOW_QUOTAS                 0x00000020
> #define PR_ALLOW_SOCKET_AF              0x00000040
> #define PR_ALLOW_MOUNT_DEVFS            0x00000080
> #define PR_ALLOW_MOUNT_NULLFS           0x00000100
> #define PR_ALLOW_MOUNT_ZFS              0x00000200
> #define PR_ALLOW_MOUNT_PROCFS           0x00000400
> #define PR_ALLOW_MOUNT_TMPFS            0x00000800
> #define PR_ALLOW_MOUNT_FDESCFS          0x00001000
> #define PR_ALLOW_MOUNT_LINPROCFS        0x00002000
> #define PR_ALLOW_MOUNT_LINSYSFS         0x00004000
> #define PR_ALLOW_RESERVED_PORTS         0x00008000
> #define PR_ALLOW_KMEM_ACCESS            0x00010000      /* reserved, 
> not used yet */
> #define PR_ALLOW_ALL                    0x0001ffff
> ---snip---
>
> Then kern_jail.c needs a little patch:
> ---snip---
> Index: sys/kern/kern_jail.c
> ===================================================================
> --- sys/kern/kern_jail.c    (revision 321365)
> +++ sys/kern/kern_jail.c    (working copy)
> @@ -200,6 +200,7 @@
>         "allow.mount.linprocfs",
>         "allow.mount.linsysfs",
>         "allow.reserved_ports",
> +       "allow.kmem_access",
>  };
>  const size_t pr_allow_names_size = sizeof(pr_allow_names);
>
> @@ -220,6 +221,7 @@
>         "allow.mount.nolinprocfs",
>         "allow.mount.nolinsysfs",
>         "allow.noreserved_ports",
> +       "allow.nokmem_access",
>  };
>  const size_t pr_allow_nonames_size = sizeof(pr_allow_nonames);
>
> @@ -3344,6 +3346,27 @@
>                 return (0);
>
>                 /*
> +                * Allow access to /dev/io in a jail if the non-jailed 
> admin
> +                * requests this and if /dev/io exists in the jail. This
> +                * allows Xorg to probe a card.
> +                */
> +       case PRIV_IO:
> +               if (cred->cr_prison->pr_allow & PR_ALLOW_KMEM_ACCESS)
> +                       return (0);
> +               else
> +                       return (EPERM);
> +
> +               /*
> +                * Allow low level access to KMEM-like devices (e.g. to
> +                * allow Xorg to use DRI).
> +                */
> +       case PRIV_KMEM_WRITE:
> +               if (cred->cr_prison->pr_allow & PR_ALLOW_KMEM_ACCESS)
> +                       return (0);
> +               else
> +                       return (EPERM);
> +
> +               /*
>                  * Allow jailed root to set loginclass.
>                  */
>         case PRIV_PROC_SETLOGINCLASS:
> ---snip---
>
> For 11.1 one little change is needed to get DRM access... see 
> https://svnweb.freebsd.org/changeset/base/320827
>
> The jail then needs to be started with "allow.kmem_access" and 
> appropriate devfs rules (the jail needs rule 15 or 18, you may want to 
> give more or less access, depending on your needs, review with care):
> ---snip---
> [devfsrules_unhide_audio=5]
> add path 'audio*' unhide
> add path 'dsp*' unhide
> add path midistat unhide
> add path 'mixer*' unhide
> add path 'music*' unhide
> add path 'sequencer*' unhide
> add path sndstat unhide
> add path speaker unhide
>
> [devfsrules_unhide_input=7]
> add path 'atkbd*' unhide
> add path 'kbd*' unhide
> add path 'joy*' unhide
> add path 'psm*' unhide
> add path sysmouse unhide
> add path 'ukbd*' unhide
> add path 'ums*' unhide
>
> [devfsrules_unhide_xorg=8]
> add path agpgart unhide
> #add path console unhide
> add path dri unhide
> add path 'dri/card*' unhide
> add path nvidiactl unhide
> add path 'nvidia*' unhide
> add path io unhide
> add path mem unhide
> add path pci unhide
> add path tty unhide
> add path ttyv0 unhide
> add path ttyv1 unhide
> add path ttyv8 unhide
>
> [devfsrules_unhide_kmem=11]
> add path kmem unhide
>
> [devfsrules_unhide_zfs=12]
> add path zfs unhide
>
> #
> # This allows to run a desktop system in a jail.  Think about what you 
> want to
> # achieve before you use this, it opens up the entire machine to 
> access from
> # this jail to any sophisticated program.
> #
> [devfsrules_jail_desktop=15]
> add include $devfsrules_hide_all
> add include $devfsrules_unhide_basic
> add include $devfsrules_unhide_login
> add include $devfsrules_unhide_audio
> add include $devfsrules_unhide_input
> add include $devfsrules_unhide_xorg
> add include $devfsrules_unhide_kmem
>
> [devfsrules_jail_desktop_withzfs=18]
> add include $devfsrules_hide_all
> add include $devfsrules_unhide_basic
> add include $devfsrules_unhide_login
> add include $devfsrules_unhide_audio
> add include $devfsrules_unhide_input
> add include $devfsrules_unhide_xorg
> add include $devfsrules_unhide_kmem
> add include $devfsrules_unhide_zfs
> ---snip---
>
> Bye,
> Alexander.
>

Thanks


Giulio




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4a1a99a5-35ea-19c9-7ac8-77875ac6f71f>