LXD 2.0: LXD in LXD [8/12]

This is the eighth blog post in this series about LXD 2.0.

LXD logo

Introduction

In the previous post I covered how to run Docker inside LXD which is a good way to get access to the portfolio of application provided by Docker while running in the safety of the LXD environment.

One use case I mentioned was offering a LXD container to your users and then have them use their container to run Docker. Well, what if they themselves want to run other Linux distributions inside their container using LXD, or even allow another group of people to have access to a Linux system by running a container for them?

Turns out, LXD makes it very simple to allow your users to run nested containers.

Nesting LXD

The most simple case can be shown by using an Ubuntu 16.04 image. Ubuntu 16.04 cloud images come with LXD pre-installed. The daemon itself isn’t running as it’s socket-activated so it doesn’t use any resources until you actually talk to it.

So lets start an Ubuntu 16.04 container with nesting enabled:

lxc launch ubuntu-daily:16.04 c1 -c security.nesting=true

You can also set the security.nesting key on an existing container with:

lxc config set <container name> security.nesting true

Or for all containers using a particular profile with:

lxc profile set <profile name> security.nesting true

With that container started, you can now get a shell inside it, configure LXD and spawn a container:

stgraber@dakara:~$ lxc launch ubuntu-daily:16.04 c1 -c security.nesting=true
Creating c1
Starting c1

stgraber@dakara:~$ lxc exec c1 bash
root@c1:~# lxd init
Name of the storage backend to use (dir or zfs): dir

We detected that you are running inside an unprivileged container.
This means that unless you manually configured your host otherwise,
you will not have enough uid and gid to allocate to your containers.

LXD can re-use your container's own allocation to avoid the problem.
Doing so makes your nested containers slightly less safe as they could
in theory attack their parent container and gain more privileges than
they otherwise would.

Would you like to have your containers share their parent's allocation (yes/no)? yes
Would you like LXD to be available over the network (yes/no)? no
Do you want to configure the LXD bridge (yes/no)? yes
Warning: Stopping lxd.service, but it can still be activated by:
 lxd.socket
LXD has been successfully configured.

root@c1:~# lxc launch ubuntu:14.04 trusty
Generating a client certificate. This may take a minute...
If this is your first time using LXD, you should also run: sudo lxd init

Creating trusty
Retrieving image: 100%
Starting trusty

root@c1:~# lxc list
+--------+---------+-----------------------+----------------------------------------------+------------+-----------+
|  NAME  |  STATE  |         IPV4          |                     IPV6                     |    TYPE    | SNAPSHOTS |
+--------+---------+-----------------------+----------------------------------------------+------------+-----------+
| trusty | RUNNING | 10.153.141.124 (eth0) | fd7:f15d:d1d6:da14:216:3eff:fef1:4002 (eth0) | PERSISTENT | 0         |
+--------+---------+-----------------------+----------------------------------------------+------------+-----------+
root@c1:~#

It really is that simple!

The online demo server

As this post is pretty short, I figured I would spend a bit of time to talk about the demo server we’re running. We also just reached the 10000 sessions mark earlier today!

That server is basically just a normal LXD running inside a pretty beefy virtual machine with a tiny daemon implementing the REST API used by our website.

When you accept the terms of service, a new LXD container is created for you with security.nesting enabled as we saw above. You are then attached to that container as you would when using “lxc exec” except that we’re doing it using websockets and javascript.

The containers you then create inside this environment are all nested LXD containers.
You can then nest even further in there if you want to.

We are using the whole range of LXD resource limitations to prevent one user’s actions from impacting the others and pretty closely monitor the server for any sign of abuse.

If you want to run your own similar server, you can grab the code for our website and the daemon with:

git clone https://github.com/lxc/linuxcontainers.org
git clone https://github.com/lxc/lxd-demo-server

Extra information

The main LXD website is at: https://linuxcontainers.org/lxd
Development happens on Github at: https://github.com/lxc/lxd
Mailing-list support happens on: https://lists.linuxcontainers.org
IRC support happens in: #lxcontainers on irc.freenode.net
Try LXD online: https://linuxcontainers.org/lxd/try-it

About Stéphane Graber

Project leader of Linux Containers, Linux hacker, Ubuntu core developer, conference organizer and speaker.
This entry was posted in Canonical voices, LXD, Planet Ubuntu and tagged . Bookmark the permalink.

11 Responses to LXD 2.0: LXD in LXD [8/12]

  1. Thomas says:

    As I understand only directory (dir) is supported for the moment for nested containers.

    Installing zfsutils-linux and trying to use zfs is not possible because lack of /dev/zfs.
    For the nested containers, when is it expected to support zfs as storage backend?

    1. That’s correct, only “dir” works right now for nested containers. If /var/lib/lxd happens to be on btrfs and you passed the right mount option to btrfs on the host, then btrfs will be used too (that’s what we do on the demo server to minimize disk usage).

      As for zfs, right now their access control is entirely based on access to /dev/zfs. Last I talked with zfs upstream, they did mention planned work to support zfs inside containers with proper access delegation, but I have no idea as to the timeline for this.

      1. Stephen Warren says:

        What is “the right mount option to btrfs on the host”?

        1. user_subvol_rm_allowed

      2. Eric says:

        Looking forward to access delegation, so our containers can be tasked with dataset management instead of requiring to be root on the host

  2. Stephen Warren says:

    I noticed the article only shows re-using the outer container’s uid range, but doesn’t show nested uid mapping. Here are some commands that worked for me to set that up:

    swarren@cam:~$ cat /etc/subuid
    lxd:100000:131072
    root:100000:131072
    swarren:300000:199608

    swarren@cam:~$ cat /etc/subgid
    lxd:100000:131072
    root:100000:131072
    swarren:300000:199608

    # I assume this is needed to pick up the changes to /etc/sub[ug]id
    swarren@cam:~$ sudo service lxd restart

    swarren@cam:~$ lxc launch -c security.nesting=true ubuntu:16.04 uboot-jenkins
    Creating uboot-jenkins
    Starting uboot-jenkins

    swarren@cam:~$ lxc exec uboot-jenkins /bin/bash

    root@uboot-jenkins:~# cat > /etc/subuid
    lxd:65536:65536
    root:65536:65536
    ubuntu:131072:65536
    ^D

    root@uboot-jenkins:~# cat > /etc/subgid
    lxd:65536:65536
    root:65536:65536
    ubuntu:131072:65536
    ^D

    root@uboot-jenkins:~# lxd init
    Name of the storage backend to use (dir or zfs): dir

    We detected that you are running inside an unprivileged container.
    This means that unless you manually configured your host otherwise,
    you will not have enough uid and gid to allocate to your containers.

    LXD can re-use your container’s own allocation to avoid the problem.
    Doing so makes your nested containers slightly less safe as they could
    in theory attack their parent container and gain more privileges than
    they otherwise would.

    Would you like to have your containers share their parent’s allocation (yes/no)? no
    Would you like LXD to be available over the network (yes/no)? no
    Do you want to configure the LXD bridge (yes/no)? yes
    Warning: Stopping lxd.service, but it can still be activated by:
    lxd.socket
    LXD has been successfully configured.

    root@uboot-jenkins:~# lxc remote add outer 192.168.63.3
    Generating a client certificate. This may take a minute…
    If this is your first time using LXD, you should also run: sudo lxd init

    Certificate fingerprint: 191acad76df9d46cc8680997b1c84f36d61cce9fdfc3054cf2beb534fb3e9454
    ok (y/n)? y
    Admin password for outer:
    Client certificate stored at server: outer

    root@uboot-jenkins:~# lxc launch outer:f4c4c60a6b75 nested
    Creating nested
    Retrieving image: 100%
    Starting nested

    root@uboot-jenkins:~# lxc exec nested — bash

    root@nested:~#

    swarren@cam:~$ lxc list
    +—————+———+——————————–+——+————+———–+
    | NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
    +—————+———+——————————–+——+————+———–+
    | uboot-jenkins | RUNNING | 10.149.93.35 (eth0) | | PERSISTENT | 0 |
    | | | 10.142.94.1 (lxdbr0) | | | |
    +—————+———+——————————–+——+————+———–+

    swarren@cam:~$ lxc exec uboot-jenkins — lxc list
    +——–+———+———————+——+————+———–+
    | NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
    +——–+———+———————+——+————+———–+
    | nested | RUNNING | 10.142.94.15 (eth0) | | PERSISTENT | 0 |
    +——–+———+———————+——+————+———–+

    1. Thanks, I’m sure that will be useful to some people!

  3. Michael Hudson-Doyle says:

    Typo alert:

    git clone github.com/lxc/linuxcontainers.org
    git clone github.com/lxc/lxd-demo-server

    You forgot the https://s there, too much ‘go get’? 🙂

    1. I guess so 🙂 Fixed!

  4. Muhfi Asbin Sagala says:

    Is it possible to limit the number of container creation in nested lxd?

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.