Category Archives: Planet Revolution-Linux

Getting started with LXD – the container lightervisor

Introduction

For the past 6 months, Serge Hallyn, Tycho Andersen, Chuck Short, Ryan Harper and myself have been very busy working on a new container project called LXD.

Ubuntu 15.04, due to be released this Thursday, will contain LXD 0.7 in its repository. This is still the early days and while we’re confident LXD 0.7 is functional and ready for users to experiment, we still have some work to do before it’s ready for critical production use.

LXD logo

So what’s LXD?

LXD is what we call our container “lightervisor”. The core of LXD is a daemon which offers a REST API to drive full system containers just like you’d drive virtual machines.

The LXD daemon runs on every container host and client tools then connect to those to manage those containers or to move or copy them to another LXD.

We provide two such clients:

  • A command line tool called “lxc”
  • An OpenStack Nova plugin called nova-compute-lxd

The former is mostly aimed at small deployments ranging from a single machine (your laptop) to a few dozen hosts. The latter seamlessly integrates inside your OpenStack infrastructure and lets you manage containers exactly like you would virtual machines.

Why LXD?

LXC has been around for about 7 years now, it evolved from a set of very limited tools which would get you something only marginally better than a chroot, all the way to the stable set of tools, stable library and active user and development community that we have today.

Over those years, a lot of extra security features were added to the Linux kernel and LXC grew support for all of them. As we saw the need for people to build their own solution on top of LXC, we’ve developed a public API and a set of bindings. And last year, we’ve put out our first long term support release which has been a great success so far.

That being said, for a while now, we’ve been wanting to do a few big changes:

  • Make LXC secure by default (rather than it being optional).
  • Completely rework the tools to make them simpler and less confusing to newcomers.
  • Rely on container images rather than using “templates” to build them locally.
  • Proper checkpoint/restore support (live migration).

Unfortunately, solving any of those means doing very drastic changes to LXC which would likely break our existing users or at least force them to rethink the way they do things.

Instead, LXD is our opportunity to start fresh. We’re keeping LXC as the great low level container manager that it is. And build LXD on top of it, using LXC’s API to do all the low level work. That achieves the best of both worlds, we keep our low level container manager with its API and bindings but skip using its tools and templates, instead replacing those by the new experience that LXD provides.

How does LXD relate to LXC, Docker, Rocket and other container projects?

LXD is currently based on top of LXC. It uses the stable LXC API to do all the container management behind the scene, adding the REST API on top and providing a much simpler, more consistent user experience.

The focus of LXD is on system containers. That is, a container which runs a clean copy of a Linux distribution or a full appliance. From a design perspective, LXD doesn’t care about what’s running in the container.

That’s very different from Docker or Rocket which are application container managers (as opposed to system container managers) and so focus on distributing apps as containers and so very much care about what runs inside the container.

There is absolutely nothing wrong with using LXD to run a bunch of full containers which then run Docker or Rocket inside of them to run their different applications. So letting LXD manage the host resources for you, applying all the security restrictions to make the container safe and then using whatever application distribution mechanism you want inside.

Getting started with LXD

The simplest way for somebody to try LXD is by using it with its command line tool. This can easily be done on your laptop or desktop machine.

On an Ubuntu 15.04 system (or by using ppa:ubuntu-lxc/lxd-stable on 14.04 or above), you can install LXD with:

sudo apt-get install lxd

Then either logout and login again to get your group membership refreshed, or use:

newgrp lxd

From that point on, you can interact with your newly installed LXD daemon.

The “lxc” command line tool lets you interact with one or multiple LXD daemons. By default it will interact with the local daemon, but you can easily add more of them.

As an easy way to start experimenting with remote servers, you can add our public LXD server at https://images.linuxcontainers.org:8443
That server is an image-only read-only server, so all you can do with it is list images, copy images from it or start containers from it.

You’ll have to do the following to: add the server, list all of its images and then start a container from one of them:

lxc remote add images images.linuxcontainers.org
lxc image list images:
lxc launch images:ubuntu/trusty/i386 ubuntu-32

What the above does is define a new “remote” called “images” which points to images.linuxcontainers.org. Then list all of its images and finally start a local container called “ubuntu-32” from the ubuntu/trusty/i386 image. The image will automatically be cached locally so that future containers are started instantly.

The “<remote name>:” syntax is used throughout the lxc client. When not specified, the default “local” remote is assumed. Should you only care about managing a remote server, the default remote can be changed with “lxc remote set-default”.

Now that you have a running container, you can check its status and IP information with:

lxc list

Or get even more details with:

lxc info ubuntu-32

To get a shell inside the container, or to run any other command that you want, you may do:

lxc exec ubuntu-32 /bin/bash

And you can also directly pull or push files from/to the container with:

lxc file pull ubuntu-32/path/to/file .
lxc file push /path/to/file ubuntu-32/

When done, you can stop or delete your container with one of those:

lxc stop ubuntu-32
lxc delete ubuntu-32

What’s next?

The above should be a reasonably comprehensive guide to how to use LXD on a single system. Of course, that’s not the most interesting thing to do with LXD. All the commands shown above can work against multiple hosts, containers can be remotely created, moved around, copied, …

LXD also supports live migration, snapshots, configuration profiles, device pass-through and more.

I intend to write some more posts to cover those use cases and features as well as highlight some of the work we’re currently busy doing.

LXD is a pretty young but very active project. We’ve had great contributions from existing LXC developers as well as newcomers.

The project is entirely developed in the open at https://github.com/lxc/lxd. We keep track of upcoming features and improvements through the project’s issue tracker, so it’s easy to see what will be coming soon. We also have a set of issues marked “Easy” which are meant for new contributors as easy ways to get to know the LXD code and contribute to the project.

LXD is an Apache2 licensed project, written in Go and which doesn’t require a CLA to contribute to (we do however require the standard DCO Signed-off-by). It can be built with both golang and gccgo and so works on almost all architectures.

Extra resources

More information can be found on the official LXD website:
https://linuxcontainers.org/lxd

The code, issues and pull requests can all be found on Github:
https://github.com/lxc/lxd

And a good overview of the LXD design and its API may be found in our specs:
https://github.com/lxc/lxd/tree/master/specs

Conclusion

LXD is a new and exciting project. It’s an amazing opportunity to think fresh about system containers and provide the best user experience possible, alongside great features and rock solid security.

With 7 releases and close to a thousand commits by 20 contributors, it’s a very active, fast paced project. Lots of things still remain to be implemented before we get to our 1.0 milestone release in early 2016 but looking at what was achieved in just 5 months, I’m confident we’ll have an incredible LXD in another 12 months!

For now, we’d welcome your feedback, so install LXD, play around with it, file bugs and let us know what’s important for you next.

Posted in Canonical voices, LXC, LXD, Planet Revolution-Linux, Planet Ubuntu | Tagged | 40 Comments

State of LXC in Ubuntu 11.04

A while ago I posted about LXC and how to use it on Ubuntu 9.10, I think it’s time to update these instructions to the current state of LXC in Ubuntu 11.04.

As a quick reminder LXC stands for Linux Containers and uses the relatively recent cgroup and namespace features of the Linux kernel to offer something that’s between a chroot and a virtual machine. That’s, basically a chroot but with fine grained resource allocation, its own network stack and its own pid namespace.
LXC is very similar to OpenVZ and Linux-Vserver but doesn’t depend on kernel patches to work.

So here’s now how to get it working on Ubuntu 11.04 in a much easier way than back in Ubuntu 9.10, thanks to all the work done upstream.

To get LXC working on Ubuntu 11.04, you’ll need to do the following:

  • Install a few packages: lxc, debootstrap and bridge-utils
  • Create a bridge interface with masquerading and a local IP address
  • Create a mountpoint for the cgroup filesystem and make sure it’s mounted
  • Write a network configuration file for your container
  • Create your container (the template gets generated with the first container)

To make it even easier, I wrote the following script that you can start as root to do all the above.
It’ll add a “br-lxc” interface using the 192.168.254.0/24 network and configure masquerading.
The cgroup filesystem will be mounted at boot time in /cgroup.
A first container called natty01 will be created and started with IP 192.168.254.2 and default root password “root”.

The script is (I think) well commented and I’ve clearly indicated what’s to be run once (to setup LXC) and what’s to be run for every container you may want to create.
Script can be downloaded from: http://www.stgraber.org/download/lxc-demo.sh.

Once you have a container started, you can start playing with:

  • Attach to a VT: lxc-console –name natty01
  • Get the status: lxc-info –name natty01
  • Get the list of running processes: lxc-ps –name natty01 aux
  • Start/Stop containers: lxc-start/lxc-stop

Have fun!

Posted in Canonical voices, LXC, Planet Revolution-Linux, Planet Ubuntu | 20 Comments

More on WebLive and the Software Center integration

As Michael mentioned on Friday we now have a pretty well working WebLive integration directly in Natty’s software-center.

All you need is qtnx to be installed and an up to date Software Center.
Edubuntu 11.04 will ship with qtnx installed by default. We want to test the feature with all Edubuntu users first to see how it scales and make sure everything is stable before we consider using it for Ubuntu.

Now, quite a few people have been wondering what’s currently available on WebLive.
The complete package list is available here: http://www.stgraber.org/download/weblive-list.html (generated using the WebLive API)

It’s basically all the packages you can find in Edubuntu 10.10, Edubuntu 11.04 and Ubuntu 11.04 as well as the top-30 from the Software Center.
So if you want a package to appear on WebLive, write a review in the Software-Center and rate it.

Edubuntu WebLive

I also would like to remember everyone that you can still get full desktop session by going at: http://www.edubuntu.org/weblive

If you want to point a friend who doesn’t have the new software center to a single-app session, you can use: http://www.edubuntu.org/weblive/app/gedit (by replacing gedit by anything that’s available on WebLive).

Enjoy !

Posted in Edubuntu, Planet Revolution-Linux, Planet Ubuntu, WebLive | 2 Comments

Introducing the WebLive API

After working on it for the last two weeks, I’m proud to finally announce the WebLive API.
As a reminder, WebLive is the name used for the daemon, Drupal plugin and scripts used to run http://www.edubuntu.org/weblive
Since last week, all the code is now available here: https://launchpad.net/weblive

The API is exported over JSON and example code is available in the branch called ltsp-cluster-agent-vmmanager in the client directory. When interfacing with python, it’s recommended to use the “weblive.py” module as it’ll be updated should the JSON API change or be extended in the near future.

Exported functions are:

  • create_user(serverid, username, fullname, password, session)
  • list_everything()
  • list_locales(serverid)
  • list_packages(serverid)
  • list_servers()

The following functions are exported over authenticated XML-RPC (for management):

  • delete_user(serverid, username)
  • set_disabled(serverid,status)
  • list_users(serverid,all=False)

The main weblive instance is available at https://weblive.stgraber.org/weblive/json and is the one used by Edubuntu.

The drupal-weblive branch contains the Drupal module which is now just a client to the JSON API.

The first use of that API after the Drupal module is Natty’s software-center which since last week ships with code to connect to WebLive.
If you use up to date Natty, you’ll need to install “qtnx” from universe and then start: software-center –with-weblive

All the packages available in WebLive will then have a “Test drive” button you can click to test that app remotely from a WebLive server.

WebLive integration in software-center

The software-center integration is experimental and will hopefully be improved by the time Natty is released. As Ubuntu doesn’t ship qtnx by default, WebLive won’t be visible in the default Ubuntu install, though it’ll be for Edubuntu.

Note: It can take up to a minute to connect to a server. There’s currently no user feedback during the connection process unless you watch the terminal from which you started the software center.

The current code requires your username and hostname to be ascii lowercase alpha characters only. I posted instructions as a comment to get the development branch that doesn’t have this restriction.

Posted in Edubuntu, LTSP, Planet Revolution-Linux, Planet Ubuntu, WebLive | Tagged | 12 Comments

Edubuntu’s installer ready for 11.04

After spending the last day or so working on it, I’m proud to announce that Edubuntu finally has its long awaited fine-grained package selection in Ubiquity.

Here’s a screenshot of what will be available in Natty:

Ubiquity fine-grained package selection

With it, users can easily opt out of some of our meta-packages or individual packages.
In the future you will also be able to install extra packages this way (that aren’t installed in the live environment but are present on the DVD).

Other than this installer change, Edubuntu has also been updated to ship with a classic gnome interface by default but offer Unity as an installation option. Unity-2d will also ship by default and we’ll try to have it as fall-back of Unity for these who choose to use Unity as their desktop interface.

Enjoy !

Posted in Edubuntu, Planet Revolution-Linux, Planet Ubuntu | Tagged , | 13 Comments