LXC 1.0: Troubleshooting and debugging [10/10]

This is post 10 out of 10 in the LXC 1.0 blog post series.

Logging

Most LXC commands take two options:

  • -o, –logfile=FILE: Location of the logfile (defaults to stder)
  • -l, –logpriority=LEVEL: Log priority (defaults to ERROR)

The valid log priorities are:

  • FATAL
  • ALERT
  • CRIT
  • ERROR
  • WARN
  • NOTICE
  • INFO
  • DEBUG
  • TRACE

FATAL, ALERT and CRIT are mostly unused at this time, ERROR is pretty common and so are the others except for TRACE. If you want to see all possible log entries, set the log priority to TRACE.

There are also two matching configuration options which you can put in your container’s configuration:

  • lxc.logfile
  • lxc.loglevel

They behave exactly like their command like counterparts. However note that if the command line options are passed, any value set in the configuration will be ignored and instead will be overridden by those passed by the user.

When reporting a bug against LXC, it’s usually a good idea to attach a log of the container’s action with a logpriority of at least DEBUG.

API debugging

When debugging a problem using the API it’s often a good idea to try and re-implement the failing bit of code in C using liblxc directly, that helps get the binding out of the way and usually leads to cleaner stack traces and easier bug reports.

It’s also useful to set lxc.loglevel to DEBUG using set_config_item on your container so you can get a log of what LXC is doing.

Testing

Before digging to deep into an issue with the code you are working on, it’s usually a good idea to make sure that LXC itself is behaving as it should on your machine.

To check that, run “lxc-checkconfig” and look for any missing kernel feature, if all looks good, then install (or build) the tests. In Ubuntu, those are shipped in a separate “lxc-tests” package. Most of those tests are expecting to be run on an Ubuntu system (patches welcome…) but should do fine on any distro that’s compatible with the lxc-ubuntu template.

Run each of the lxc-test-* binaries as root and note any failure. Note that it’s possible that they leave some cruft behind on failure, if so, please cleanup any of those leftover containers before processing to the next test as unfortunately that cruft may cause failure by itself.

Reporting bugs

The primary LXC bug tracker is available at: https://github.com/lxc/lxc/issues

You may also report bugs directly through the distributions (though it’s often preferred to still file an upstream bug and then link the two), for example for Ubuntu, LXC bugs are tracked here: https://bugs.launchpad.net/ubuntu/+source/lxc

If you’ve already done some work tracking down the bug, you may also directly contact us on our mailing-list (see below).

Sending patches

We always welcome contributions and are very happy to have such an active development community around LXC (Over 60 people contributed to LXC 1.0). We don’t have many rules governing contributions, we just ask that your contributions be properly licensed and that you own the copyright on the code you are sending us (and indicate so by putting a Signed-off-by line in your commit).

As for the licensing, anything which ends up in the library (liblxc) or its bindings must be LGPLv2.1+ or compatible with it and not adding any additional restriction. Standalone binaries and scripts can either be LGLPv2.1+ (the project default) or GPLv2. If unsure, LGPLv2.1+ is usually a safe bet for any new file in LXC.

Patches may be sent using two different ways:

  • Inline to the lxc-devel@lists.linuxcontainers.org (using git send-email or similar)
  • Using a pull request on github (we will then grab the .patch URL and treat it as if they were e-mails sent to our list)

Getting in touch with us

The primary way of contacting the upstream LXC team is through our mailing-lists. We have two, one for LXC development and one for LXC users questions:

For more real-time discussion, you can also find a lot of LXC users and most of the developers in #lxcontainers on irc.freenode.net.

Final notes

So this is my final blog post before LXC 1.0 is finally released. We’re currently at rc3 with an rc4 coming a bit later today and a final release scheduled for tomorrow evening or Thursday morning.

I hope you have enjoyed this blog post series and that it’ll be a useful reference for people deploying LXC 1.0.

About Stéphane Graber

Project leader of Linux Containers, Linux hacker, Ubuntu core developer, conference organizer and speaker.
This entry was posted in Canonical voices, LXC, Planet Ubuntu and tagged . Bookmark the permalink.

10 Responses to LXC 1.0: Troubleshooting and debugging [10/10]

  1. Elan Ruusamäe says:

    (defaults to stder) looks like a typo 🙂

    while at it, the C API documentation said:
    if (!c->createl(c, “download”, NULL, NULL, LXC_CREATE_QUIET,

    shouldn’t it be c->create() ?

    1. No. There is a createl function. As the documentation says: is a list variant of create (whatever it means).

  2. Peter says:

    I would like to set up socket-activated services inside LXC using systemd. Are there any guidelines on how to set up systemd to run properly inside a container? Should I just wait until 14.04 is released and hope some community systemd support is available?

  3. raul says:

    Thanks for the series of articles. The wheezy repos have an older version so I compiled lxc 1.0.1 on a stock debian wheezy 64 bit install and it’s worked without issues so far.

    But a couple of commands like lxc-ls –fancy and lxc-device seem to be missing. I learnt you need to compile this with a –enable-python flag, and when I do this I get a ‘configure: error: You must install python3-dev’. The thing is Python3-dev is installed. Is this an issue with the lxc configure script or the wheezy dev environment.

  4. raul says:

    Sorry for replying to myself but for Debian wheezy users if you are compiling LXC from source (version 1.0 and 1.0.1) you need to compile with some extra python flags if you need the additional python functionality.

    If you don’t need the python functionality compiling LXC normally with ./configure works without issue. To clarify LXC doesn’t need the python bindings to function properly. Its is fully functional without it.

    The extra functionality brought by python is basically ‘lxc-ls –fancy’ option now works and you also get an additional tool ‘lxc-devices’. You need to install python3-dev package and compile LXC with ‘./configure –enable-python’. Normally this should work but there may be an issue with the Wheezy python dev environment or the LXC script so it fails complaining about a missing python3-dev package, inspite of it being installed.

    What worked for me was this – ‘./configure –with-distro=debian –enable-python PYTHONDEV_CFLAGS=-I/usr/include/python-3.2mu PYTHONDEV_LIBS=-l/usr/include/python-3.2mu/’

    It should compile and install fine with this. lxc-ls may give an error about a missing lxc module and you need to ensure ‘/usr/local/lib/python3’ is in your python path or symlink it to ‘/usr/lib/python3/’

  5. Joachim Nielandt says:

    Hello everyone,

    I have recently configured an Ubuntu install on a VPS, with BTRFS as the underlying filesystem. I’m using unprivileged containers, all running under the same host user ‘lxc’. They start, I seem to be able to snapshot them, the BTRFS subvolumes get created correctly … however, everytime I do something filesystem related using the lxc command I get a warning/error (?) about uids:

    lxc@xxx.xxx.xxx.xxx:~$ lxc-snapshot -n mail -c ~/snapcomment
    newgidmap: gid range [165536-165537) -> [331072-331073) not allowed
    error mapping child
    setgid: Invalid argument

    I have put some extra info on stackoverflow, including the uids I used and some tests I did: http://stackoverflow.com/questions/36485103/lxc-btrfs-snapshot-of-unprivileged-container-uid-mapping-error

    Hoping to get to the bottom of this, and also hoping this isn’t some sort of crucial error :).
    Thanks!

  6. Kane says:

    Hello,

    I just have finished to read this serie and learned a lot, thanks for your time writing all of this!
    That made me wonder how far is LXC 5.x from LXC 1.0? I know that Incus is the new thing now and even before that LXD took the place of it’s wrapper, but I really like using it in it’s original form. I havent searched your whole website yet, but did you made some other articles about it’s evolution? What have changed and not, if there are improved or recommended way to use it now?

    Thanks in advance for your time and answser!
    Kane.

    1. It still works in much the same way.
      Some config keys were renamed with LXC 2.0 and 3.0 and some extra features like seccomp interception and the like were added in LXC 4.0 and 5.0, but that’s about it.

      The general behavior, commands and structure hasn’t really changed, we’ve mostly spent our time working at the higher level with Incus.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.