Category Archives: IPv6

Dual-stack 3G connections

A few months ago, I received two test SIM cards for Orange Poland’s new IPv6 network.

The interesting thing about this network is that it’s running IPv6 in a fairly unusual configuration and it was interesting to see how to get that work on Ubuntu.

This network uses two separate APNs, one for IPv4 (internet) and one for IPv6 (internetipv6).
Using two separate APNs is certainly easier on the carrier’s infrastructure side as they can get IPv6 online without actually changing anything on the IPv4 equipement, however that means that any client wanting to use both protocols at once needs to use multiple PDP contexts.

I’m now going to detail how to manually configure ppp to connect to such a network:

connect "/usr/sbin/chat -e -f /etc/ppp/peers/orange-connect"


'' \nAT
OK 'AT+CGDCONT=1,"IP","internet"'
OK 'AT+CGDCONT=2,"IPV6","internetipv6"'
OK ATD*99#

Then all that’s needed is a good old:

pon orange

And a few seconds later, I’m getting the following on ppp0:

ppp0      Link encap:Point-to-Point Protocol  
          inet addr:  P-t-P:  Mask:
          inet6 addr: 2a00:f40:2100:ac9:8c1e:da60:93e2:c234/64 Scope:Global
          inet6 addr: 2a00:f40:2100:ac9::1/64 Scope:Global
          inet6 addr: fe80::1/10 Scope:Link
          RX packets:13 errors:0 dropped:0 overruns:0 frame:0
          TX packets:23 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:3 
          RX bytes:354 (354.0 B)  TX bytes:767 (767.0 B

This config should work for any mobile network using a similar setup (likely to become more and more popular as the various RIRs are running out of IPv4).

Sadly ModemManager/NetworkManager don’t support mutliple PDP contexts yet, though it’s being discussed upstream, so we can hope to see something land soon.

Apparently multiple PDP contexts support is also dependant on hardware. In my case, I’ve been using an “old” Nokia E51 over USB as I didn’t have any luck getting that to work with an Android phone. My Nokia N900 also worked but required a custom kernel to be installed first to properly handle IPv6.

Posted in Canonical voices, IPv6, Planet Ubuntu | 3 Comments

Networking in Ubuntu 12.04 LTS

One of my focus for this cycle is to get Ubuntu’s support for complex networking working in a predictable way. The idea was to review exactly what’s happening at boot time, get a list of possible scenario that are used on servers in corporate environment and make sure these always work.


Bonding basically means aggregating multiple physical link into one virtual link for high availability and load balancing. There are different ways of setting up such a link though the industry standard is 802.3ad (LACP – Link Aggregation Control Protocol). In that mode your server will negotiate with your switch to establish an aggregate link, then send monitoring packets to detect failure. LACP also does load balancing (MAC, IP and protocol based depending on hardware support).

One problem we had since at least Ubuntu 10.04 LTS is that Ubuntu’s boot sequence is event based, including bringing up network interfaces. The good old “ifup -a” is only done at the end of the boot sequence to try and fix anything that wasn’t brought up through events.

Unfortunately that meant that if your server takes a long time to detect the hardware, your bond would be initialised before the network cards have been detected, giving you a bond0 without a MAC address, making DHCP queries fail in pretty weird ways and making bridging or tagging fail with “Operation not permitted”.
As that all depends on hardware detection timing, it was racy, giving you random results at boot time.

Thankfully that should now be all fixed in 12.04, the new ifenslave-2.6 I uploaded a few weeks ago now initialises the bond whenever the first slave appears. If no slave appeared by the time we get to the catch-all “ifup -a”, it’ll simply wait for up to an additional minute for a slave to appear before giving up and continuing the boot sequence.
To avoid another common race condition where a bridge is brought up with a bond as one of its members before the bond is ready, ifenslave will now detect a bond is part of a bridge and add it only once ready.


Another pretty common thing on corporate networks is the use of VLANs (802.1q), letting you create up to 4096 virtual networks on one link.
In the past, Ubuntu would rely on the catch all “ifup -a” to create any required vlan interface, once again, that’s a problem when an interface that depends on that vlan interface is initialised before the vlan interface is created.

To fix that, Ubuntu 12.04’s vlan package now ships with a udev rule that triggers the creation of the vlan interface whenever its parent interface is created.


Bridging on Linux can be seen as creating a virtual switch on your system (including STP support).

Bridges have been working pretty well for a while on Ubuntu as we’ve been shipping a udev rule similar to the one for vlans for a few releases already. Members are simply added to the bridge as they appear on the system. The changes to ifenslave and the vlan package make sure that even bond interfaces with VLANs get properly added to bridges.

Complex network configuration example

My current test setup for networking on Ubuntu 12.04 is actually something I’ve been using on my network for years.

As you may know, I’m also working on LXC (Linux Containers), so my servers usually run somewhere between 15 and 80 containers, each of these container has a virtual ethernet interface that’s bridged.
I have one bridge per network zone, each of these network zone being a separate VLAN. These VLANs are created on top of a two gigabit link bond.

At boot time, the following happens (roughly):

  1. One of the two network interfaces appear
  2. The bond is initialised and the first interface is enslaved
  3. This triggers the creation of all the VLAN interfaces
  4. Creating the VLAN interfaces triggers the creation of all the bridges
  5. All the VLAN interfaces are added to their respective bridge
  6. The other network interface appear and gets added to the bond

My /etc/network/interfaces can be found here:

This contains the very strict minimum needed for LACP to work. One thing worth noting is that the two physical interfaces are listed before bond0, this is to ensure that even if the events don’t work and we have to rely on the fallback “ifup -a”, the interfaces will be initialised in the right order avoiding the 60s delay.

Please note that this example will only reliably work with Ubuntu Precise (to become 12.04 LTS). It’s still a correct configuration for previous releases but race conditions may give you a random result.

I’ll be trying to push these changes to Ubuntu 11.10 as they are pretty easy to backport there, however it’d be much harder and very likely dangerous to backport these to even older releases.
For these, the only recommendation I can give is to add some “pre-up sleep 5” or similar to your bridges and vlan interfaces to make sure whatever interface they depend on exists and is ready by the time the “ifup -a” call is reached.


Another interesting topic for 12.04 is IPv6, as a release that’ll be supported for 5 years on both servers and desktops, it’s important to get IPv6 right.

Ubuntu 12.04 LTS will be the first Ubuntu release shipping with IPv6 private extensions turned on by default. Ubuntu 11.10 already brought most of what’s needed for IPv6 support on the desktop and in the installer, supporting SLAAC (stateless autoconfiguration), stateless DHCPv6 and stateful DHCPv6.

Once we get a new ifupdown in Ubuntu Precise, we’ll have full support for IPv6 also for people that aren’t using Network Manager (mostly servers) which should at this point give us support for any IPv6 setup you may find.

The userspace has been working pretty well with IPv6 for years. I recently made my whole network dual-stack and now have all my servers and services defaulting to IPv6 for a total of 40% of my network traffic (roughly 1.5TB a month of IPv6 traffic). The only user space related problem I noticed is the lack of IPv6 support in Nagios’ nrpe plugin, meaning I can’t start converting servers to single stack IPv6 as I’d loose monitoring …

I also wrote a python script using pypcap to give me the percentage of ipv6 and ipv4 traffic going through a given interface, the script can be found here: (start with: python eth0)

What now ?

At this point, I think Ubuntu Precise is pretty much ready as far as networking is concerned. The only remaining change is the new ifupdown and the small installer change that comes with it for DHCPv6 support.

If you have a spare machine or VM, now is the time to grab a recent build of Ubuntu Precise and make sure whatever network configuration you use in production works reliably for you and that any hack you needed in the past can now be dropped.
If your configuration doesn’t work at all or fails randomly, please file a bug so we can have a look at your configuration and fix any bug (or documentation).

Posted in Canonical voices, IPv6, LXC, Planet Ubuntu | 44 Comments

State of IPv6 in Ubuntu Oneiric

One of my focus for the Oneiric development cycle is to make sure we get proper support of IPv6 both at install time and during regular use of the system.

To achieve this, I started working on the list of all possible scenarios I could think of with all possible combinations of IPv4 and IPv6. Then checked how well these were supported on Ubuntu.

Since Ubuntu 11.04, we now have a DHCPv6 aware DHCP client but that’s not working as well as it should because Network Manager didn’t do IPv6 by default back then and because the DHCP client configuration for IPv6 wasn’t too clear (dhclient wasn’t requesting any attribute).

Most of these issues are now fixed in Oneiric with Mathieu Trudel-Lapierre‘s great work on updating Network Manager in Oneiric to have IPv6 on by default and make sure people don’t have to wait for IPv6 to timeout to get their IPv4 connectivity.

The result is something like you can see below, on a network that has both DHCPv4 and stateless DHCPv6:

The use cases that are currently tested are:

  • Single stack: SLAAC IPv6-only network
  • Single stack: Stateful DHCPv6 IPv6-only network
  • Single stack: Stateless DHCPv6 IPv6-only network
  • Single stack: DHCPv4 IPv4-only network
  • Dual stack: SLAAC + DHCPv4 network
  • Dual stack: Stateful DHCPv6 + DHCPv4 network
  • Dual stack: Stateless DHCPv6 + DHCPv4 network

For these interested, you can look at the following files to get some example DHCPv4, DHCPv6 and RADVD configuration:

It’s worth noting that you have to start a separate dhcpd server for IPv6 (with the -6 flag) as dhcpd can’t answer both dhcpv4 and dhcpv6 at the same time. You need two separate daemons with two separate configuration files.

As you can see from the files above, I have a pretty complete IPv6 test setup, running on libvirt. I’m now working on automating all of this so we can get some easy regression testing of IPv6 support on Ubuntu.

During our sprint last month in Dublin, Colin Watson also got netcfg to support IPv6 thereby making debian-installer working with IPv6. The missing piece now is ifupdown support of DHCPv6 (so you can configure DHCPv6 in /etc/network/interfaces) and we should then have Ubuntu install on IPv6 from the alternate/server disks.

IPv6 support is starting to look really good for Oneiric and should be awesome for the next LTS.
If you’re already running Oneiric on an IPv6 capable network, please test the new Network Manager and if you encounter any problem, please file bugs or poke me so I can add some more tests to my list!

Posted in Canonical voices, Conferences, IPv6, Planet Ubuntu | 20 Comments