Fixing repo init to check out Chromium OS code

Building Chromium OS is a little tricky. Although the instructions from the project page are pretty detailed, one bump I ran into was this error when I put in the "repo init ..." command:

fatal: Cannot get https://chromium.googlesource.com/a/external/repo.git/clone.bundle
fatal: HTTP error 401

Turns out that there's an issue with in the repo init code that makes it fail if there's anything wrong with your ~/.netrc file. My system didn't even have a .netrc file, which made it hard to know how to fix that problem.

OpenVPN Update: Fixed!

I finally had time to really do some more troubleshooting on the OpenVPN problem I recently posted about.  After trying some of the same old things to get the server to be able to access the Internet and have the VPN service running at the same time, I decided it was time to purge the installation and start fresh:

sudo apt-get purge openvpn

No fun.  It's always depressing when a configuration problem gets to the point where the only thing left you can think to try is to start over from scratch.  I did have the good sense to back up all the server's keys and configuration files, which in the long run did end up being useful.

As I was once again going through the OpenVPN tutorial provided by Ubuntu and scanning through the help comments in /etc/openvpn/server.conf, when I found the following:

Use "dev tap0" if you are ethernet bridging and have precreated a tap0 virtual interface and bridged it with your ethernet interface.

For a while, I mistakenly thought that this might be the answer I was looking for, since it seemed like that was what I was doing, but only because I didn't understand everything that was going on under the hood as well as I do now.  You see, when the tutorial has you add the line:

up "/etc/openvpn/up.sh br0 eth1"

this isn't exactly what is called to bring up the bridged interfaces.  Taking another look at the up.sh script, there are three network interface devices referenced, even though only two are explicitly given in the configuration line above.  OpenVPN adds the necessary tap0 device to the system, and then adds its interface name as a parameter to the end of the string you specify.

A minimum of three interfaces are needed to create a bridge of the sort that OpenVPN needs: 1) the physical interface with access to the "real world" through which the VPN traffic will initially come in, 2) the destination interface which is actually handled by OpenVPN, and 3) the bridging interface that connects the first two at the kernel level.

As I was figuring out all this, I thought to check the man page of brctl since I knew that the up.sh script called by OpenVPN as it starts up uses brctl to do the kernel-level changes to bridge eth1 and tap0.  Once I really started to understand how the various options worked, I tried refreshing the bridged interface by calling brctl dellbr br0 and then brctl addbr br0, followed by restarting the openvpn service.

Doing this had positive results, but none of them persisted through a reboot of the machine.  Finally, after trying several different things, I came up with hard-coding a refresh of the bridged interface in the up.sh script.  This, plus changing the network configuration of eth0 from static to dhcp seems to have done the trick.  The machine has access to the Internet, and clients can connect just as they could before.  I even copied over the old configuration, certificate, and key files, which prevented me from having to re-issue all the client keys and re-distributing the server's certificate.

The new contents of my up.sh script are below:

#!/bin/sh

BR=$1
ETHDEV=$2
TAPDEV=$3

# Reset the bridge device first
/sbin/ip link set "$BR" down
/sbin/brctl delbr $BR
/sbin/brctl addbr $BR
/sbin/brctl addif $BR $ETHDEV

/sbin/ip link set "$TAPDEV" up
/sbin/ip link set "$ETHDEV" promisc on
/sbin/brctl addif $BR $TAPDEV

Troubles with OpenVPN in Ubuntu 12.04

Update: Check out my solution to the problem described below.

Last year at work I set up a VPN using OpenVPN in Ubuntu Server 11.10.  There were a number of issues with supporting the specific topology of the local network I was trying to make accessible that made it tricky to get traffic from a connected client properly routed in and back.  To be upfront, I'm not a Linux guru.  I get by with the principles I've learned in the classroom, the tips I've been given from more enthusiastic friends and family, and from what I can glean from forums, help pages, and manual pages, and generally I do alright.

Back to my story.  So last year I honestly don't know what exactly it was that got things to work, which is bad, I know.  At one point I had two other guys helping me while we looked at the traffic going between a client and the VPN server with Wireshark on both ends.  We could see ping packets making it successfully to the server and we could see the responses generated by the server, but no replies were making it back to the client.  Weird.  In that case, I finally figured out that in the different configurations I had tried, at one point I had commented out the bridged interface, br0, from /etc/network/interfaces.  I un-commented those lines, things started working, and I never touched it for fear of breaking things.  Restarting the system kept things going, so it must have been some sort of persistent configuration.

Fast forward to a couple weeks ago, when my overconfidence and excitement at the release of Ubuntu 12.04 LTS got the better of me, overpowering my previous fear of breaking the VPN.  I think I said to myself something like the following. "Well, all the configuration files will carry over, so things can only get better by running 'sudo do-release-upgrade', right?  After all, this is a Long-Term Support release, so it's nice and stable."

Wrong.  Things broke.  So far, I've overcome one hurdle, but things still aren't working, so I have no idea how many other obstacles are in my way to being back up and running.  Here is the evidence I've gathered so far:

• In /var/log/syslog, there was an error about the security level setting for running external scripts.  Adding the line "script-security 3" before calling any scripts made that go away and allowed OpenVPN to successfully start up.  The script I'm running is up.sh as described in the OpenVPN tutorial on the Ubuntu help pages, under the section "Prepare server config for bridging".
• Initially after the release upgrade, the primary connection, eth0, no longer had any access to the outside world.  I could ping the router, but nothing outside the local network.  I found that setting the tap0 device to down and changing eth0 from static to dhcp seemed to fix this.  Unfortunately, later something reverted back and the server no longer has outside access again.  I tried the tricks I used before and even tried stopping OpenVPN with '/etc/init.d/openvpn stop' with no success, even after restarting networking and rebooting the server.
• Even when the server could access external addresses things didn't seem to be routing packets correctly.  I found that by disabling the br0, eth1, or tap0 (I can't remember...), web pages loaded faster and everything seemed to work properly.  Running 'route -vn' was what gave me the idea in the first place because there multiple devices specified as the outgoing for the same destination range.  Unfortunately I didn't memorize the details and things aren't at the point they were previously to try and figure it out.

I'm sure there are some specifics of how OpenVPN works that would help me troubleshoot this, but my struggle with problems like this (which seem to present themselves more often in Linux because I'm doing more advanced things) is always trying to balance understanding the details of whatever I'm working with and just getting the dumb thing to work because I need it done last week or last month...

Hopefully I'll get to the bottom of all this and I'll be able to report my findings soon.  In the mean time, take a lesson from my book, and be a little more cautious than I was with updating to the latest release.