Goodbye Blogger!

Blogger has been good to me over the years. I set up my first blog on Blogger before I had any other digital real estate to speak of. But ever since setting up mikemabey.com, I've been looking for a way to migrate off of Blogger and have everything all together on the same site. I recently completed that objective, and in this post I'll share how I did it and what obstacles I encountered along the way.

Naturally, this is the final post I'll share on Blogger. Future blog entries will be available on the blog section of my website.

Using git-stash without losing staged changes

When I learned about git-stash, my productivity with git from the command line went way up. But it wasn't until recently that I figured out how to properly handle the situation where I had already staged changes but needed to stash everything to do something without .

The problem comes from the fact that stash lumps all the changes to the tracked files together, without separating all the things you've added to the index. If you've done a patch add with git add -p, this can mean a lot of frustration at having to pick through all the changes again once you do git stash pop.

To avoid this frustration, do the following and you'll be able to keep your staged changes over the course of using stash.

1. First, do a stash of only the things you haven't yet staged:

git stash save --keep-index "Un-staged changes"

At this point, if you run git status, you should see all your changes that have been staged still there, but none of the changes NOT staged for commit (except of course for any untracked files).

2. Next, stash the staged changes as you normally would:

git stash save "Staged"

3. Now is when you want to do any work you need to while the other changes are stashed. NOTE: If you need to make changes that are based on a file in the "Un-staged changes" stash, you will of course need to pop that stash specifically with git stash pop stash@{1}. But, in order to get the staged changes back, you'll need to do another git stash save to do things in the correct order.

4. To get your changes back, you're going to pop them off the stash in the reverse order. If you haven't stashed any other changes, you can do a simple pop:

git stash pop

Otherwise, you should run:

git stash list

and find the correct name of the stash to pop, e.g. stash@{1}.

5. All the changes you just popped aren't staged at this point but should be, so go ahead and add all of them to the index with git add file1 file2 ...

6. Finally, run git stash pop one more time to recover the un-staged changes you saved in step 1 and you'll have everything back to the way it was at the beginning.

How to check out an old version of Chromium OS

You gotta love the combination of open source projects and git. Getting up and running with a project like Chromium OS is pretty straightforward with the guide available on the Chromium Wiki. But what if, for some reason, you're interested in a version other than the most recent? How do you dig through the combinations of repositories to get it? I recently had to figure this out for a project I'm working on, so I thought I'd pass it along.

First off, you should know that version numbers between Chrome, Chrome OS, Chromium, and Chromium OS do have a certain correlation, but isn't guaranteed to be the same across the repositories for all of them.

Once you've figured out which version of you want, go to https://chromium.googlesource.com/chromiumos/manifest-versions/+/master/paladin/buildspecs/, click on the version number in the list, then select a build number from the next page.

What you see next is a long XML file that tells repo (Google's wrapper around git) what versions of all the Chromium repositories correspond to that build number. Scroll all the way down to the bottom of the page and click on the "txt" link to download it. This will be a Base64-encoded version of the XML file you were just looking at.

Next, decode the text file with a command like this:

base64 -d [manifest_version].txt > [manifest_version].xml

This assumes you're working on a *nix system. A quick Google search will help Windows users. Be sure to replace [manifest_version] as appropriate.

Assuming you've checked out the Chromium OS repositories before using the repo command, run the sync command using the manifest you just downloaded with:

repo sync -m /path/to/[manifest_version].xml

If you haven't checked out the repositories before, follow the instructions on the guide listed above, then replace the repo sync command with the one above. Keep in mind that the sync process takes a VERY long time.

At this point, if you were to try and enter the SDK with the usual cros_sdk and do any commands, they would likely fail because it is too new. Get the appropriate version with:

cros_sdk --replace

This command will also take a long time to complete.

Now you can enter the SDK with cros_sdk. Now you should rebuild all the packages with:

build_packages --nowithdebug --board=${BOARD}

This command will also take a long time to complete.

Finally you can build an image with the usual command (adding any extra flags you need):

build_image --board=${BOARD}

That's it! Good luck working with Chromium OS!

Getting "n-dimensional" to look right in LaTeX

LaTeX has some interesting rules for dashes and hyphenation that are hard to get right in certain scenarios. For example, if you use the term "n-dimensional" and it happens to fall near the end of a line, you'll end up with something like this:

text text text text text n-
dimensional

To get this right, try inserting this snippet in your document's preamble:

\usepackage{amsmath}
\renewcommand{\ndimensional}[1]{#1\nobreakdash\discretionary{-}{-}{-}\hspace{0pt}dimensional}

Then, replace "n-dimensional" in the document with the following:

\ndimensional{n}

or, to get the math environment look to the "n", do this:

\ndimensional{$n$}

What's going on in the snippet?

The #1 refers to whatever you added as a parameter to the command, which in the example above is "n".

The \nobreakdash needs to be combined with a dash character. That means that in other situations you may just need to do this:

\renewcommand{\ndimensional}[1]{#1\nobreakdash-\hspace{0pt}dimensional}

While the output will be fine for many use cases, the previous snippet with the \discretionary command is a more general-case solution. (See this forum post for more info.)

The \hspace{0pt} tells LaTex not to allow any space (including a newline) between the previous character and the one that follows, acting like glue between the dash added with the \discretionary command and the "d" of dimensional. Without it, you'll get output like this:

text text text text text n-
-dimensional

Edit: Turns out you need the amsmath package for LaTeX to know what the \nobreakdash command is. I've added the necessary command.

Fixing repo init to check out Chromium OS code

Building Chromium OS is a little tricky. Although the instructions from the project page are pretty detailed, one bump I ran into was this error when I put in the "repo init ..." command:

fatal: Cannot get https://chromium.googlesource.com/a/external/repo.git/clone.bundle
fatal: HTTP error 401

Turns out that there's an issue with in the repo init code that makes it fail if there's anything wrong with your ~/.netrc file. My system didn't even have a .netrc file, which made it hard to know how to fix that problem.

The Last Straw for Excel?

Microsoft Office and I haven't been getting along well lately, which is too bad because I have really liked all the new features of Office 2013. But after upgrading Windows to 8.1 this week, Excel decided to stop working. As in, it completely crashes whenever I try to go beyond the view of cells available when it opens. Not cool.

Normally I'd try to figure something out to get everything working, but this weekend I had to input my students' grades so I could discuss their test scores with them on Monday, so I didn't have time to go on a wild Windoze chase for a solution. Instead, I tried out the import feature of Google Docs, and overall I was pretty impressed. Let me share the details...

Getting into Developer Mode on the HP Pavilion 14 Chromebook

So I recently added the HP Chromebook to my cohort of tech gadgets, and naturally wanted to put the device into developer mode. While the information posted on the Chromium Wiki about the HP Chromebook was helpful, it turns out that it is incomplete. So below the break are my abbreviated instructions for getting it into developer mode from start to finish. Another great resource is the HP repair manual, available here.

Downgraded to Python 2.7.1? How did that happen?

So I was doing some work on my home computer tonight which involved a little bit of Python hacking in Windows.  At one point I wanted to make sure that I had a certain library installed such that it was accessible to the script I was working with, so naturally I opened up an interpreter and tried importing it.  It failed, which surprised me since I had just finished using pip to install it and no errors occurred.

The script I was working with was one I wrote a few years ago, so my first thought was that maybe they released a newer version of the library with different capitalization.  Kind of a strange guess, I know, but not too far-fetched considering PEP 8 is still not completely followed by the community and some maintainers of certain libraries are changing this to conform to the standard, but I digress.  I run the pip install command one more time, just to make sure that there wasn't an issue with naming.  Nope, dependency already resolved.

Finally, I noticed that the interpreter I was running was Python 2.7.1.  Well, I knew that couldn't be the problem, but I like to always have the latest version of Python installed so I thought I'd take a minute to rectify that.  But that was another issue... why wasn't I running 2.7.3?  It was weird that I had such an old version running when I'm usually so on top of updating Python.  Whatever.

So I first tried a repair.  Still showed up as 2.7.1.  Downloaded a fresh copy of the installer and verify the checksum.  Still showed up as 2.7.1.  Removed it completely after making a backup of site-packages, then re-installed it.  Still showed up as 2.7.1.  Uninstalled it again without re-installing it.  The interpreter still ran and showed up as 2.7.1.  What the...?

I was starting to get fed up at this point.  The system path didn't have anything out of the ordinary, like a separate folder for an old installation, nothing.  Finally, I found this StackOverflow article which, oddly enough, had an example of exactly what I needed to figure out what was going on and where this rouge installation of Python was that was driving me crazy. Here's what I got:

C:\Users\Mike>for %i in (python.exe) do @echo %~$PATH:i
C:\Program Files\Box Sync\Python.exe

Box Sync includes its own Python interpreter?  That's what's going on?  And it somehow became the default interpreter instead of my 2.7.3 installation?  Okay... I guess that's great they're using Python, but why force my computer to use an old interpreter?  I'm no Windows expert, but it seems like there should be a better way to do whatever it is they're trying to do.  But I can also have sympathy for them not wanting to use Py2Exe... shudder

Apparently, this is something Box is already aware of, but the whole thing is still just so weird.  I don't use Box that much as it is, so I think I'll just nuke their version of Python for the time being.  And yes, it did fix my problem.

The Internet is so awesome (in case you didn't already know)

As I usually do, I was reading up on some tech blogs during my lunch today, and I came across two articles from Engadget that made me stop and ponder how cool it is that you can do so much without going anywhere.

The first article was about an amateur archaeologist named Angela Micol of Maiden, North Carolina, who "has been conducting satellite archaeology research for over ten years, searching for ancient sites from space using satellite imagery. ... Google Earth has allowed her to document many possible archaeological sites, including a potential underwater city off the coast of the Yucatan peninsula" (from the Google Earth Anomalies article).  But this time, she found two previously undiscovered sites where there may potentially be several Egyptian pyramids buried under the sands of however many centuries have passed since their construction.

The second article detailed how an entomologist named Shaun Winterton was browsing Flickr and noticed distinct characteristics in a picture of a Malaysian Lacewing, which is a small insect obviously native to Malaysia.  After some cross-referencing some other material, Shaun confirmed that this was indeed a new species.  Shaun corresponded with the original photographer and a guy at the Natural History Museum to compose a paper in Google Docs on the new species.  Shaun named it after his daughter, which I'm taking to be an honorary gesture.

It really is awe-inspiring to hear stories of individuals that are passionate about a field and have their abilities enhanced and extended across continents and oceans to discover something new.  Now if only a greater percentage of the Internet's users would use it only for purposes like these, I could sleep more easily at night...

Debian on Android and my quest for a full-fledged terminal Python IDE

So when I recently discovered that my Asus Transformer Prime has Python on it, but that it only has version 2.6.2 and is missing some needed libraries, I could tell that setting up a sweet Python programming environment was going to be possible, but a little longer road than I thought it was going to be.  Head past the break to read how I successfully overcame multiple obstacles to get things working.

OpenVPN Update: Fixed!

I finally had time to really do some more troubleshooting on the OpenVPN problem I recently posted about.  After trying some of the same old things to get the server to be able to access the Internet and have the VPN service running at the same time, I decided it was time to purge the installation and start fresh:

sudo apt-get purge openvpn

No fun.  It's always depressing when a configuration problem gets to the point where the only thing left you can think to try is to start over from scratch.  I did have the good sense to back up all the server's keys and configuration files, which in the long run did end up being useful.

As I was once again going through the OpenVPN tutorial provided by Ubuntu and scanning through the help comments in /etc/openvpn/server.conf, when I found the following:

Use "dev tap0" if you are ethernet bridging and have precreated a tap0 virtual interface and bridged it with your ethernet interface.

For a while, I mistakenly thought that this might be the answer I was looking for, since it seemed like that was what I was doing, but only because I didn't understand everything that was going on under the hood as well as I do now.  You see, when the tutorial has you add the line:

up "/etc/openvpn/up.sh br0 eth1"

this isn't exactly what is called to bring up the bridged interfaces.  Taking another look at the up.sh script, there are three network interface devices referenced, even though only two are explicitly given in the configuration line above.  OpenVPN adds the necessary tap0 device to the system, and then adds its interface name as a parameter to the end of the string you specify.

A minimum of three interfaces are needed to create a bridge of the sort that OpenVPN needs: 1) the physical interface with access to the "real world" through which the VPN traffic will initially come in, 2) the destination interface which is actually handled by OpenVPN, and 3) the bridging interface that connects the first two at the kernel level.

As I was figuring out all this, I thought to check the man page of brctl since I knew that the up.sh script called by OpenVPN as it starts up uses brctl to do the kernel-level changes to bridge eth1 and tap0.  Once I really started to understand how the various options worked, I tried refreshing the bridged interface by calling brctl dellbr br0 and then brctl addbr br0, followed by restarting the openvpn service.

Doing this had positive results, but none of them persisted through a reboot of the machine.  Finally, after trying several different things, I came up with hard-coding a refresh of the bridged interface in the up.sh script.  This, plus changing the network configuration of eth0 from static to dhcp seems to have done the trick.  The machine has access to the Internet, and clients can connect just as they could before.  I even copied over the old configuration, certificate, and key files, which prevented me from having to re-issue all the client keys and re-distributing the server's certificate.

The new contents of my up.sh script are below:

#!/bin/sh

BR=$1
ETHDEV=$2
TAPDEV=$3

# Reset the bridge device first
/sbin/ip link set "$BR" down
/sbin/brctl delbr $BR
/sbin/brctl addbr $BR
/sbin/brctl addif $BR $ETHDEV

/sbin/ip link set "$TAPDEV" up
/sbin/ip link set "$ETHDEV" promisc on
/sbin/brctl addif $BR $TAPDEV

Pydroid at last!

Well, hello there, my little friend!  So good to see you where I had given up hope you would ever be:

So just a few minutes ago, I was poking around on my Asus Transformer Prime using the old reliable Terminal Emulator and ran ls /system/bin just to see what kind of executables were there, and lo and behold there was Python installed!  I was a bit surprised to see it, since I haven't done anything to modify the default installation of Ice Cream Sandwich from the factory, but here's the evidence that it runs:

There are a few things to take note of here.  First, it's running Python 2.6.2, which is a little dated since as of this writing, the 2.6 series is up to 2.6.8, which was released April 10 of this year.  So that means there are a number of security patches absent, but I'll likely only be running my own code on here.  Still, I wish I could have some of the new features from 2.7 like the multiple clause support in the with statement to do things like this:

with open(file1) as var1, open(file2) as var2:

    ... 

Second thing of note is the warning "Could not find platform independent libraries...", which may be a result of:

• Running Python from within an app, which all have limited permissions that may be preventing access to those libraries
• Terminal Emulator may need to be told to export the path to the libraries
• Something else that will undoubtedly be tedious to discover and fix

In any case, I'm one step closer to getting my tablet to completely replace my notebook, which I'm totally okay with.

Troubles with OpenVPN in Ubuntu 12.04

Update: Check out my solution to the problem described below.

Last year at work I set up a VPN using OpenVPN in Ubuntu Server 11.10.  There were a number of issues with supporting the specific topology of the local network I was trying to make accessible that made it tricky to get traffic from a connected client properly routed in and back.  To be upfront, I'm not a Linux guru.  I get by with the principles I've learned in the classroom, the tips I've been given from more enthusiastic friends and family, and from what I can glean from forums, help pages, and manual pages, and generally I do alright.

Back to my story.  So last year I honestly don't know what exactly it was that got things to work, which is bad, I know.  At one point I had two other guys helping me while we looked at the traffic going between a client and the VPN server with Wireshark on both ends.  We could see ping packets making it successfully to the server and we could see the responses generated by the server, but no replies were making it back to the client.  Weird.  In that case, I finally figured out that in the different configurations I had tried, at one point I had commented out the bridged interface, br0, from /etc/network/interfaces.  I un-commented those lines, things started working, and I never touched it for fear of breaking things.  Restarting the system kept things going, so it must have been some sort of persistent configuration.

Fast forward to a couple weeks ago, when my overconfidence and excitement at the release of Ubuntu 12.04 LTS got the better of me, overpowering my previous fear of breaking the VPN.  I think I said to myself something like the following. "Well, all the configuration files will carry over, so things can only get better by running 'sudo do-release-upgrade', right?  After all, this is a Long-Term Support release, so it's nice and stable."

Wrong.  Things broke.  So far, I've overcome one hurdle, but things still aren't working, so I have no idea how many other obstacles are in my way to being back up and running.  Here is the evidence I've gathered so far:

• In /var/log/syslog, there was an error about the security level setting for running external scripts.  Adding the line "script-security 3" before calling any scripts made that go away and allowed OpenVPN to successfully start up.  The script I'm running is up.sh as described in the OpenVPN tutorial on the Ubuntu help pages, under the section "Prepare server config for bridging".
• Initially after the release upgrade, the primary connection, eth0, no longer had any access to the outside world.  I could ping the router, but nothing outside the local network.  I found that setting the tap0 device to down and changing eth0 from static to dhcp seemed to fix this.  Unfortunately, later something reverted back and the server no longer has outside access again.  I tried the tricks I used before and even tried stopping OpenVPN with '/etc/init.d/openvpn stop' with no success, even after restarting networking and rebooting the server.
• Even when the server could access external addresses things didn't seem to be routing packets correctly.  I found that by disabling the br0, eth1, or tap0 (I can't remember...), web pages loaded faster and everything seemed to work properly.  Running 'route -vn' was what gave me the idea in the first place because there multiple devices specified as the outgoing for the same destination range.  Unfortunately I didn't memorize the details and things aren't at the point they were previously to try and figure it out.

I'm sure there are some specifics of how OpenVPN works that would help me troubleshoot this, but my struggle with problems like this (which seem to present themselves more often in Linux because I'm doing more advanced things) is always trying to balance understanding the details of whatever I'm working with and just getting the dumb thing to work because I need it done last week or last month...

Hopefully I'll get to the bottom of all this and I'll be able to report my findings soon.  In the mean time, take a lesson from my book, and be a little more cautious than I was with updating to the latest release.

Risk Intelligence: Get better at seeing the future without the two Strawberry Nirvanas

Here's a well-known fact:  It's against human nature to be very good at guessing what's going to happen, whether in the realm of world events, stock prices, personal relationships, or the reliability of your work (read: how good your code is).  As a recent Wall Street Journal article on risk intelligence pointed out:

"Most of us, for example, tend to vastly overestimate our chances of winning the lottery, while similarly underestimating the chances that we will get divorced."

The good new is that apparently we can improve our risk intelligence with practice.  Many professional gamblers keep meticulous notes of how successful they were so they can analyze their methods and adjust accordingly.  Likewise, by going through the exercise of estimating on a weekly basis what we think the likelihood is for any set of events, assigning a percentage or other numerical value to our estimate, and then reviewing how good we did, "we can go a long way toward correcting for our most common errors."

Sounds like there's something I'll be adding to my weekly planner.  Because even though the boost is free, the cost of the smoothie it comes with does add up.

First Post

Welcome to my blog!  The purpose of my entries will be to record my thoughts, discoveries, and progress as I take on new projects, research new topics, and generally seek to improve myself professionally and personally.  I hope you find some of my entries of interest or of help in your own life.