The Offbytwo Blog

In case you didn’t know the pre-login message you see at a Linux console typically comes from /etc/issue

You can customize this file to alter the message with some escape codes that will show things like the current date and time, machine name and domain, kernel version, etc. But one thing you can’t easily display is the IP address of a machine. Showing the IP address is especially useful when building a virtual machine that will use DHCP, like the Ubuntu development VM I use on my Macbook Pro. This way I can start VMware Fusion, see the IP address of the VM and then login over SSH.

In order to get the IP address to show in /etc/issue I needed to write a custom script that will rewrite /etc/issue with the IP address when the network interface is brought up. The first step was writing a simple script that will output the current IP address when run (by looking at the output of ifconfig).

/sbin/ifconfig | grep “inet addr” | grep -v “127.0.0.1″ | awk ‘{ print $2 }’ | awk -F: ‘{ print $2 }’

The above script will run ifconfig and print out the IP address (after filtering out the localhost interface). I saved this script to /usr/local/bin/get-ip-address. In order to get this into /etc/issue I decided to first copy /etc/issue to /etc/issue-standard, then create the following script that when run will overwrite /etc/issue with the contents of /etc/issue-standard + IP address.

#!/bin/sh
if [ "$METHOD" = loopback ]; then
    exit 0
fi

# Only run from ifup.
if [ "$MODE" != start ]; then
    exit 0
fi

cp /etc/issue-standard /etc/issue
/usr/local/bin/get-ip-address >> /etc/issue
echo "" >> /etc/issue

Since I’m using Ubuntu I put this script in /etc/network/if-up.d/ and mark it executable. This will run the script automatically each time an interface is brought up. After restarting networking the IP address now shows up at the pre-login message.

Update: the post below is about installing lxml 2.0 (2.0.5 is the specific version I installed).

One way to install lxml is to install the py25-lxml package from MacPorts, but I hate installing python packages from MacPorts as it insists on installing it’s own version of Python. I like installing setuptools and using easy_install to manage my Python packages. I got lxml to install just fine in the past, but when I tried to do it again last night I ran into some problems so I figured this time I’m going to write down exactly how I solved it so I can refer back to it next time.

I vaguly remembered from last time that I needed to install libxml2 and libxslt in order to build lxml. So I used MacPorts to install libxml2 and libxslt and I tried building lxml again. Still had the same problems as before. It seems that Leopard comes with a version of libxml2 (/usr/include/libxml2), but lxml refuses to build against it (seems to be missing some header files, I’m guessing it’s the wrong version but I didn’t investigate further). It seems that lxml was picking up the system libxml2 instead of the MacPorts version.

Looking through the lxml documentation I found some options for specifying the library to build against

python setup.py build --with-xslt-config=/path/to/xslt-config

If this doesn’t help, you may have to add the location of the header files to the include path like:

python setup.py build_ext -i  -I /usr/include/libxml2

I don’t really like this options since I really want to do easy_install lxml and have it work. But the first suggestion got me thinking that maybe I need to change my path to have /opt/local/bin before /usr/bin. So I added /opt/local/bin in front of my PATH and sure enough lxml now installs with easy_install without a problem.

A summary for the lazy readers:

sudo port install libxml2 libxslt
export PATH=/opt/local/bin:$PATH
sudo easy_install lxml

A Python regular expression for extracting the engine, username, password, host and port from a database connection string of the form:

engine://[username[:password]@]host[:port]

r’:(mysql|postgres)://(?:([\w]+)(?::([\w]+))?@)?([\w.-]+)(?::([\d]{1,5}))?’

I’ve started using git for all my development recently (since it integrates so nicely with svn). I wanted to experiment with running my test as a pre-commit hook in git. In case you’re curious all the hooks in git live in the hooks folder inside of .git

Inside of this folder you will see various example scripts. The names should make it obvious when each hook is supposed to run. For example the pre-commit file will run before a commit (before you’re even asked for the commit message). There are also hooks that can intercept the commit message, run after updates happen, etc. By default none of these files are executable, so git doesn’t actually run them. If you would like to execute a hook simply put your code in the correct file and mark it executable.

In the case of the pre-commit hook git will abort the commit if the pre-commit file returns with a status code other than 0. By default this file contains some perl code that checks for lines with trailing spaces and lines that have a space before a tab at the beginning. You can safely remove this code (I found the trailing space to be an annoying check)).

So let’s say you want to run your unit tests before each commit (and abort the commit if they fail). I’m going to use nose (a Python unit testing framework) as an example. To run your nose tests you can simply issue the nosetests command. This will discover your tests, run them and exit with status code of 0 if everything passed. So you can simply put

#!/usr/bin/env bash
nosetests

in your pre-commit file and now your unit tests will run and your commit will abort if the are test failures. This works well, unless you have tests that you expect to fail but still have something you would like to commit. You have to choices: either remove the executable bit from the pre-commit file or adjust your script to give you some options. Here is a little script I put together to prompt you if you would like to commit anyway in the even of test failures. Keep in mind I know very little if any bash scripting so if there is a better way to do this please let me know.

nosetests
code=$?

if [ "$code" == "0" ]; then
exit 0
fi

echo -n “Not all tests pass. Commit (y/n): ”
read response
if [ "$response" == "y" ]; then
exit 0
fi

exit $code

Hope this helps.

It took me way too long to find the answer to this so I decided to post it here, hopefully it will help someone in a similar situation. I have my Comcast DVR connected to my Harman Kardon AVR-247 and even though both devices come with universal remotes I could not find codes to make both devices run from a single remote. Harman Kardon doesn’t even list Motorola under the list of vendors for cable boxes. I eventually found out that Motoral purchased General Instruments a couple years back so I took another look at the list of codes for the AVR-247 remotes and sure enough, there is an entry for General Instruments.

The code for the Motoral cable box/dvr provided by Comcast is 210. And if you can’t find the manual for the AVR-247 here is how to program it. Hold down the key for the input your cable box is connected to (in my case it was HDMI-1). If your device is on the second input for the key (for example HDMI-2) then you need to press it twice then hold (even if it was previously selected). Hold until you see the light flash. Now press the button for the type of device you are programming (look above the video 1, 2, 3 buttons). Now enter the code and make sure your device turns off. If it was successful press the input key again (in my case HDMI-1) and you’re done.

Another useful tip is the punch through functionality of the remote. Typically if you select one of the input sources you can’t change the volume on the AVR until you select the AVR button on the remote. To fix this press and hold the key for the input you want to work with (just like above, in my case it was HDMI-1). When the light starts flashing press the volume up key (to punch through volume control) then press the input key for the device you would like volume control to go to (you most likely want the AVR button).

I could not get the DVR functionality to work on the new remote but I can change channels, see the guide, and access on demand. Let me know if you find a way to get DVR functionality working on the AVR-247 remote, or if you find a way to control the sound of the AVR-247 from the Comcast remote.

def fib(n):
     q = (1 + math.sqrt(5)) / 2
     return int(((q ** n) - (1 - q) ** n) / math.sqrt(5))

This is known as the Binet’s formula and it’s using the Golden Ratio to compute the nth Fibonacci number in constant time. This uses floating point operations so it only works within the range of floating point numbers available on your platform. Making this work with unlimited precision floating point numbers is probably less efficient than the linear implementation.

Try it out

fib(10)
55

fib(20)
6765

I’ve been banging my head for a while trying to get mod_proxy to work nicely on my desktop (using Ubuntu Gutsy). I kept getting errors of the form

client denied by server configuration: proxy

Eventually I found out that the proxy.conf (under mods-available) was disable all proxy requests. So I changed theProxy section to

         <Proxy *>
                AddDefaultCharset off
                Order deny,allow
                Allow from all #
        </Proxy>

This seemed to help but things still didn’t work. Instead I got the following error

“No protocol handler was valid for the URL [...]. If you are using a DSO version of mod_proxy, make sure the proxy submodules are included in the configuration using LoadModule.”

Turns out I needed to run

 a2enmod proxy_http

which you can install by running apt-get install libapache2-mod-proxy-html. I could swear I previously enabled that module so I was surprised that it wasn’t loaded, but I’m guessing I accidentally disabled it while poking around. So if you’re seeing errors about “no protocol handler” make sure you have proxy_html enabled.

By the way, for a pretty comprehensive way for running a reverse proxy under Apache see http://www.apachetutor.org/admin/reverseproxies

Python 2.5 has support for partial function evaluation in the functools module. Here’s is something I hacked out to do the same in Python 2.4

def partial(f, *args, **kw):
    def newf(*nargs, **nkw):
        targs = list(args)
        targs.extend(nargs)
        tkw = kw.copy()
        for k in nkw:
            tkw[k] = nkw[k]
        try:
            return f(*targs, **tkw)
        except TypeError, te:
            return partial(f, *targs, **tkw)

    return newf

def partial_decorator(f):
    def evaluate(*args, **kw):
        try:
            return f(*args, **kw)
        except TypeError:
            return partial(f, *args, **kw)

    return evaluate

Here is an example

@partial_decorator
def f(x, y):
    return x + y

Let’s try it out

In [5]: g = f(2)

In [6]: g(3)
Out[6]: 5

This is not as complete as the wrapper in Python 2.5: it doesn’t attempt to preserve doc strings, etc. But it should work just fine for most cases. Be warned however that it doesn’t know how many parameters the function being decorated is supposed to take so if you end up passing it more than necessary you’ll never a result. Also if your function raises a TypeError for other reasons it won’t work as well. If you have any suggestions or improvements you’d like to share post them below.

I ran across this scheme to LLVM compiler. It’s a toy compiler so performance isn’t great but I think it’s a great example of how to write a small language and target it to LLVM.

So as everyone probably knows by now Paul Graham’s Arc programming language is finally out in limited beta. The new language has been met with a fair bit of criticism. Paul has responded by expressing the motivation behind his design: helping make programs shorter. Paul’s idea is that languages that allow one to write shorter programs are inherently more powerful. He backed this up with the Arc challenge: writing a simple web application in other languages and seeing how they compare to Arc.

If you get impatient here’s the challenge

Write a program that causes the url said (e.g. http://localhost:port/said) to produce a page with an input field and a submit button. When the submit button is pressed, that should produce a second page with a single link saying “click here.” When that is clicked it should lead to a third page that says “you said: …” where … is whatever the user typed in the original input field. The third page must only show what the user actually typed. I.e. the value entered in the input field must not be passed in the url, or it would be possible to change the behavior of the final page by editing the url.

And if you’re curious here’s the solution in Arc

(defop said req
  (aform [w/link (pr "you said: " (arg _ "foo"))
           (pr "click here")]
    (input “foo”)
    (submit)))

I’ll have to admit this is pretty impressive at first sight. (Well, there’s a small trick here and that’s that PG patented the approach used here, although the patent does currently belong to Yahoo). Folks have complained that this isn’t relevant because they need designer friendly HTML, cross browser support and a couple of other features of a real world site. Paul called this corporate bureaucracy and to his defense Hacker News is running in Arc and works pretty good for me.

I thought about this challenge some more and realized this is a terrible test for a language that claims to be “the hundred-year language“. A problem is that the challenge only illustrates Arc’s power at creating web applications (at least simple prototypes of applications). My reason for this complaint is that I happen to do more than just web programming. I occasionally have to write software for embedded systems, real-time video processing, interfacing with hardware devices over C APIs or serial port communication, shell scripting and text processing.

So what I really need is a language that can excel at all of the above. Personally I think that’s pretty tough for any one language to accomplish. I think there are trade-offs involved so when a language becomes good at one thing it starts to be worse at others. Some languages might excel at text processing and scripting but not be great at writing assembly level code. Others might allow you to write operating systems and probably not be great at writing web applications.

I’m not convinced a language can excel at all of the above. But there is a solution: languages that allow the programmer to create domain specific languages. This allows the programmer to create the necessary abstractions for the given domain and then write an application in that new language. This is guaranteed to create the most succinct and clear way to express the problem at hand.

The good news is that some languages already exist that allow programmers to do this. The most likely candidates come from the Lisp family: namely Common Lisp and Scheme). Now there are problems with both that I won’t even start to address here. But I’ll quote Paul on this:

McCarthy didn’t get very far along it in his paper. And after that the language passed into the hands of his grad students, who at the time were more worried about the exigencies of making an interpreter run on the IBM 704 than continuing McCarthy’s axiomatic approach. We’ve been living with their hacks ever since. Steele and Sussman tried to start over when they first began working on Scheme, but they seem to have been practically the only ones. And they made, at least from the point of view of brevity/power, some serious mistakes early on.

So there is room for improvement.

What should a language do to be a true “hundred-year language”? It should make it easier for programmers to write DSLs. So here’s my challenge:

Implement Arc in your favor language (or better yet in a language you invent) and then see how your implementation of Arc compares to other languages. The language with the “shortest” code wins. I’ll use Paul’s rules and measure the size of the code tree.

I believe the winner of this challenge will have the most powerful macro system and I believe might have a shot at being a true hundred-year language. Since Paul wrote the book on macros this should be interesting.