I’ve also uploaded a script I wrote called get_quad.py, which will download USGS topographic maps automatically. This script naturally complements gdal_slice.py; using the two together allows for almost-totally automatic population of GPSdrive’s map database with high quality topographic maps.

usage: get_quad.py [-h] [-q] [-s STATE] (-g | -c COORD COORD)

Get DRG USGS quad for given latitude and longitude from LibreMaps.org

optional arguments:
 -h, --help show this help message and exit
 -q, --silent don't produce any output
 -s STATE, --state STATE
 Postal abbr. for the state in which the quad falls, so
 we don't need to look it up
 -g, --gps get coordinates from GPSd
 -c COORD COORD, --coords COORD COORD
 specify latitude N and longitude E (respectively) in
 decimal degrees; longitude should be negative, as all
 maps fall in the western hemisphere

Unfortunately, because of the way USGS maps are organized, the script needs to know the state in which the coordinates fall. If the -s flag is not given, this is inferred from the coordinates, but this lookup requires internet access.

Although I had never been aware of the term “indie web” (which seems to have been around for almost 15 years, judging by this article from uzine.net), I’ve been building my own independent web presence for more than four years. In the words of the IndieWebCamp home page:

Rather than posting content on many third-party silos of content, we should all begin owning the content we’re creating. Publish short status updates on your own domain, and syndicate to Twitter. Publish photos on your own domain, syndicate to Flickr, etc, etc.

For some good summaries of the ideas and goals of the IndieWebCamp, this article from GigaOm and this post from Mark Hendrickson are a great place to start, while each of the camp’s guests also have their own websites, each with a wealth of information.

While I’ve been using this domain for quite a while, it’s been in use alongside Facebook, Delicious (and Flickr, to a smaller degree) as a part of my web presence. However, since I ditched Facebook five or six months ago, I’ve been relying on this domain for more, and overall I’ve been pleased. However, as the IndieWebCamp is readily willing to admit, there is room for a lot of room for technical innovation in expanding the functionality of the indie web. Towards that end, I see three threads of work that I’m interested in pursuing:

• Using my site for Flickr-like photo sharing. While this might include syndication to Flickr at some point, I mostly want a light-weight DIY solution to photo sharing.
• Similarly, using my site for bookmarking. I really like the service that delicious offers, but I want to own the data and let delicious simply keep a copy of it (see http://adactio.com/journal/4197/ for some ideas).
• Finally, I’m hugely enamored of the work on delivery-agnostic messaging that took place at IndieWebCamp, which you can find details of here. Not only is it a ridiculously simple protocol, but it starts to address some real issues of a distributed social web, in that it allows two-party communication based solely on domain names as unique identifiers, therefore eliminating the middleman in social-web-type communications.

Luckily, I quickly found out that GDAL ships with Python bindings. I installed GDAL from source using the instructions from their wiki. To make sure it built with Python modules I ran ./configure –with-python. After that make and sudo make install ran without a hitch.

I then set about re-implementing gdal_slice.sh in Python. Once I got a handle on how to use the Python bindings it was fairly easy to write. You can find the script here.

Using gdal_slice.py is very similar to using gdal_slice.sh:

$ ./gdal_slice.py -h
Usage: gdal_slice.py [options] FILENAME

Options:
  -h, --help            show this help message and exit
  -o OVERLAP, --overlap=OVERLAP
                        percentage tiles will overlap. should be at least 20%
  -a, --add             write map info to map_koord.txt in current working
                        directory
  -m, --map             use *_map folders for output; use if input image is
                        UTM. Default behavior
  -t, --topo            use *_top folders for output; use if input image is
                        not UTM
  -v, --verbose         enable debugging output

The biggest difference is that gdal_slice.py does not perform any format conversion; it will write maps to TIFF files. Although the file size is decidedly larger than that of PNGs, my version of GDAL is unable to create PNGs, and disk space is (fairly) cheap. If I’m motivated I’ll get around to adding a format option, at least to allow conversion to PNG.

Running gdal_slice.py filename.ext will create a folder named filename_map (or _top, if you use the -t flag) that contains a set of 1280×1024 TIFF files, as well as a map_koord_draft.txt file. Moving this folder into GPSDrive’s map directory, and merging filename_[map|top]/map_koord_draft.txt with map_koord.txt will make the new maps available to GPSDrive. Note that filename.ext must be in the current working directory, as the script handles filenames/paths somewhat naivëly.

So far I’ve only tested this with GeoTIFFs from LibreMaps, but it seems to work great, with the caveat that LibreMaps’ files include the borders of map, and my script doesn’t do any cropping, so you see borders if you’re near the edges of quads. This doesn’t bother me terribly much, but you may need to turn mapsets on and off in the Map Control dialog to see the appropriate images.

Update 7-14-11: I’ve modified the script with an additional flag (-a) that will automatically merge the generated map_koords file with map_koord.txt in the current working directory. This means that if you run the script from inside .gpsdrive/maps with the -a flag the generated tiles will be automatically added to GPSDrive’s database. The script avoids creating duplicate entries in map_koord.txt, as well as alphabetizing all entries.

I’ve modified the documentation here, as well as the links, to reference the latest version.

Cryptonomicon

Randall Lawrence Waterhouse

Current meatspace coordinates, hot from the GPS receiver card in my laptop:

8 degrees, 52.33 minutes N latitude 117 degrees, 42.75 minutes E longitude

Nearest geographical feature: Palawan, the Philippines

(from http://www.euskalnet.net/larraorma/crypto/slide51.html)

Using GPSd’s python bindings, and the free, CC-licenced RESTful API from GeoNames.org, I wrote a short python script that will produce a similar output:

Seth Just

Current meatspace coordinates, hot from the GPS receiver card in my laptop:

41.751 N latitude, 111.807 W longitude

Nearest geographical feature: Logan Canyon, Utah, United States

The core of the script is two functions — one that gets GPS coordinates from GPSd, and a second that does a lookup for geographical features (mountains, canyons, islands, etc) on GeoNames.org. For more details, see the final section of this post.

Using gpssig.py

To use gpssig.py, first download it from here, and change the extension to .py.

In order to be useful as an email signature, I had to produce output in HTML format, which allows almost any mail client to use the output as a signature. I’ve gone through the effort of setting it up with the two clients I use the most: Mozilla Thunderbird and Apple’s Mail.app. Of these two, Thunderbird has documented support for HTML signatures (see here for details), and should be the most like other email clients. This script has been tested on Mac OS X Snow Leopard, and should run on Linux without problems (although I haven’t tried it on versions of python higher than 2.6.1).

Using gpssig.py takes two steps: first configure your Mail client to take a signature from a file, and then configure gpssig.py to run automatically to update that file.

Thunderbird is relatively easy to set up with an HTML signature. Because it only supports one signature per account, you simply need to tell it to draw that signature from a file. Under each account’s settings there’s an option to “Attach the signature from a file” (see here for a detailed howto). I chose to use ~/.signature.html, but you can use any file you’d like to, just remember the location you choose.

Mail.app is a more finicky beast — although the script supports it, I recommend using Thunderbird, mainly because Mail only refreshes its signatures from files on every launch, whereas Thunderbird will read the file each time you compose a message. However, if that limitation is acceptable (if you don’t move very much, perhaps), it will work just fine. The main issue with Mail.app is that it doesn’t store signatures in an HTML file, but instead in the proprietary .webarchive format. These files can be found in ~/Library/Mail/Signatures. To use gpssig.py with Mail, you need to follow steps 4&5 from http://bytes.com/topic/macosx/insights/825900-setup-html-signatures-apple-mail-mail-app. First create a new signature (under Preferences->Signatures). This will create a new file in the signature folder — take note of the file name. You will use this file name to configure gpssig.py.

gpssig.py is configured with command line options:

Usage: gpssig.py [options] NAME FILENAME
Note: NAME should be enclosed in quotes, unless it contains no spaces.

Options:
  -h, --help   show this help message and exit
  --no-gps     don't attempt to get coordinates from GPS
  -a, --amail  generate files for Apple's Mail.app (.webarchive)
  --lat=LAT    default latitude, if GPS is unavailable
  --lon=LON    default longitude, if GPS is unavailable

The simplest way to use it is manually — simply running it whenever you want to update your signature. For example, I use ./gpssig.py –lat=41.752 –lon=-111.793 ‘Seth Just’ ~/.signature.html to generate a signature for Thunderbird.

If you want to make this automatic, you’ll need to have the script run automatically. On OS X you’ll need to use launchd, while on linux you need cron. Both of these make setting up repeating actions very easy — see here for information on launchd and here for information on cron.

Sadly I haven’t yet figured out a way to run the script whenever you write an email — if I figure it out, I’ll be sure to post something.

The Script

gpssig.py is built around three major functions — one gets coordinates from GPSd, one gets geographical feature information from

The function I use to get GPS information is a bit hacky — there doesn’t seem to be terribly much documentation on GPSd’s python bindings, so I made do with what I could figure out:

class NoGPSError (Exception): pass
 
def get_lat_lon():
  # GPSd Python bindings
  import gps
 
  # Create GPS object
  session = gps.gps()
 
  # Set GPS object as an iterator over reports from GPSd
  session.stream(gps.WATCH_ENABLE|gps.WATCH_NEWSTYLE)
 
  # Loop until we get a report with lat and lon. The limit of 5 loops should be more than enough -- my gps never takes more than three when it has a lock
  i = 0
  while (1):
    try:
      session.next()
      lat, lon = session.data['lat'], session.data['lon']
      break
    except:
      if (i > 5): raise NoGPSError 
      i += 1
 
  return lat, lon

The other function is more straightforward — it makes two http requests to GeoNames.org to get the nearest geographical feature and locality. It then combines those responses in a way that should provide a good response almost anywhere in the world (provided that GeoNames has good, local data). I’ve tested it with a variety of coordinates (provided by http://www.getlatlon.com/), and the responses seem to be pretty good.

def get_geo_string(lat, lon):
  # Modules for REST/XML request from GeoNames
  import urllib
  from xml.etree import ElementTree as ET
 
  # Request the name of the nearest geographical feature
  placename = ET.parse(
      urllib.urlopen("http://api.geonames.org/findNearby?lat=" + str(lat) + "&lng=" + str(lon) + "&featureClass=T&username=sethjust")
      )
  subdiv = ET.parse(
      urllib.urlopen("http://api.geonames.org/countrySubdivision?lat=" + str(lat) + "&lng=" + str(lon) + "&username=sethjust")
      )
 
  # Format and return place, region, country
  try:
    country = subdiv.getiterator("countryName")[0].text
  except:
    try:
      country = placename.getiterator("countryName")[0].text
    except:
      country = ''
  try:
    region = subdiv.getiterator("adminName1")[0].text
  except:
    region = ''
  try:
    name = placename.getiterator("name")[0].text
  except:
    name = ''
 
  if country:
    if region:
      if name:
        result = "%s, %s, %s" % (name, region, country)
      else:
        result = "%s, %s" % (region, country)
    else:
      if name:
        result = "%s, %s" % (name, country)
      else:
        result = country
  else:
    result = "unknown"
 
  return result

Finally, the main method of the program parses arguments, determines what coordinates to use, gets the geographical feature string, and then generates the proper HTML and copies it into the correct place.

if (__name__ == "__main__"):
  # Parse command line options
  from optparse import OptionParser
  parser = OptionParser("Usage: %prog [options] NAME FILENAME\nNote: NAME should be enclosed in quotes, unless it contains no spaces.")
  parser.add_option("--no-gps", action="store_true", dest="nogps", default=False, help="don't attempt to get coordinates from GPS")
  parser.add_option("-a", "--amail", action="store_true", dest="amail", default=False, help="generate files for Apple's Mail.app (.webarchive)")
  parser.add_option("--lat", action="store", type="float", dest="lat", help="default latitude, if GPS is unavailable")
  parser.add_option("--lon", action="store", type="float", dest="lon", help="default longitude, if GPS is unavailable")
 
  options, args = parser.parse_args()
 
  try: assert(len(args) == 2)
  except AssertionError:
    print "You must provide NAME FILENAME\n"
    parser.print_help()
    exit()
 
  try: assert(((options.lat!=None) & (options.lon!=None)) | ((options.lat==None) & (options.lon==None)))
  except AssertionError:
    print "You must provide both lat and lon arguments\n"
    parser.print_help()
    exit()
 
  # Modules for creating signature file
  import tempfile, os
 
  try:
    if (options.nogps): raise NoGPSError
    lat, lon = get_lat_lon()
    print "Got %f, %f from gps" % (lat, lon)
  except NoGPSError:
    if (not options.nogps): print "Failed to get coordinates from GPS"
    try:
      assert((options.lat!=None) & (options.lon!=None))
    except AssertionError:
      print "No default coordinates provided; exiting"
      exit()
    lat, lon = options.lat, options.lon
    print "Using %f, %f as coordinates" % (lat, lon)
 
  geostring = get_geo_string(lat, lon)
 
  lac, loc = 'N', 'E'
  if lat < 0:
    lat *= -1
    lac = 'S'
  if lon < 0:
    lon *= -1
    loc = 'W'
 
  file = tempfile.NamedTemporaryFile()
 
  file.write("<html><body>\n<p>")
  file.write(args[0])
  file.write("</p>\n<p>")
  file.write("Current meatspace coordinates, hot from the GPS receiver card in my laptop:")
  file.write("</p>\n<p>")
  file.write("%.3f %s latitude, %.3f %s longitude" % (lat, lac, lon, loc))
  file.write("</p>\n<p>")
  file.write("Nearest geographical feature: " + geostring)
  file.write("</p>\n")
  file.write("</html></body>\n")
 
  file.flush()
 
  if (options.amail):
    os.system("textutil -convert webarchive -output " + args[1] + " " + file.name)
  else:
    os.system("cp " + file.name + " " + args[1])

In September 2010, Hans Viksler posted an article about the insecurity of the stored-value smart card system used by Web Laundry (now Wash Laundry). I’ll let that article stand on its own — it gives a good overview of the bone-headed system that the engineers there had in place. In short, they used a secure memory card made by Atmel (specifically, from the CryptoMemory line), the AT88SC0404C (datasheet here), but neglected to change the default master password. This means that all you need to read or write the stored data is a ISO-7816-3 compliant smart card reader.

This is where my story starts — this summer I’m living in a dorm that uses a Web Laundry stored value system. That was enough of a push for me to buy a smart card reader, seeing as I’d always wanted to play with them, and this looked like a good opportunity. I picked up a smart card reader from Amazon.com, an SCM Microsystems SCR3310. It was both cheap and, according to the manufacturer’s product page, offers full ISO 7816, CCID and PC/SC compatibility (i.e. all of the relevant standards), as well as drivers for OS X, which is my primary operating system for the time being.

Doing some research, it turns out that Apple ships a (modified) version of the open-source PCSC Lite framework for smart card integration. This means that all I would need to get up and running was a driver for my device and a way to interface it. Once the reader arrived I installed the drivers, and ran pcsctest (from PCSC Lite) on the command line. Sadly, all I got was an error message:

Testing SCardEstablishContext    : Service not available.

Googling told me that the appropriate services are supposed to start whenever a smart card reader with appropriate drivers is plugged in, which made this error all the more confusing. Luckily, I found this thread on SCM’s forums — it seems that the driver they have posted doesn’t include the USB identifiers for the version of the reader I had, but a helpful poster left instructions to modify a plist file in the driver (mirrored here for archival purposes). With that, I was up and running!

pcsctest spat out useful output this time around:

$ pcsctest

MUSCLE PC/SC Lite Test Program

Testing SCardEstablishContext    : Command successful.
Testing SCardGetStatusChange
Please insert a working reader   : Command successful.
Testing SCardListReaders         : Command successful.
Reader 01: SCM SCR 3310 00 00
Enter the reader number          : 1
Waiting for card insertion
                                 : Command successful.
Testing SCardConnect             : Command successful.
Testing SCardStatus              : Command successful.
Current Reader Name              : SCM SCR 3310 00 00
Current Reader State             : 0x34
Current Reader Protocol          : 0x0
Current Reader ATR Size          : 8 (0x8)
Current Reader ATR Value         : 3B B2 11 00 10 80 00 04
Testing SCardDisconnect          : Command successful.
Testing SCardReleaseContext      : Command successful.

That left me with the issue of interacting with the smart card somehow. In case you hadn’t guessed from the post title, that solution was Python. Specifically, the pyscard bindings for PCSC Lite, from pyscard.sourceforge.net. There’s a nice array of examples, and so I wrote up a python script for interacting with this particular CryptoMemory card, based on the command set from the datasheet and the examples. It’s not as polished or feature-rich as the software Hans posted, but if you’re running OS X or Linux, it should be helpful. At the moment this script is extremely rough — it gets the job done, but in an ugly and unsafe way. I’m posting it here, but you SHOULD NOT USE IT IF YOU DON’T KNOW WHAT YOU’RE DOING. If you aren’t careful, it will mess up your card. Hopefully I’ll have time to wrap it all in a library that will abstract away the messiness and provide nice features like exception handling; if I do, you’ll see it here.

Using my script I was able to verify Hans’ findings — although the card is configured to require cryptographic authentication to access the user zones (where the goodies can be found), the password is left at the default value, which makes overwriting the access registers and conducting a replay attack (as demonstrated by the video in his post) a walk in the park. Following his example, I won’t post any information on how the cards store their values, but it’s surprising that this security hole still isn’t fixed, seeing as it would cost nothing to overwrite the master password when they first configure the cards.

Hopefully this post will give some of you who’ve wanted to work with smart cards a push in that direction — although there are a few hurdles along the way, it’s an interesting area to work on. I’m hoping to move on from poking at other people’s cards and start using my new hardware for authentication and encryption.

Happy hacking!

My goal for this post is to record my process in putting this project together. I’m nowhere near done at the moment, but I’ll keep updating it as I go.

Planning

The plan for this project came together slowly – I knew that I wanted to make a large RGB LED display, and I had already bought one light strip to play with a year or so ago. I’m not sure how the Rainbowduino came across my radar, but it seemed perfect, because it could drive up to 64 RGB LEDs simultaneously, so could support up to 6 strips. However, the funding for the project chose to use 3, and I felt that that was more than enough to get the effect I was shooting for.

Because the strips run at 12 volts, and are each rated for 6 amps, I decided to use a spare PC power supply to provide power because it supplies a well-regulated voltage and plenty of current.

Finally, the Rainbowduino can operate in one of three ways: standalone mode, serial mode, or I2C mode. The second and third allow you to stream patterns to the Rainbowduino from another microcontroller or computer. However, I want this sun to stand on its own, so I’m planning on having it run in standalone mode. This means I’ll need a way of programming it with whatever patterns I want it to use. The Rainbowduino has a 6-pin ICSP plug, which will work perfectly with the AVR programmer I already have, Lady Ada’s UsbTinyISP.

Preparation

To prepare for putting the whole display together, I wanted to test my LEDs, power supply and control board. I used an RGB driver based on a Teensy 2.0, a few FETs and a wall-wart (which I know works well with these strips) as a simple tester for the LEDs, and found no problems. The power supply also checked out, I took apart a single 4-pin plug to get the +12V and GND wires and checked the voltage with a multimeter. The reading was good and steady, but attaching it to the Rainbowduino caused one of the power-filtering capacitors to explode (which was unexpected, given the board is rated for 12V), but removing the damaged cap fixed the problem, and soon I had it powered up and lighting LEDs. I also connected the UsbTinyISP and used Avrdude to ensure I could program the Rainbowduino. (As an aside, the Avrdude part name for the ATMEGA328P is ‘m328p’, which took me way too long to figure out).

At this point I felt comfortable that I had everything I needed to put the display together (except for some sort of mounting board. However, the end of the weekend was drawing to a close, so I had to wait to get more done.

Steeling Myself

Before I prepared to cut my 3 strips into 30 smaller ones, and soldered 4 wires to each one, I decided to get a picture of what the final wiring would look like. This was a job for Cadsoft’s Eagle, the defacto-standard for making schematics and PCB layouts. Even though I’m planning on laying out the display by hand, with wires, I felt that having a reference for wiring the matrix would be worthwhile. I built a schematic to show the control layout for the LEDs, as a matrix, which is what the Rainbowduino is designed to drive. I then built a layout based on the schematic that corresponds (roughly, and not to scale) to how I’m planning to arrange the LED strips, although the footprints of all the parts are completely arbitrary. However, the pin layout of the headers on either side match the layout of those on the Rainbowduino, which will make it helpful when it comes time to connect all 120 wires.

Wiring

Update, May 2: I’ve now finished wiring the matrix: I attached all of the strips to a piece of foamcore board using their adhesive backing and soldered all of the connections. Everything worked on the first try, and I flashed the code (rainbowduino_v1_0_4) from here. The demo patterns looked amazing, but sadly, shortly after powering it up (before Icould even get a video), something stopped working. Not only were the patterns not being displayed, but I couldn’t get my programmer to connect to the rainbowduino! Luckily, I emailed SeeedStudio, and one of their very helpful employe has promised me that a replacement is already on the way. Hopefully it arrives before the fifth, when I was hoping to set up the display on campus. Until it arrives, and I can replace the defective board on the display, enjoy the only two photos I took while it was working (before I flashed the new firmware onto it).

Final words

Despite the setback with my Rainbowduino, I went ahead with getting the display ready for this weekend, where (hopefully) it will be sitting in a central place, entertaining people with colorful patterns. To do this, I used a large zip-tie to strap the ATX power supply to the back of the piece of foamcore board, and used some more zip ties to position all of the wires out of sight, as well as to securely fasten the two wires running 12V to the display. All in all it looks good, and I’m excited to load some exciting patters onto it once my new control board arrives.

Also, this post is still in progress — check back soon for updates! Apologies for the lack of photos, they should be coming soon.

It is now replacing an older, wireless, but more compact keyboard from the same company, which, despite serving me well, had been getting flaky in age.

Hit the jump for another picture and a review/comparison of the two keyboards

As I said, this keyboard is replacing my old wireless one, and offers several advantages over it. The first thing I noticed after taking it out of the box was the size: the new one is considerably bigger, allowing it room for a full, proper numpad and, arrow keys, and Home/End/PgUp/PgDn/Ins/Del keys that need a Fn key on the old keyboard.

The second major difference is the new key design, which they call “chocolate key”. It’s the same design as Macbook keyboards, with flat keys shaped like rounded rectangles. The combination of a new key design and layout mean that the new keyboard is a lot roomier — my hands don’t feel cramped and can move freely.

The last thing I noticed quickly after getting the new keyboard set up is the feel of the keys. While the old, wireless keyboard has the mush feel of most laptop keyboards, the new key design has a much crisper feel and a more “clicky” feel, even though it still uses scissor-style key switches. While this makes typing slightly louder, it feels much nicer, and makes typing more enjoyable and responsive.

In all, the new keyboard is a welcome update. However, it won’t completely replace the old one. The biggest reason is the wireless keyboard’s range: because it will work anywhere within hearing range of the computer, I use it as a remote control for my music player.

The source code of the script used to generate the images is available here.

Get the code here.

$ time python mandel.py > \dev\null
real	0m0.051s
user	0m0.036s
sys	0m0.010s
$ time perl mandel.pl > \dev\null
real	0m3.518s
user	0m3.463s
sys	0m0.029s

You can find the code here.

This script works well for zooms, as long as you stay below a few thousand iterations. The following picture was generated with x=-1.1887204, y=-0.3032472, width=0.01 and 150 iterations.

Just Another Perl Hacker

I figured that it was about time in my hacking career (read: I was bored enough) that I should make a japh script. After a couple attempts I came up with this:

#!/usr/bin/perl
while(<DATA>){
 	for (map{ord($_)-33}split ''){
		$__++;
		$_||(print(chr($__+19))&&($__=0));
	}
}
 
print "\n";
 
__END__
                   /|                        |\
      !            ; :                        : :
                  | Y,                      ,P |
     !             |  Yb.        __        ,dP  |
                  l\  YMMb,_ _,/  \,_ _,dMMP  /f
  !                 j;  `YMMP'  `--'  `YMMP'  ;j
                   : \   YP`-._    _.-'YP   / ;
  !            !      \ `\,  _,\_    _/,_  ,/' /
                     `,_,   \`o>  <o'/   ,_,!'
                         `\            /'
                           |  _    _  |
      !                     / 88b  d88 \
                           \ `8P  Y8' /``-.
             !               `\,    ,/      \
                             _`----'        \
                  !        _.'/ '          `, \
                        .'_,'_..._          \ ;
       !               .',/.dMMMMMMb. |      /  \
                    .'./.MMMMMMMMMMM ; !    /    ;
                   / /.MMMMMMMMMMMMM/     /     ;
                  | ;dMMMMMMMMMMM!P/'   ,/'   !    ;
                  |/dMMMMMMMMMP'     .'          |
   !               ;dMMMMMMMMM|     _/            |
                 ;dMMMMMMMMMMb_|_!| /;            ;\,
                 dMMMMMMMMMMMM`M`M`;.---..      '   `\,
                :M!MMMMMMMMMMMMMMMP/'     `\            \
               /|MMMMMMMMMMMMMMP/          \    _!_      `.
  !            / |MMMMMMMMMMMMMM|            |.-'  `-._    \         .'\
             ;!  ;MMMMMMMMMMMMMM|            :         `-._ `-.     /  _\
             |   ;!MMMMMMMMMMMMM;            |             `-. `-.._.'/
              \   `\YMMMMM!MMMP/'\           |                `~~----'
               `-._, ``YMMMP'_.-'\         ! ;_
                 |      /         `-.,_./     7
                _/    `\,    !        /      _/
             ,-' ,  ,   /         _.'      /
            (_(_(__(__.'       !,-'        /
                              (__(__(_(_/'

Download.

I’ll let you go ahead and figure it out on your own. It’s not super-hard, but it’s fun.

Mandelbrot

In another fit of boredom I decided that it was finally time to create a mandelbrot set renderer. I originally tried to make one of these in basic, long before I had the math to do so. I was proud that I got the real axis to render, and figured it was time to complexify it. To keep things simple I decided to make it render an ASCII-art version of the set that would fit in a terminal window. The output looks like this:

                        ...............................:::::oo@@@@o::::..........
                      ...............................::::::O@@@@@@@@o:::::.......
                     .............................::::::::oO@@@@@@@@o::::::::....
                   ...........................::::OOO8ooO@O88@@@@@@8@O8o::::Oo:..
                  .......................:::::::::o8@@@@@@@@@@@@@@@@@@@@OO@@@@::.
                 ...................:::::::::::::oOO@@@@@@@@@@@@@@@@@@@@@@@@@o:::
                ................:::::::::::::::o@@@@@@@@@@@@@@@@@@@@@@@@@@@@Oo:::
               ..............::::@oo::oOoo:::ooo@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@8o:
               ...........:::::::oO@@@O@@8@OooO8@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@O::
              ..........:::::::::oO@@@@@@@@@@88@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@::
              ........:::::::8ooO8@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@o::
              .:::::::::::::oO@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@o:::
              @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@Oo::::
              .:::::::::::::oO@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@o:::
              ........:::::::8ooO8@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@o::
              ..........:::::::::oO@@@@@@@@@@88@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@::
               ...........:::::::oO@@@O@@8@OooO8@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@O::
               ..............::::@oo::oOoo:::ooo@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@8o:
                ................:::::::::::::::o@@@@@@@@@@@@@@@@@@@@@@@@@@@@Oo:::
                 ...................:::::::::::::oOO@@@@@@@@@@@@@@@@@@@@@@@@@o:::
                  .......................:::::::::o8@@@@@@@@@@@@@@@@@@@@OO@@@@::.
                   ...........................::::OOO8ooO@O88@@@@@@8@O8o::::Oo:..
                     .............................::::::::oO@@@@@@@@o::::::::....
                      ...............................::::::O@@@@@@@@o:::::.......
                        ...............................:::::oo@@@@o::::..........

The code’s far from polished and not what I like to publish, but it’s a fun thing to look at and offers you some neat abilities to poke things around and fix some pesky problems that just need clear thinking applied to them. It’s available here.

• A fitness function, that determines which individuals are most fit to reproduce
• A mutate function, that will add random chance into each generation
• A breed function that allows the best individuals to reproduce.

I ended up implementing a very simple algorithm, but it’s fairly fast and very generic – it can be easily adapted to just about any task. Sadly, I have no fascinating application just yet, but if I stumble across one, I’ll be sure to post about it.

After the jump, I’ll put up some of the code I used and a link to the script, all for your viewing pleasure.

The heart of the algorithm is the mutate and breed functions:

sub mutate{     # Mutates each allele of an individual by a random, weighted amount,
                #    controlled by certain parameters. It re-initializes if the allele
                #    is zero, and takes abs()
    my $ind = shift;
 
    for (@$ind) {
        $_ = eval($init_string) unless ($_);
        $_ += (rand(2) - 1)*($mut_weight*($_)+$mut_offset);
        $_ = -$_ unless ($_>0);
    }
 
    return 0;
}
 
sub breed{      # Breeds each individual with the top 20 percent of the population randomly
                #    by averaging each allele of the individual with the corresponding allele
                #    of a random individual in the top quintile. Note that this means that
                #    each individual breeds with, at most (and ideally), as many individuals
                #    as it has alleles, which breaks down the parent / child model slightly.
    my $indvs = shift;
    my @list;
    ($minimize) || (@list = sort {fitness($b)  fitness($a)} @$indvs);  # Sort asc. vs. desc.
    ($minimize) && (@list = sort {fitness($a)  fitness($b)} @$indvs);
 
    @list = @list[1..(int(scalar(@list)/5)+1)];
 
    for (@$indvs) {              # Iterate through individuals
        my $ind_ref = $_;
        my $i = 0;
        for (@$ind_ref) {        # Iterate through alleles
            $_ = ((@list[int(rand(length(@list)-1))])->[$i] + $_)/2;      # Average given allele with
                                                                          #    a random, fit,
                                                                          #    corresponding allele
            $_ = (rand(1)<$reset_prob?eval($init_string):$_);             # Re-init the allele some
                                                                          #    percent of the time
            $i++;
        }
    }
 
    return 0;
}

Between the comments and the code, it should be pretty clear what’s going on in here, although some of the data structures are kinda hard to get from this section. As I explain in the code:

A randomized population is created. An array holds references to the specified number of “individuals”. Each individual is an array of alleles (scalar values) that are chosen by evaluating a specified string. These are currently positive real numbers.

These references (or references to the population array) are passed as arguments to the functions each iteration.

If you’re interested, be sure to check out the code and please give me some feedback!

UPDATE: Ivan in the comments made the recommendation of using an anonymous function to intialize individuals. I made that change he recommended and updated the code.

To start, installing Finance::Quote is a breeze. As long as you have CPAN installed, just type at the command line:

$perl -MCPAN -e shell
>install Finance::Quote

From there it’s easy to get start, it’s just a matter of importing the module and creating and instance. This means the beginning of our script will look like:

#!/usr/bin/perl -w
use strict;
 
# import module
use Finance::Quote;
# create object
my $q = Finance::Quote->new();

The instance that we created has one main method: fetch(). It takes a list of arguments, with the first being a the exchange to look for quotes on, and the remaining being a list or array of ticker symbols. The Finance::Quote documentation provides the following list of valid exchanges:

australia           Australan Stock Exchange
dwsfunds            Deutsche Bank Gruppe funds
fidelity            Fidelity Investments
tiaacref            TIAA-CREF
troweprice          T. Rowe Price
europe              European Markets
canada              Canadian Markets
usa                 USA Markets
nyse                New York Stock Exchange
nasdaq              NASDAQ
uk_unit_trusts      UK Unit Trusts
vanguard            Vanguard Investments
vwd                 Vereinigte Wirtschaftsdienste GmbH

For the purposes of this script, I’m going to stick to “usa” because it covers both NYSE and NASDAQ, and those are the stocks I’m interested in.
Fetch() returns a two-dimensional hash of results. The first index (or dimension) is the ticker symbol of the stock and the second is the label for the specific piece of information. For now, the labels that we’re interested are the name of the company (‘name’) and the price (‘price’). A full listing of labels is available in the Finance::Quote documentation.
Using this information we can now build a simple script to fetch a stock quote:

#!/usr/bin/perl -w
use strict;
 
# import module
use Finance::Quote;
 
# create object
my $q = Finance::Quote->new();
 
# retrieve stock quote
my %data = $q->fetch('usa', 'GOOG');
 
# print price
print $data{'GOOG', 'price'} . "\n";

This script, although it does what it’s supposed to, isn’t very pretty. By adding a little code to read ticker symbols from the command line we can check on multiple stocks at the same time, as well as remove the need to hard-code the ticker symbols. We can also make it produce much prettier results by adding more information about the company and add some basic error checking, which yields the following script:

#!/usr/bin/perl -w
use strict;
 
# import module
use Finance::Quote;
 
# create object
my $q = Finance::Quote->new();
 
#print usage information
if (length(@ARGV) == 0) {
	print "Usage: $0 ticker1, ticker2, ... , tickerN\n";
	exit;
}
 
# get stock symbols from the command line and
# format them correctly (uppercase)
for (@ARGV){
	$_ = uc();
}
 
# retrieve stock quote
my %data = $q->fetch('usa', @ARGV);
 
# print result for each stock
for (@ARGV){
	if ($data{$_, 'success'}) {		# if getting the quote succeeded
		my $name = $data{$_, 'name'};	# build a report
		my $price = $data{$_, 'price'};
		my $message = '';
		$message .= $name . ' (' . $_ . ')';
		$message .= ' ' x(25 - length($message));
		$message .= "$$price\n";
		print $message;
	}
	else { print "Failed to retrieve quote for $_: $data{$_, 'errormsg'}\n"; }
}

This is very simple, but it does its job well:

$ quote goog msft aapl dell java
GOOGLE (GOOG)            $352.32
MICROSOFT CP (MSFT)      $22.32
APPLE INC (AAPL)         $98.23
DELL INC (DELL)          $11.99
SUN MICROSYSTEMS (JAVA)  $4.54

Download the script here