An Asteroid’s A-Comin’

At the bowling alley tonight, after discussing the stock market for a while, my friend says to me, “Did you hear about the asteroid that’s supposed to hit us tomorrow?” I thought he was kidding. So I whipped out my shiny iPhone and checked to see what Google could tell me about this. To my surprise, it told me quite a bit. According to this story from the National Geographic Society,

A boulder-size asteroid discovered just a few hours ago will become a bright fireball when it enters Earth’s atmosphere at about 10:46 eastern time tonight, astronomers announced.

Holy carp! I added the boldface over the really important bits. Let’s read it together, shall we? A boulder-size asteroid, discovered just a few hours ago… That’s really unsettling. I thought we had satellites whose sole purpose was to watch for these things. Oh well, at least it’s a small one that will most likely burn up in the atmosphere. The article did go on to say

“If the object was ten times the size [as the one detected today], we would have picked it up several days in advance,” Spahr said.

“Then we could say, OK, you guys in Africa, pick up and move 50 miles [80 kilometers].”

And people think astronomers have no sense of humor.

Why I Love Closures

I’ve been a big fan of closures for years. I was first introduced to them in Smalltalk, where they were just called blocks. Ruby has them and also calls them blocks. Java does not have them, though there are proposals (such as this one) to add them to a future version of the language. Groovy has them now, and while they aren’t as shiny as those in Ruby, they do work.

So what’s so great about closures anyway? They are blocks of code that retain links to things that were in scope when they were created, but they also have access to things that are in scope when they execute. They can be passed around, usually to methods that will execute them at a later time, in a possibly different context. That doesn’t sound all that exciting, but what is exciting is when the language in question lets you pass in a closure to do some work, and then cleans up after you when the work is done. Here’s an example of some Ruby code that I have written in tons of scripts.

count = 0

DBI.connect(*CREDENTIALS) do |con|
  File.open("results.txt", "w") do |file|
    con.select_all(sql) do |row|
      file.puts "... interesting data from the query ..."
      count += 1
    end
  end
end

puts "Total records: #{count}"

What that code does is declare a variable, count, that will be used to keep track of how many records we processed. It then connects to a database, creates a file called “results.txt,” executes a SQL select, writes some of the data from each row to the file and increments the count variable. At the end, we print out the count variable.

There are three closures in that bit of code. They begin on lines 3, 4 and 5, respectively. The first connects to the database. The second creates the output file and the third performs a SQL select, writes the output and bumps the counter. Rather concise code, don’t you think?

Do you notice anything missing from this code? Two very important things are not there: closing the file and closing the database connection. I said these bits are not there, but they really are. The block version of DBI.connect ensures that no matter how events unfold, either successfully or with an exception, the database connection will get closed. Similarly, the File.open ensures the file will get closed. This is one of the most beautiful aspects of languages that support closures.

As I said earlier, Groovy has closure support baked in. While I’m not completely thrilled with the syntax, it’s close enough to Ruby’s that it’s not bad. As I was writing this post, I was surprised to discover that Groovy’s SQL module doesn’t support a closure passed to the connection, which means you still have to worry about closing your connection when you’re done. Anyway, Groovy’s file-writing idiom looks like this

new FileWriter("results.txt").withWriter {writer ->
  writer.write("... interesting data ...")
}

I don’t really like using the -> as the closure parameter delimiter; I’d rather use a | (pipe symbol) like Smalltalk and Ruby do. Regardless of syntax differences, that’s how you write the file, but what about the SQL stuff? Since we can’t use a closure, it’s a bit more involved.

con = Sql.newInstance(url, user, pass, driver)

try
{
  new FileWriter("output.txt").withWriter {writer ->
    con.eachRow("select * from foo") {row ->
      writer.write("Foo: ${row.id}n")
    }
  }
}
finally
{
  con.close()
}

You can see that even though we can’t use a closure to ensure the database connection gets closed, there are still two closures in use. The one beginning on line five opens the file, while the one beginning on line six writes out some of the data from each row returned by the query. Pretty nifty, eh? I’m disappointed that Groovy doesn’t support passing a closure to the database connection, but maybe we’ll get that in a future version.

For comparison, here’s how you would write this program in straight Java.

Connection con = null;
Statement stmt = null;
ResultSet rs = null;

try
{
  Class.forName(driver);
  con = DriverManager.getConnection(url, user, pass);
  stmt = con.createStatement();
	
  rs = stmt.executeQuery("select * from Foo");
	
  PrintWriter writer = new PrintWriter(new FileWriter("output.txt"));

  try
  {
    while (rs.next())
    {
      writer.println("Foo: " + rs.getString("Id"));
    }
  }
  finally
  {
    writer.close();
  }			
}
catch (Exception e)
{
  e.printStackTrace();
}
finally
{
  try
  {
    if (rs != null)
    {
      rs.close();
    }
	
    if (stmt != null)
    {
      stmt.close();
    }
	
    if (con != null)
    {
      con.close();
    }
  }
  catch (Exception e)
  {
    e.printStackTrace();
  }
}

The majority of that code deals with cleaning up when the real work has been finished. I don’t know what Java’s closures will ultimately look like, or if we’ll get them at all. I’m hopeful, though, that we’ll get a decent implementation, and can then jettison code like you see above. (I know that using Hibernate or some other mapping tool can eliminate code like this, but not every situation needs that type of framework.)

Really Clever ASCII Spam

I use Gmail, which has an awesome spam filter, so I very rarely end up with real spam in my inbox. Spammers, of course, are always trying to come up with ways around spam filters, and today I ended up with what has to be the most clever spam I’ve ever seen. The subject was “Order:157-0585-035,” and even though I knew it was spam based on that and what I saw in the snippet that Gmail shows you, I opened it anyway, just out of curiosity. Here’s what I saw when I opened it

Click to enlarge

Click to enlarge

That looks pretty much like any other spam, right? But it isn’t. It’s actually ASCII art. In other words, the text of the email is only “readable” because of the patterns in the text. Here’s another screenshot, but this time the text is magnified to show you what it really looks like

Click to enlarge

Click to enlarge

Interesting, eh? Since this one is nothing but whitespace and numbers, a spam filter could actually filter this one out pretty easily, once it knew what to look for. If the spammers start interspersing letters with the numbers, it will get harder to spot. And if they can get whole words in there, too, it will be harder still. I hate spam as much as anyone, but I have to actually give these guys credit for trying.

My Google Apps Migration Is Complete

I mentioned [cref i-just-switched-to-google-apps-for-my-domain the other day] that I’d switched over to Google Apps, and had initiated a POP3 transfer of all my mail from my previous Gmail account to the new one. I’m happy to report that it finally finished. I started it POPing on Tuesday evening, 09/16/2008, and it finished some time this morning. It was pulling mail every twenty minutes or so for ten days. In case you’re interested, it pulled just under 30,000 emails over, which was just under 1 GB in size.

As I said the other day, I’d be happy to pay Google for a quick and easy migration tool. But, at least it finally finished.

When Does an HQL Typo *Not* Cause A Parse Error?

I found something interesting at work yesterday. One of our developers mentioned that when he called a certain method with various sets of parameters, he wasn’t getting back what he expected to get back, based on what he knew was in the database. I put on my sleuth hat and began my investigation.

We use Hibernate for our database layer, and we prefer to use the @NamedQuery annotation to store our queries with the entities they represent. This works out very well for us. But back to the problem. I quickly got to the appropriate .java file and inspected the query. (Obviously I’ve changed the class names, but this is essentially what I found in the file.)

select distinct f
from Foo f
left join fetch f.bar
left join fetch f.baz
lef join fetch f.plonk
where f.id = :fooId

Now, do you notice the typo? Check out line five. It says “lef” instead of “left.” When I first saw this, I thought to myself, “How can this even get parsed by Hibernate into SQL, let alone return any results.” We all work in a large room together, so I mused out loud about this problem. One of our other Really Smart Guys™ came over to have a look. He saw the typo, thought for a second and said, “‘lef’ is being treated as an alias for f.baz on the previous line.” And sure enough, he was right. Just as on the second line where you see “from Foo f,” that ‘f’ is an alias for the entity called Foo. We could have put aliases on each “left join” line, at each line’s end, had we wanted or needed to. By misspelling “left” on line five as “lef,” we unintentionally slapped an alias on the join that is on line four. Even though the query is split across multiple lines here, the HQL parser would see it as one continuous string, and after passing by “fetch f.baz” the next token it would see would be “lef,” which it would interpret as an alias for “f.baz.”

So, as far as parsing the query and translating it into SQL goes, everything is just fine. But there is still a problem caused by the misspelling. Since the parser decided that “lef” was actually an alias, the next bit that it sees is “join fetch f.plonk” which results in a regular inner join, instead of the outer join we really wanted. What this means is that for records in the Foo table who can’t be joined to records in the Plonk table, either because the key is null, or there just isn’t a record in the Plonk table that matches, those records will be excluded from the result set. That’s the behavior our developer was seeing. Changing “lef” to “left” made the whole thing work and the developer got the results he needed.

I Just Switched To Google Apps For My Domain

I’ve been using Gmail for a few years now, just having it send mail as joey@joeygibson.com, and not using the actual @gmail.com address at all. Or so I thought. Most email clients displayed email from me the way I wanted, but Outlook showed it like this

From: joey2048@gmail.com (on behalf of joey@joeygibson.com)

I knew about this back in 2006, but I thought it had been “fixed.” I put the word fixed in quotes, because spoofing headers isn’t really a correct thing to do. The thing was that most email clients showed the spoofed address, but Outlook showed the “correct” one. Anyway, it bugged me, knowing that people might be seeing my Gmail address instead of my proper address.

Enter Google Apps. I had heard about this before, but never really investigated it. I looked into it last week, and switched over on Tuesday. It’s free, and it means my Outlook problem is solved. For those who don’t know about Google Apps, you change your MX records on your DNS server to set a Google machine as your mail server. After making this change Gmail no longer need to spoof your domain in outgoing emails, since they effectively are your domain. (Don’t worry; they don’t become your web host, just your mail server.) I changed my MX records Tuesday night and then began migrating email from my old Gmail account to my new one.

Migration is one area where the experience is not so great, and I’d actually be willing to pay a bit for a better way to migrate. You’d think that migrating from one Gmail account to another would be a painless, quick and easy affair. And you would be wrong. The only way to get your mail moved is to have the new account make enough POP3 calls against your old account, pulling 200 messages at a time. I started POPing last Tuesday night (09/16/2008) and as of this moment, it’s still running. Granted, I had over 29,000 emails, which was about 900 MB of space, but still! Google ought to be able to come up with a better way to do this. Oh well, it will finish one of these days.

One thing I’d like to point out is that you need to add one more record to your DNS in order to make your Google mail SPF-compliant. I discovered this when I sent a test email from my new Gmail account to my work account. We have an Exchange server at work, and while the email did come through, the subject line had [spf] appended to it. After some checking, I saw in the headers that our mail gateway had marked it as failing an SPF check. I did some googling and found this article that explains how to set things up specifically for GoDaddy, but the general concepts should work for wherever your DNS lives. I setup the new TXT record, ran the test recommended in the article and things are good now. I just sent a test email to my work account, and the gateway must now be happy since there was no [spf] appended to the subject. There might have been a recommendation on the Google Apps setup screens about the SPF stuff, but I don’t remember seeing it.

Anyway, so far I’m happy with my choice to move to Google Apps. Besides the migration issue, the only other complaint I have is that I can’t use my @joeygibson.com id with Google Reader. I still have to use a “real” Gmail account for that. That’s essentially a minor annoyance, but it would still be nice to just jettison the old @gmail.com account altogether.

I’m Loving Grammar Girl

I may be a bit late in discovering her podcast, but last week I found out about Grammar Girl, and it’s been love ever since. For those of you who are later to the game than I, Grammar Girl is a semi-weekly podcast about English grammar. I am a self-styled Grammar Nazi, so when I found out about her podcast, I knew I had to listen. Over the course of two days this week, I listened to around 30 episodes. Each is around 5 minutes long, so you can listen to several of them during a normal commute. Her topics ranged from the correct use of hyphens, to when to use “who” and “whom,” and on to “You and I” vs. “You and Me,” plus many other topics. This stuff is fascinating, but more than that, it’s useful. She explains what the correct usage is, and then usually goes on to explain why it’s correct, which is why I think I like it so much. If you like learning things, you should definitely check her out.

I Am Full of Teh Happy

Yesterday was a good day for me on several fronts. Let me tell you them.

First, whilst searching for various things in the iTunes store, I saw that Metallica had released their new album, Death Magnetic. I had bought the first single, My Apocalypse, a couple of weeks ago when it came out, and I was excited that Metallica might be good again. I was a Metallica fan from way back in the 80’s. I loved, loved, loved every album up to and including …And Justice For All. I didn’t like “the black album” at all for several years, but then it grew on me. I thought everything after that, starting with Load, sucked out loud. I listened to the samples of the new album and immediately clicked the “Add to Cart” button. I’ve listened to the whole album about 5 times now, and it’s playing again as I write this. This album is full of awesome. It’s fast and heavy with glorious Kirk Hammett guitar solos throughout. If you liked “old” Metallica, you will love this album. Buy it. Memorize it. Love it. Standout songs include “My Apocalypse,” “Broken, Beaten & Scarred,” “The Judas Kiss” and “All Nightmare Long.” 

I was also happy to see that iTunes had finally added 0 + 2 = 1 by the very strange band NoMeansNo. I had this as a cassette, back when it was originally released, but I haven’t been able to find it in any format since. iTunes had one or two NoMeansNo records, but not this one. I have been checking periodically, but they never had it. Until yesterday. I’ve listened to it twice since buying it. It’s heavy and a bit odd, but very good. “0 + 2 = 1” and “The Valley Of the Blind” are the best songs on the record.

And if that weren’t enough musical goodness, Dar Williams’ new album, Promised Land, was also available. I think this is one of her best albums ever. I have her entire catalog, and this album has already moved to the top of the list for me. It has her signature lyrical twists, and it’s quite upbeat, with beautiful melodies. I’ve only listened to it twice since buying it, but it’s really good. My favorite songs, so far, are “It’s Alright,” “Buzzer” and “Troubled Times.”

Yes, I have eclectic musical tastes.

Next, as anyone with an iPhone knows, Apple released iPhone OS 2.1 yesterday. I was really looking forward to this update because OS 2.0.2 had lots (and lots) of problems and annoyances. The biggest problem I had was with how long it took to backup the phone. Every time I plugged it into my Mac, it would easily take over an hour to do a full sync. That’s absurd. I have less than 2 gigabytes of stuff on the iPhone and it took one-hour+. I have 60 gigabytes of stuff on my iPhone, and a sync never takes more than a few minutes. The other major annoyances were a terrible lag when using the onscreen keyboard, and the fact that when you updated an application, it didn’t stay where you put it, instead moving to the first available open spot. Not good. 

I’m very happy to report that iPhone OS 2.1 has fixed these problems, for me, anyway. A full sync is taking around five minutes, which is completely reasonable. The keyboard feels responsive, and after updating applications, they stay where I put them. Bravo, Apple. Keep the goodness coming, KTHX.

And finally, the first two discs of season 3 of Weeds arrived from Netflix. I watched the first three episodes last night, staying up far, far too late in the process. Damn, I love this show, even though I know I shouldn’t like it, if you know what I mean.

Problems With Latest Version of iTerm

I love iTerm as a replacement for Terminal.app, but this morning after letting iTerm upgrade itself to “Build 0.9.5.0909”, all my settings, profiles and bookmarks were lost. I don’t know why, but that’s what happened. I pulled back the iTerm.plist from ~/Library/Preferences using Time Machine, but that didn’t seem to fix it. I also tried to restore iTerm.app using Time Machine, but there was something funky going on with TM, so I wasn’t able to.

In the end I just reset the preferences as best I could from memory and started recreating my bookmarks. You might want to wait to upgrade.

Microfortnights Make Me Giggle

A long time ago (15+ years) I worked on a VAX. If memory serves, it was a MicroVAX 3900. I don’t recall ever seeing this unit of measure mentioned in the docs, but I may have. It’s the “microfortnight” and reading about it always makes me laugh. From the Jargon file:

microfortnight n.

1/1000000 of the fundamental unit of time in the Furlong/Firkin/Fortnight system of measurement; 1.2096 sec. (A furlong is 1/8th of a mile; a firkin is 1/4th of a barrel; the mass unit of the system is taken to be a firkin of water). The VMS operating system has a lot of tuning parameters that you can set with the SYSGEN utility, and one of these is TIMEPROMPTWAIT, the time the system will wait for an operator to set the correct date and time at boot if it realizes that the current value is bogus. This time is specified in microfortnights!

Wikipedia’s article on microfortnights has a bit more info

The joke is in having a rather large unit (fortnight) combined with a fractional SI prefix (micro) to counteract that. The practical purpose is to discourage setting such parameters without some thought. The unit was selected because the time is only approximately one second, being established by some near-infinite loops rather than a real clock unit (which isn’t active at the time), and rather than field complaints about this being “not exactly a second”, the unit was invented.

Heck, I don’t even have to be reading it to laugh about it. Just thinking about it makes me laugh.