t3knomanser's Fustian Deposits

Remy & Minna's Guide to Buying a Home, pt. 1

Sat, 07 Nov 2009 03:39:30 GMT

I'm putting Minna's name on this, but she may desire edits. Right now, we're in the process of buying our first home, and we've taken our time doing it. The main reason for taking our time: avoiding the mistakes so endemic to buying a home, and learning about the process. We initially started the homebuying process before leaving Albany, over two years ago. We've been engaged in homebuying ever since.

Homebuying

The Process

Before you can buy a home, you need to understand what you're in for. This is a high-level view of the steps:

Save
Find a place
Put in an offer
Get it inspected
Get a mortgage
Close

Each of these steps is surprisingly complicated. At every step, someone is out to rip you off. At least one someone, probably a bunch. The entire industry of real estate is built around ripping people off. So this homebuying guide is going to focus on avoiding getting ripped off, and avoiding the many pitfalls that most first-time homebuyers fall into. It's only through some good fortune that we've avoided some of them, and some education has helped us avoid the rest.

One note: this assumes that we're talking about conventional, fixed rate mortgages. For some people, an adjustable rate mortgage may make sense, but in most cases, it's a risky gamble. ARMs played a big role in the economic problems we're seeing today, so let's just avoid them and let people who have some experience buying and selling property worry about them. A first time homebuyer should never consider an ARM.

Homebuying Myths

You have to buy a home!

From now on, if someone tells you, "Renting is just throwing money away!", kick them in the groin. Hard. Renting is not throwing money away. Renting is a straightforward economic exchange. The property owner provides a service, and the renter pays a fair market value for said service. If you were homeless and paying rent, then you'd be throwing money away. You're paying money for a service.

The difference between renting and owning is that, when you own, some of your mortgage payment is becoming equity, which I'll discuss in a moment. Short version: you get to keep some of the money you pay into a mortgage, you keep none of the money you pay in rent.

This is true, however, renting gives you freedom that you don't get owning. If you don't like the place you're renting, you're stuck with it, at most, for a year. Maybe less, if there's good reasons for breaking lease. This means you can rent a place that you'll never be willing to own. For example, Minna and I have been renting a 1BR apartment, but we'd never buy a one bedroom place. In the long term, we want more space, but for now, we're willing to settle so that we can save money.

Homeowning builds equity!

This isn't exactly a myth. Yes, owning a home builds equity. The question is: is it a good way to build equity? Absolutely not.

Equity, simply put, is material wealth. It's money or property that can be turned into money. The cash in your wallet is equity. The money in your savings account is equity. Your car is equity. Anything you own with monetary value is equity.

Buying a home does allow you to build equity. Each monthly payment you make against your mortgage is split between principal (the money you borrowed) and interest (the bank's profit). The only part that is equity is the principal.

At the start of a mortgage, most of your monthly payment is interest. Very little of it goes against your principal. With each successive payment, you chip away at the principal, and slowly but surely, your interest payments drop. By the end of a mortgage, most of your payment is against the principal, very little of it is interest.

So, think about it. Which is going to build equity faster? Paying $1,000 as a mortgage payment every month, or paying $1,000 into your savings account. In the first case, most of that $1,000 is you paying interest, in the last case, the bank pays you interest.

Assume you can spend $1,000 a month for living space, and assume the best mortgage you can get offers 5% interest. Assume the best rent you can find is $500/mo. If you take out a $100,000 mortgage, your monthly payment is going to be $994, and for the first year, more than $500 of that is interest. After 14 months, the rental payment exceeds the interest payment. But go back to myth #1- you're willing to rent a place you wouldn't be willing to buy. What you can rent for $500/mo. is probably nicer than what you can mortgage for $1,000/mo. This, of course, varies by region and economic conditions. But remember: equity is just money. There are plenty of ways to save money, such as, saving money.

Note, above, I assumed that you weren't investing the money you saved by renting. If you did, it would probably beat buying for the first few years. This is important: renting, in the near term, beats buying in the same term, in most cases, in most economies. In the long term, buying may win. The NYTimes has a good rent vs. buy calculator. Plug in numbers, and you can see how long it takes for owning to beat renting. This graphic assumes a 30 year mortgage, but the advanced settings let you change that.

It's important to note: if Minna and I were happy living in a one bedroom apartment for the rest of our lives, owning would never beat renting. We would never actually save money by owning, unless rent or property value were increasing at much better than inflation.

The Intangibles make it worthwhile

This isn't a myth so much as personal preference. If you own a place, you can paint the walls pink. You can change the light fixtures. You can drill a hole in a wall, line it with silicone, and fuck the shit out of it, if you want. If you rent, you can't make many modifications, in general. It varies by landlord (my landlord in Albany didn't care what you did so long as it was reverted to rentable condition).

If you really want to have pink walls that you can fuck, then by all means, buy. But if you don't really care what color the walls are, don't figure intangibles into the equation.

Your Realtor is there to help

It doesn't get much more false than this. When you want to buy a place, you will often retain a buyer's agent to help you in the transaction. You probably should have a buyer's agent. But you should never, ever, trust a buyer's agent.

You won't pay a dime for retaining a buyer's agent, in most cases. Buyer's agents are paid in commission by the seller. In PA, for example, 6% of the sales price is paid in commission. 3% to the seller's agent, and 3% to the buyer's agent.

The more you spend, the more your agent makes. At a fundamental level, your interests (the best property for the lowest price) are fundamentally opposed to your agent's interests (the more you spend, the more they take home). Until you've found a place you totally want to buy because it is absolutely perfect, do not listen to your agent. You should pick the places you want to see, you should make the appointments for showings. You decide what you think a fair offer is based on your own research (tools like Zillow are invaluable for this). Know, intimately, how much you're willing to spend, and play with your preferred mortgage calculator to understand what you can afford.

Not only that, but there's a knowledge disparity. They buy and sell houses every day. You are going to do it a handful of times in your entire life. This gives a realtor an incredible advantage over you. You need their knowledge, but they also are dealing from a position of power: they can screw you up the ass and you won't even know what just happened. The only way to protect yourself is research research research.

It's also worth mentioning exclusive buyer's agents- they work for a flat fee, not commission, and never sell property. If you're worried about potential conflicts of interest, this is an option. We considered going this route, but we found an agent we liked, and decided to use her. You may save money doing this- since an EBA doesn't take commission, the seller doesn't have to pay them as much, and may be willing to take that out of the sale price.

Just get a starter home

A starter home is about the worst thing you can possibly do, financially. The idea of a "starter home", is that you buy a property that you don't want, so that you can "build equity" and sell it in a few years to buy what you do want. This assumes that your property increases or holds its value, that you've actually chipped away at the principal owed, and that someone is willing to buy your property. Plus, selling a home isn't free; you have to pay some percent of the sale price to the realtors, plus transfer taxes (2% of the sale price, split between the buyer and seller, but it does vary by region), and depending how desperate you are to sell, meeting some of the seller's demands for repairs/enhancements.

As a rule, don't buy a property unless you're planning to be there for 10 years. Otherwise, you're gambling that the real estate market is going to be in the state you hope it's in when you try to sell.

You can save on taxes!

Yes, you can. Every dollar you spend in interest is tax deductible. But they're still dollars you spend. If you're teetering on the edge of one tax bracket, the loan interest may send you into a lower tax bracket, and save you money. But with each loan payment, the amount of interest you owe is going to decrease. In the first year, you may pay $1,000 in interest, but by the 10th year, you'll only be paying $200 in interest.

And either way, you're still paying interest. You may as well donate to charity instead, if you just want to reduce your income to save on taxes. It's the same deal, either way. The difference is who gets the money: a charity that means something to you, or the bank.

A Thirty Year Mortgage is fine

Again, not exactly a myth, but you can save a great deal with a 15 or 20 year mortgage. In the short term, your payments will be higher, but in the long term, you will see a significant cost savings. Minna and I need to borrow $159,950 to buy the place we're interested in.

Current rates for a 15 year mortgage mean we will have payments of $1,200 with 4 3/8% interest (4.375%). At the end of the loan, we would have paid $159,950 against the principal, and $58,464 in interest, for a total payment of $218,000.

In a thirty year mortgage, our monthly payment would only be $860, but by the end of the loan (at 5% interest), we would have paid $309,000. Short term savings turn into long term losses. Certainly, one could invest the difference (nearly $400/mo.) into an index fund and expect about 10% returns over the course of the loan- at the risk of being leveraged for 30 years. And there's always the selling problem- if the price of your home tanks, due to market fluctuations, you may be up shit creek when you try to sell. In a 15 year mortgage, you build up equity much faster, for less expense to yourself, and hence have more freedom to sell.

Your down payment controls how much you're going to need to borrow, so this leads right into the:

Steps of buying a home

Save!

If you wanted to buy in the neighborhood I live in, a basic two bedroom home runs more than $200K. That means, just to buy the home with 20% down, you need at least $40,000. If you don't have 20% down, you have to pay mortgage insurance, which really is throwing money away. Every region, of course, is different. On the West Coast, $200K would buy you a small closet, in Manhattan, it might get you a mailbox. In a more rural area, $200K is a mansion.

Before you buy a home, you need to research how much a home you like goes for in your area. It varies a great deal, based on geography. Only you know what you like, and only you know what you're looking to spend each month. Don't assume your income is going to increase every year, either- economic contractions like the one we're in put lie to that strategy.

In addition to that 20% down ($40K in my example), you're going to need to pay closing costs (a generous assumption is 25% of your down payment, so in this example, another $10K). This means, to buy a $200,000 home, you need at least $50,000 in the bank. And you don't want to clear out your entire savings- you may have an emergency or want to improve the home you purchase.

Let's say you buy a $200,000 place and get a 15 year mortgage. If you only did the minimum down payment, you're facing a $1,200/mo. payment. If you only have $1,000 budgeted for housing, that may be too much to spend. But if you did a 30% down payment, you'd only be facing a $1,062 monthly payment: much closer to your goal.

The more you save, the easier it is to buy a house. And the more you have on hand, the easier it is for you to get the home you want, instead of settling for a "starter home".

Find a Place

Once upon a time, a realtor was your gateway to the wild, wonderful world of the Multi-List Service, MLS. If you wanted to find a property for sale, your realtor would have to do the research and get back to you with listings. No more!

First, any realtor worth having can give you access to the MLS system online, and you'll be able to do the search yourself. Second, many tools have access to MLS already, like Trulia, which I used a great deal during my home search. And don't overlook Craigslist either! (I actually found this place on Craigslist first)

Trulia lags by a few days, and depending on how hungry the market is, that time may make the difference. Honestly, though, you don't want to get in a bidding war, so that lag just means you notice things that haven't already gotten scooped up.

If you've already secured a buyer's agent, you can get them to show you properties you're interested in. If you haven't, you can contact the selling agent (attached to the listing), or see if there's an open house. What worked well for us, was talking to the selling agents. We met one we really liked, so when we found a place to buy, we called her to represent us in the transaction.

How do you know which place is right for you? Obviously, price is a big factor, but most important: you have to like the place. Sadly, it's harder to know that than you think it is. If you let yourself get sucked up into "I MUST BUY A HOUSE" mode, you'll find yourself liking things that, a few months later, you would regret. Like somebody desperate for a date, you settle and then find yourself stuck with someone that you can't get rid of.

The best thing to do is look at a bunch of places. Your brain is instantly going to give you an emotional vibe off of them. Listen to that vibe, and then cross-check it with your common sense. Spend some time deciding how much time and effort you're willing to put into making a place "yours", the "must have" features, etc. Jonah Leher's book, How We Decide talks a bunch about how people choose homes, and surprisingly, an emotional decision usually makes people the happiest.

Putting an Offer In

Okay, you've found a place. You love it. But real estate isn't like the supermarket- you can't just drop coin and walk away with it. Putting in an offer is a formal ritual, and realtors are the priests.

Research the local market. While your realtor should know what fair prices are for property in the area, remember what I said earlier: their interests and yours are not the same. Listen to them, but don't trust them. Keep in mind that they'd rather close the transaction faster than cheaper, and you'll be all right.

Your local county website will likely have valuable information about the property, including its last tax assessment and the previous purchase price. Knowing what the seller paid, and when, gives you a feel for how much of a profit they're trying to turn. They've likely put at least some work into it, so if they're trying to turn a big profit, ask your realtor to find out what they've done. They may be trying to turn a big profit, but they may just be trying to recoup big expenses.

Regardless of all this, if you've done your homework, you've got an intuitive feel for what a place like this should cost. Go with your instinct, run it past the realtor, and then decide how much under that you think you should offer. Usually, if there's already a bid on the place, the selling agent will let you know. Don't get involved in a bidding war. My advice, and you can decide if you want to apply this or not, is not to waste time. Decide what you're willing to pay, offer that, and say it's your "last and best". You won't move up above that point, so it's up to the other prospective buyer to beat your offer. This prevents the seller from being able to keep going back and forth, getting you up a grand here, them up a grand there, until you're paying more than you want. Maybe you like negotiating, but me, I just wanted to get it over with. If there's another offer on the table, just go in with your best offer you're willing to make, and let the chips fall where they may.

Your realtor will help you assemble the offer, and at this point, it is very valuable to be pre-approved for a mortgage. Call your bank, or any other lender, and get them to run you through the process. You aren't committed to anything, and neither are they. They've just stated that they'd theoretically be willing to lend you $X amount with Y% interest. A pre-approval letter shows the seller that you're not wasting their time and could probably get the mortgage. This does require a credit check, so only do it when you're putting your offer in.

The other thing you need to do is provide "hand money". Again, the seller needs to know that you're not wasting their time. So, when you submit your offer, you include a check for 5-10% of your offer. This hand money is your commitment to the deal. It gets put into an escrow account, and will be part of your down payment when you close, if your offer is accepted. If your offer isn't accepted, the check just gets ripped up, and nobody gets your money.

But, if you flake out and walk away from the deal for no good reason after the seller accepts your offer, the seller gets your hand money. There are contingencies that let you walk away: if you actually can't get a mortgage, if there's a deal-breaker in the home inspection, for example. But you must make an honest effort to see it through to the end, or you lose your hand money.

Despite that, once the seller accepts your offer, it's easier for you to walk away than them. But it's still not a done deal.

The remaining steps, getting an inspection, getting a mortgage, and closing, are ones I haven't done yet. But, once your offer has been accepted, you have a clock ticking to accomplish two major tasks: get an inspection and apply for a mortgage.

Depending on your local rules, and the stuff you put in your offer, you're going to have about 10-15 days to get these tasks done, so it is important to jump on them quickly. I applied for a mortgage today, but it takes days or weeks to get an approval back (depends on the bank, your credit rating, and how their underwriting process works). But when you apply, you do lock in your interest rate, usually for 60 days. This means, when we're approved, we're locked in for a 4.375% interest rate which is suhweet.

As for the inspection, you do want to find an American Society of Home Inspectors (ASHI) certified inspector, and there's a lot of things to consider in that step.

Everything I've covered so far in this post has been stuff I've actually done. Since I haven't actually gotten approved for a mortgage or had a home inspected, I will refrain from commenting further on them, for now. Expect a part two next week, discussing the inspection, and future installments as I cover the remaining steps.

Well, that was sudden

Tue, 03 Nov 2009 22:03:37 GMT

Last week, we noticed a listing for a property that was interesting. Our sort of place, from the pictures. Not perfectly located, but well located for us. It's one mile from all the little shops we like, and closer to even some of the other shops we like. Near a park, on a quiet, quiet street.

We saw an open house on Sunday, got a buyer's agent and saw it again on Monday night, went back to her office, wrote up an offer, and submitted it. It was accepted today. So… we're buying a house, in Shadyside. 2BR, 1.5 bath. The first floor is a gorgeous open plan space. Kitchen is brand new; the stove has never even been used. A new patio was installed. Furnace, hot water, all new. The upstairs isn't quite as nice, but certainly nice enough. The washer and dryer are antiques and will need to be replaced.

It's not under contract with us yet; we need the realtor to bring over some paperwork. Tomorrow I'll try and hire an inspector, and then we'll work on getting the mortgage. We're excited, and a little trepidacious.

Email to Notational Velocity : an Idea Mining Solution

Sun, 25 Oct 2009 23:39:40 GMT

I've discussed Idea Mining before: every idea that pops into your head, jot it down. When you're looking for inspiration, or have some time to develop some idea, poke around in your mine and see what grabs you.

I started using Evernote, which had the advantage of syncing, but recently switched to Notational Velocity. It lacks syncing, but is much more lightweight. When I'm sitting at my computer, the amount of friction in jotting down new notes is minimized. It's a great tool, but by switching to it, I gave up syncing. This means, for example, I can't jot ideas on my phone and have them show up on my computer.

At least, not by default. But I'm not going to be thwarted by such details. So, I sat down, banged out some settings and scripts that let you sync emails in your GMail account to Notational Velocity.

What follows is a high level overview of the steps. It's not hard to do, although it took a little doing to figure out.

You're going to need the following things:

OSX Leopard or later
Developer tools installed (on your OSX install disk, or downloaded from Apple)
Notational Velocity
A Gmail account with IMAP enabled

Configure Notational Velocity

Once you have Notational Velocity installed, you're going to need to make one settings change: on the Notes tab of the Preferences window, change "Store and read notes on disk as:" to Plain Text Files. Note also the Folder that you store your notes in on this pane.

Configure Gmail

You need to enable IMAP from the settings pane.

You also need to configure a filter for a pseudo-address. In the filter section of settings, create a filter along these lines:

Matches: to:([your email]+mine@gmail.com)
Do this: Skip Inbox, Apply label "Notes"

This means that, every time you send an email to [your address]+mine@gmail.com, it's actually going to come to your account, but get labeled with Notes (which is a folder in IMAP).

Get SSH Prepared

This step, sadly, does involve a little command-lineage. You can only connect to Gmail's IMAP interface over SSH, and fetchmail needs the certificates where it can find them for this to work.

From /Applications/Utilities fire up Terminal, and type the following:
openssl s_client -connect imap.gmail.com:993 -showcerts

This command will output a big pile of of text, but up at the top, you'll see a large block of output that starts with:

-----BEGIN CERTIFICATE-----
(a big bunch of characters here)
-----END CERTIFICATE-----

(there are two of these blocks, you want the first one, which is actually for imap.gmail.com.

Copy that text (including the BEGIN and END lines) and save it to a file called ~/.ssl/certs/gmailimap.pem. You will likely need to create those directories before you can save it.

You need one more certificate, which you can download from here (where it says "Equifax Secure Certificate Authority (Base-64 encoded X.509"), and save it as ~/.ssl/certs/equifax.pem.

To make these certs usable by fetchmail, you have to run one last command at the command line:
c_rehash ~/.ssl/certs.

This step is a lot easier if you know your way around the command line. The bulk of this SSH section was adapted from here.

Notes.pl

Download this file and save it in ~/Scripts (a directory you'll need to create). You'll also need to make it executable, which involves another command-line bit:
chmod +x ~/Scripts/notes.pl

Open that file in a text editor and change the first two lines per the instructions in the file. Basically, you need to tell it where to find your mail (/var/mail/[your mac username]) and your Notes folder (remember I told you to keep track of that).

This script parses your local Unix mail (deep inside OSX, it has nothing to do with Mail.app or Thunderbird or anything you normally use for email) and turns what's there into notes in Notational Velocity.

We're almost done. The last step is to…

Configure Fetchmail

If you're using Snow Leopard or Leopard, fetchmail is already installed. It's a powerful tool for one task: downloading email from remote systems and putting them on yours. Specifically, into the local Unix mail.

Fetchmail works off of a configuration file. Create a file called ~/.fetchmailrc. In this file, you'll need to paste these settings:

set daemon 1300 #the number is the number of seconds between polling attempts

poll imap.gmail.com proto IMAP port 993 user "youraddress@gmail.com" password "yourpassword" keep folder "Notes" ssl mda "/usr/bin/procmail -d %T" postconnect "~/Scripts/notes.pl && > /var/mail/yourmacusername"

Change "youraddress" and "yourpassword" to your Gmail username and password. Change "yourmacusername" to whatever your username on your Mac is. It may contain spaces, in which case you'll need to escape them like so: /var/mail/Remy\ Porter. Note, that is a "\" followed by a space.

To test this much, send yourself some emails that should be turned into notes (don't forget the +mine!), and then, at the command prompt, simply type the command:
fetchmail.

Once it works, we want to make sure it runs every time you log on, so download this file and put it in your scripts folder. From the command line, type:
chmod +x ~/Scripts/fetchmail.command.

Now, in your OSX System Preferences, go to Accounts, select yours, and add a Login Item that runs fetchmail.command.

TADA!

Using It

Now that you've got it set up, send an email to youraccount+mine@gmail.com. The subject line is the note title, in Notational Velocity, and the body is the content of the note. It'll sync however frequently you want it to, by modifying the indicated line in your .fetchmailrc.

Also, if you want to be able to read notes on your phone, you could use something like Dropbox, which is what I do. Also, the way this works is to append to an existing note. There's no real way to edit.

Warnings

I'm not super sure how this will work if you have multiple accounts on your Mac that you want to use this. You may need to instead configure fetchmail.command to run as a system startup item, and then there may be issues with using "~/Scripts"- you may need to use an absolute path. The Googles are your friend, if that's what you decide you want to do.

Joseph Campbell Killed Genre Fiction

Thu, 15 Oct 2009 01:32:34 GMT

This has been irking me for a long time, but in light of Stross's anti-Star Trek rant and this rant about sexist/feminist influences in SF, I think it's time for me to put this idea down on page.

Joseph Campbell killed genre fiction. Mostly SF, but genre fiction in general. It wasn't his fault. Joseph Campbell did something fairly brilliant: he saw a way to take traditional stories and build them together into a single taxonomy. He found elements that crossed cultures (although it's definitely got a Western bias) and synthesized a sort of "grand unified theory" of mythology and literature.

I'm not here to debate the veracity of this idea. Campbell is right in enough ways that his ideas have explanatory power, even if there are cases that fall outside of his scope.

Sometime between when he had those ideas and today, people blew his findings out of proportion. Repeatedly, I've dealt with people that use Campbell to defend unoriginal literature, claiming he proved that all stories were the same anyway. Those not as well read might not name-drop Campbell, but they like to trot out the "finite number of stories" canard.

It's simply not true. While there are some commonly used story templates, there are as many unique stories as there are ways to arrange words into meaningful sentences.

And this gets us back to the death of genre fiction. Modern science fiction, especially in TV and movies, takes the stance that stories are the same, and are inherently about the characters, and that science fiction or fantasy is just another costume to dress the same story up in. Some might consider this the "literary" tradition. In this approach, all the trappings of the setting exist only to drive up the special effect's budget and provide trailer-fodder, while the story could be told equally well in any other setting. Star Wars is the most obvious example of this: sword and sorcery fare in spaceships. But pretty much every Star Trek episode from some point in the middle of the TNG era also fits the bill.

What you find is that the setting is just props and costumes. For some stories, that's fine, but genre fiction used to be about exploring ideas. Whether it's taking a modern idea and casting it in a different light by analogy or imagining a world drastically altered by some new technology or idea, in classical genre fiction, it was always the ideas that took center stage.

In some cases, notably the likes of Heinlein and Asimov, the result was cardboard characters that generally could be grouped into a handful of niches, few of which ever truly leapt off the page. At the same time, these books are rife with ideas. Different social codes, technologies that reshape society in strange ways.

I'm not trying to say that character driven drama is bad and idea driven drama is good. But the best genre fiction can do both. Sterling's Holy Fire builds a fascinating world, and then shows it to us through the eyes of a newly rejuvenated octogenarian learning about the gerontocracy from the ground up. Vonnegut, of course, was a master of mixing bizarre ideas with likable characters.

My main complaint is that the pendulum has swung away from "big ideas", at least in mainstream sci-fi. It's hardly dead, but sometimes, it feels like somebody slit genre fiction's throat.

"Breathtaking" code

Tue, 13 Oct 2009 23:51:46 GMT

Words like "breathtaking", "stunning", "astounding", "amazing", and "incredible" are not positive words. Oh, they're often used in a positive context, but in reality, they only convey a sense of wonder and disbelief. They can be applied to something horrifying just as easily as something delightful.

Such has been my experience over the past few days.

Once upon a time, some fragment of the giant behemoth I work for needed an application to manage some inventory tracking and similar tasks. They found an external contractor that would make it for them. In that era, the app was built in VB6 with an MS Access back end. Horrible, yes, but hey, it was the 90s.

Well, now it's time to upgrade. So they go back to the same contractor. "Hey, we want it to be .NET and use SQL Server. Here's a check."

Well, it wasn't that easy, of course. There are policies. Checkpoints. Oversight. I was part of the oversight, and my job was to get the application through our "gates". Since this was a purchased application, we didn't care if it was terribly well designed, so long as it worked. We weren't going to have to maintain it. So we got some design diagrams from the contractor, sanity checked them, and then called it a day.

More fool I.

Last week, I hear, "Hey, that application- they're not satisfied with the support from the vendor. They want to bring it internal. Hey, Remy, find out what you need to do to make it fit with our standards."

Now, when we develop an app internally, or have a contractor develop an app that we intend to support, the auditing requirements are much stricter. We don't care if an outside vendor has to suffer under horrible support issues, but if we're wasting time on stupid support work, we're unhappy.

So, the first step then, would be to audit the application via a code review. I sat down in a room with three other developers, we pulled up the code, and we started skimming through it.

It was breathtaking.

The first, most glaring thing, was that the actual design and the provided diagram had no relationship to each other. That didn't bode well. And it was all downhill from there. As a sampling of some of the amazing things we saw:

One window in the application has all of the business and data access code bundled up in it. It's like they took all the functions and dumped them into a bucket.
But not all functions. For example, when that main window wants to log an error message, it calls out to a class called "Utility". When it does that, the Utility class makes a call against the main window. Not only is the code just a disorganized pile in a big bucket, but sometimes the bucket says, "Hey, you gotta go look in another bucket to find that!" and when you do, the other bucket says, "Nope, it's actually back out in the original bucket."
All of the database access stuff is hard coded strings. With no SQL injection protections.
It stores passwords in plaintext
The most stunning thing, however, is what the application does every time it launches. Every time the application launches, it attempts to add columns to a bunch of tables. If those columns already exist, this would cause an error, so it just ignores the errors. Since, once the app has run once, the columns will exist, that means every time a user runs the app, it's trying to add columns to tables for no damn reason.

There's more, but I won't bore you with the details. I wrote a 2,200 word code review document. Normally, these documents follow this form: "File X, Line 128, variable strFoo should be named foo, we don't use Hungarian Notation." For even large projects, they don't tend to get that long, just because they're so terse. This, this was a 5 page essay. I couldn't even call out flaws by line numbers, just because the developer was so clearly incompetent. I've dealt with some bad code in my day, but this is just special.

There's a punchline. They started development using .NET 1.x, and then partway through switched to .NET 2.0. In .NET 1.x, there was no built in FTP functionality. If you wanted to do FTP, you needed to write your own class to do it, or steal some. Microsoft's developer documentation site, MSDN, had an MSFTP code sample, which demonstrated how one could implement their own FTP code. The developer copypasted this- despite the fact that .NET 2.0 had all the functions he needed- and included a class called "MSFTP" in his project.

He couldn't just use the class as it was, however. There were some quirks in our FTP server it didn't handle. And while he was at it, he added a "Dispose" method to handle cleanup, replacing Microsoft's use of the "Finalize" method. This is actually a good .NET technique for technical reasons that are irrelevant here, so he obviously read a book. He didn't read it closely enough, because his Dispose is actually implemented wrong, but that's neither here nor there.

In the comment above his "Dispose" method, he had the gall to include this:
'M$ should have implemented this. Man, this code is sloppy.

When I read that, I just started cracking up.

Until I saw how much my company paid for this. And then I started crying.

Good seats

Mon, 05 Oct 2009 05:09:41 GMT

Good seats
Originally uploaded by t3knomanser.

Weekend Revolutionaries

Fri, 25 Sep 2009 16:16:34 GMT

The G20 protests haven't gotten too out of hand, but there's a lot of broken windows. Thus far, only 66 arrests and 6 injuries, nothing serious or life threatening from what I can get from the news.

But there was something else I caught on the news that gave me pause: some of these kids out on the streets are under the impression that they're fighting a revolution. They wear masks because the government could "vanish" them away into a prison. The various G-20 protest organizing sites use the terminology "revolution" and "images of revolt".

What an incredible degree of inflated self-importance. These kids show up for their "revolution" for a few days, throw some rocks, pretend that they've done something and taken a significant risk to life and liberty, and then go back to their lives. A few months later, some other protest-worthy event comes up, rinse and repeat.

They're revolutionaries like a guy driving a Prius is an environmentalist. As someone pointed out, Pittsburgh was home to the Homestead labor strikes. Steel workers wanted to unionize and it turned into a gun battle between them and the Pinkertons. Cannons were involved.

That puts a little pepper-gas in perspective, doesn't it?

With all that in mind, I would like to clarify a few things for our visitors:

Protesting is awesome. Seriously, enjoy the fact that you live in a country where this can happen. The Beijing G20 Summit in 2005 didn't see these kinds of protests- and we all know why.
Permits are awesome too. Chants of "Who's streets? Our streets!" resonate on a deeply emotional level, but the reality is large clusters of people pose a safety risk. So when the police attempt to disperse the crowd, they're not doing it to silence anyone, they're doing it because they are concerned about property damage, injury, and the ability of emergency vehicles to get through. Also, people actually live and work on those streets, and they have just as much right to use them as a protest does. There is no conspiracy to silence you, and requiring permits for large gatherings is not an unreasonable requirement.
There is no revolution. Don't kid yourself. If there were a revolution, you wouldn't do this for the few days when G20 was in town, but every day. You wouldn't be facing down pepper-gas and rubber bullets. You'd be dealing with real bullets, made of metal, that would kill and maim you.
The idea that the protesters and the police are adversaries is a dangerous and wrong idea. Everyone is responsible for keeping an event like this safe. Only someone deficient in basic humanity would actively wish harm on the protesters or the police.
Property damage is not communication. It's shitheaded jackassery. If you want to achieve social change, you need people to build a consensus. Smashing windows isn't a good way to do that.

I could go on, but this covers the key points. As I said, dont be a jackass, and you'll go far.

Also, Pamela's was one of the places that got its windows smashed. That's just not cool man, not cool at all. Pamela's is a Pittsburgh institution and home of the best pancakes in America. Not. Cool.

I'm Local Famous

Tue, 22 Sep 2009 22:51:54 GMT

Last week, I put together DontBeAG20Jackass.com, and started sharing it with the world. Today, my phone has been ringing left and right as local news agencies call to ask about interviews and the like.

Today, I was on the local TV station, with this interview, and tomorrow I'll be on the associated radio station for a five minute interview in the morning. The local newspaper has a brief blurb about me on their paywalled-site (I'm not sure how they expect to make money, but they're trying).

Based on watching the traffic and such from the site, I'm going to put together an after-action-summary this weekend, because I learned a few things doing this.

I Still Live: Updates

Fri, 18 Sep 2009 20:24:15 GMT

I am still alive. I have not fallen off the Internet. Since the last time I actually got things together to post, life has continued unabated, and all sorts of fun and interesting things happened, mostly of personal interest.

A few things of a more global interest:
HitTrack, my first iPhone application, got approved and now languishes in the iTunes store. It's nothing spectacular, but it's a workable D&D hit point tracker application, with some nicely done eye candy.

Of more current interest, I assembled Don't be a G20 Jackass, a site that allows people in and around Pittsburgh to sign a polite request that, when the G20 protesters show up next week, that they keep in mind that people actually live here, and would very much like it if the city government doesn't have to pay huge lump sums to clean up and repair damage. I don't think it'll matter, but it was a fun, easy project, and at least grants the illusion of efficacy.

Also, I have a laptop for sale: a Core Duo MacBookPro, 2GB RAM, 2GHz processor, 100GB HDD. Let me know if you're interested.

Morons oppose security

Sat, 11 Jul 2009 23:37:51 GMT

Today, ImageShack was hacked by some anonymous children seeking to get their manifesto out there. If nothing else, it does a good job of showing off that the differently-abled can still master basic computer security tools.

Actually, I'm pretty sure this is a prank. They ramble on and on about the evils of "full disclosure". But nobody in the security industry advocates full disclosure. To the contrary, responsible disclosure is the most common approach security researchers take. When a researcher identifies a vulnerability in a software product, the first thing they do is approach the vendor to alert them to it. The vendor is given a reasonable period of time to work out a patch, before the details are revealed to the public. In some cases, the only people who get full details are the people responsible for the software- everyone else gets alerted to the vulnerability.

But in any case, the objection is: why disclose vulnerabilities at all? Why not keep them a secret, thus keeping the bad guys from exploiting them?

Because many of the bad guys are pretty smart. If I'm smart enough to find a security vulnerability, there's good odds that there's a bad guy who is also smart enough to find it. So when a vulnerability is discovered, it's good to assume that the bad guys already know about it too. Maybe they haven't exploited it yet, but they could be working on it. You certainly don't know what they know or what they're doing, and you can't control it either.

Keeping it a secret doesn't keep the bad guys from finding it, but it does keep the good guys ignorant. If the good guys don't know about the vulnerability, they don't know what they can do to defend themselves. Until the vulnerability is patched, they're sitting there exposed- and without disclosure, in blissful ignorance.

Okay, but why full disclosure? Isn't it enough to say, "Hey, I found a vulnerability?" Nope, because the software vendor's response is: "No you didn't. Prove it." And this is where the "science" aspect of computer science kicks in. The researcher has just published the results of an experiment: "I performed test X, and got result Y, which means I just pwned this system." Merely publishing the results is not enough to prove you've done it- you also need to show your work. You have to distribute your methodology, so other people can replicate your work.

The sucky thing, of course, is that, if the bad guys haven't found out about the vulnerability, they certainly have now. Which brings us back to responsible disclosure: a researcher should give a vendor a window in which to resolve the problem. The fact that this window closes is vital to keeping the vendors honest. There have been plenty of occasions in which researchers have alerted software vendors to a vulnerability, and the vendor has ignored them.

Fixing bugs costs money, and fixing security vulnerabilities looks bad- like they're admitting weakness. A company, acting on its short-term best interests, may choose to ignore reports from the security community. But if the researcher discloses the vulnerability to the public- well, now the vendor has to act.

On critique

Tue, 07 Jul 2009 23:24:39 GMT

With the recent spate of money-making, blockbuster films that are heaping piles of suck, people complain that it's unfair to call a movie bad- it's just opinion. There are no objective standards.

There are objective standards. When picking a hammer, many of the details may be a matter of the user's taste, if the hammer is unable to drive nails, we would all agree that it's a bad hammer. If your aunt knits you a sweater with no arms, while it was a sweet sentiment, we would all agree that it's a bad sweater. If someone were to take a great number of words, selected at random, put them onto page and call it a novel, I think we would all agree that it's a bad novel.

While there is a lot of room for opinion, in any creative endeavor, there are metrics for success and failure. There are, mostly qualitative, but some quantitative measures for whether a creative work is "good" or "bad".

My purpose here is not to write a treatise on critique, which I'm sure has been done better by someone else, who has devoted a great deal more time and effort to the subject. But as a critical dilettante, here's how I approach a critical examination of creative works- regardless of medium.

The Message

A creative work exists to communicate something. The creator(s) have something they would like to impart to the audience. In a work of any complexity, they probably have many things they would like to impart to the audience. The first part of critiquing something is examining the message and deciding if this is a message that is interesting and meaningful.

There's certainly a great deal of room for subjectivity in this category. Most people would probably not enjoy a film that takes the stance, "Hitler was awesome". But, even if one disagrees with the message, one can still evaluate the message objectively. Is it an interesting message? Is it something original, or a novel insight into something familiar?

I'm not claiming that good works have to have some intense or deeply meaningful message. If the purpose of the work is merely to entertain, it still wants to communicate something more than mere entertainment to the audience- photographs have subjects, novels have plots, and so on.

So, the first to questions I ask myself are: what is the message, and is this message interesting or notable in some way?

Communication

Once we've identified the message, the next question we have to examine is: was the message communicated clearly? Did the audience understand it? Was it comprehensible to begin with? What would have made it more clear?

This is largely qualitative, but there are certainly some objective measures. Whether the target audience understood it can be measured pretty objectively by asking them. Things that don't support the message can be marked as superfluous.

Technical Merit

Each medium has its own methods, metaphors and techniques. Things that work in novels can't be done in film, painting has established practices. Given the nature of the medium, the final question is, Was this work technically competent? Someone can perform a piece on the violin, express it beautifully, but if they lack technique, this can be identified by an experience violinist (I, for example, lack technique). If a filmmaker or photographer ignores the basic principles of composition, this can be identified.

Of all of my categories of critique, this is the most objective. Every medium has principles and practices that are established through trial and error- people have found what works. It is possible that someone can throw those rules out and be successful at communicating, but it's pretty unlikely. (Andy Warhol threw out the rulebook, and while he was successful in the sense that he's mysteriously popular, he didn't really communicate anything but how little he had to say.)

And that's true for all of these categories. Maybe the story has no real point, but is entertainingly told. Or the film has no budget and looks awful, but carries you with its interesting ideas.

Some examples

This was prompted by an online debate over the merits of Michael Bay films. I love ridiculous action movies, but hate his movies. Since I haven't suffered through Transformers 2, lets look at the first one.

What was the message of Transformers? Well, its primary purpose was clearly to entertain, and to entertain as a vehicle for robot battles and explosions. Nothing wrong with that, but nothing particularly exciting either. The story moves forward on the back of a McGuffin, and events happen, none of them terribly significant.

Did it communicate this message well? Not terribly. First and foremost, the main reason for seeing the movie was robotic action sequences. There weren't many of those. What little action there was confused and annoyed, more than entertained. There were plot cul de sacs, like the NSA/hacker subplot which served no purpose at all, except to put another attractive actress in the film, I guess. There was a chain of unfunny physical comedy (I love physical comedy, but this was just bad) But, honestly, pretty much everything was a plot cul de sac. By the end of the film, I was aware that things had happened, and I didn't care one whit.

What about the technical merits? People hold Bay up as the master of the special-effects spectacle. The reality couldn't be farther from the truth. Every special effects moment was specially designed to be incomprehensible. The robots didn't look terribly much like robots, and they had no real structure or proportion to their design. The action sequences did their best to be disorienting, which isn't really a good thing to do to your audience. There was no sense of structure to any of the camera work, and it really felt like they weren't planned at all. The best analogy was that it's like fingerpainting with special effects.

Contrast this with a film like Crank, which I maintain is the gold-standard for action films. In terms of their core message, they're pretty much identical. Once again, everything about the film, from the thin plot on, is about getting into action sequences. Events happen, but there's not a great deal of significance in any of them.

But in execution, the differences really start to show up. First off, Crank keeps on message for the vast majority of the film. Each scene moves from action sequence to action sequence, with the odd ridiculous sex scene thrown in for fun. Plenty of physical comedy, often grotesque. While the main character is just another gruff talking growly scowly manly-man, he doesn't look the part, and that adds character. The supporting cast, the doctor and his girlfriend, carry all of the non-actiony bits, and everything is delivered with a sense of comic timing all the way around.

Technically, it does its job. Ridiculous fight scenes move quickly, and are pretty exciting all the way through. The camera is always in the right place, you always feel grounded in what's going on. It's not a virtuoso violinist playing through a Mendelssohn concerto, but a punk rocker laying down a bass-line. There's technique there, but it's not the focus.

I'm no art school film snob. I'm not a snob about most things. Sometimes, you want a nice filet mingion served with sauteed truffles and a nutty red with a strong finish. Other times, you want a hamburger. But just because you want a hamburger doesn't mean you're going to fill your gob with the predigested slop at McDonalds and say it's delicious.

Idea Mining

Sat, 04 Jul 2009 03:15:20 GMT

So, I've been doing "this thing", the thing being "idea mining". I'm the sort of person that gets all sorts of random ideas during the day. At the time, they always strike me as interesting, but they're always transient. Idea appears, it looks interesting, and then it's forgotten as I go back to the actual task at hand.

Later, I sit down, and try and remember it, and WHOOSH. Nothing. It's gone.

So, a few weeks ago, I decided to start trying to record those ideas. The name of the practice is idea mining. The goal is to collect ones ideas and thoughts, organize them, and use them later as sources of inspiration and action. Or maybe pass them off to someone better positioned to act on them. Or just put them aside because they're impractical, or just plain dumb. Whatever.

Step One: Record

The first step in building an idea mine is recording your ideas. The obvious choice would be a notebook. I personally use Evernote on my iPhone. A handheld digital recorder, a PDA, whatever. Some people like Wikis.

Regardless of what you chose, it has to be:
a) Something that you always have with you - you can't record your ideas in it if it's not there
b) Something that is low friction and quick - you aren't going to record your ideas if it takes a significant amount of effort to record them
c) Something that is easy to work with later

That third point is my main reason for sticking with Evernote, but again- it doesn't matter what you use. Focus on the goal- record ideas when you have them. I'm the sort that does his best thinking on the toilet or as he drifts off to sleep, so my phone is great. If you do your thinking in the car, a digital recorder might be better.

It also takes some discipline. I know, several times, I've been near sleep, had an idea, and had to force myself to sit up and scribble it down. I try and say things like, "Oh, you'll remember it tomorrow," or "it's not that good an idea anyway".

And you'll have that thought a lot. "It's not a good enough idea to be recorded." Irrelevant! Maybe you're right, maybe you're wrong, but record it no matter what. Even if you ignore everything after this step, record every idea. Don't try and only record the good ones, because you're not always going to be able to tell, and the goal here is to build up the habit of recording your ideas. Once you get into the habit, you're going to get more ideas.

My experience with this step was that, for the first week, I was a virtual font of ideas. It tapered off and I went through a dry spell for a few weeks, and now I'm getting up a steady output of a few ideas a day, coupled with a few longer "jot of the day" notes- several paragraphs expanding on an idea of interest to me.

Step Two: Organize

You're going to be generating a lot of data, here. You're going to need to spend some time going through it. Fortunately, you've got all your ideas recorded, so there's no real rush on this. Even if you get backlogged, the ideas are still written down some place, and you won't forget them now.

How to organize them is up to you. Using Evernote, all the notes I record on my phone are synced to my home computer. In my Evernote client, I can tag the notes, using tags like, "good idea", "bad idea", "impractical", "actionable", "todo", "done", etc.. I tag by topic too, like "mad science", "writing", "hook", "character".

Again, the techniques to use here are largely up to you. Unlike recording, which requires immediate action, this can be done at your leisure. Just don't lose anything, because the last step is:

Step Three: Mine them thar' ideas!

Keep the organizing and acting steps separate. Periodically, check your "actionable" ideas, see if there's anything you want to tackle. When you're working on a creative project and are stumped, head back to your mine and see if there's anything in there that works for you. Use it for inspiration. Use it as a todo list. Use it as a personal reference.

My experience, after having done this for a few weeks, is that you're in for some surprises. You'll have more ideas than you think you will, even through those dry spells (in the past month, I've logged 206 notes, for an average of 6.9/day- not Manfred Macx territory, but not too bad). You'll have more bad ideas than you'll ever expect, (one of mine was a portable bidet, another was that a late night talk show hosted by William Shatner would be awesome, and an idea for a movie starring Eminem and Steve Guttenberg called, "What Mathers?"), but you'll also have a bunch that make you go, "Hey, that's pretty clever."

The final goal, of course, is to act on some of these ideas. Or hand them off to someone who can. Add a little creativity to the world, and turn ideas into end products.

Rochester in panorama, and Minna has an extra arm

Fri, 26 Jun 2009 22:05:27 GMT

Rochester in panorama, and Minna has an extra arm
Originally uploaded by t3knomanser.

That extra pudge is a side effect of the panorama process. He's husky, but not quite that husky.

Ship of Theseus

Fri, 19 Jun 2009 17:43:21 GMT

The Ship of Theseus is one of those old philosophical saws. It came up in conversation the other day, and so I skimmed the Wikipedia article, only to be surprised that none of the sources cited took the same attack to the problem that seemed intuitive to me.

So, on the off chance this is a relatively original idea, I decided that I should get it written up.

For those unfamiliar, the Ship of Theseus poses this problem: If you take a ship, and replace every component of the ship, until nothing original remains, is it the same ship? At what point does it cease to be the same ship?

Common sense tells us, of course, that it is the same ship. And everyone from Aristotle forward has tried to explain why that is, with the occasional jerk that takes the stance that it isn't the same ship.

The Aristotelian approach was to separate the formal cause and the material cause of the thing. It's a fairly Platonic approach, which isn't surprising: there's the "idea" of the ship, and the "material" of the ship- even as the material changes, the idea stays the same. Pirsig's concept of patterns is a less Platonic approach to the same idea: there's the pattern of a ship which is made up of subpatterns in the form of the different components; changes to the subpatterns don't change the over-pattern.

The other big approach is "four dimensionalism"- which, at first glance, was my preferred approach to the subject. If you view the ship not as a three dimensional object, but a four dimensional one, where its history and future are one long line, and the 3D ship is just one temporal slice out of that line, you can easily resolve this conundrum.

Except, of course, that to be complete, one would have to have a 4D timeline for all of the components, and suddenly the ship is less a ship, than a generalized area where the timelines for other objects occasionally merge.

I'm not a fan of any of these approaches, in part because they all share an unspoken assumption: that the world is composed of objects, and that these objects are distinct from one another.

Our sense organs receive inputs from the world, a whole hierarchy of processing centers pick up those signals and organize them into structures. When you look at the pathway of just the visual processing centers, you can clearly see the evolutionary steps that built the rather complex visual system we have. At low levels, we have simple abilities like recognizing lines, and increasingly complex abstractions get built up from what is, at the lowest level, the stimulation of light sensitive cells.

Photons excite your rods and cones, which in turn send signals across your nervous system, which get processed by various neurological systems and result in a mental model of the world, where those photons are interpreted as "a ship".

A large purpose of our brains is to convert the inputs to our senses into models that can be manipulated and acted upon. If my brain is able to identify that ship as Theseus's ship, and not my ship, I know not to trespass, less I earn myself an ass-kicking. And by being able to distinguish those sensory impressions as "a ship" and not, "ocean", I can do nifty things like not drown because I thought I was standing on a boat's deck while really I'm on the ocean floor.

It sounds absurd, but we know that, when failures occur in this processing system, really bizarre results can occur. Like the rather famous story of a man who thought his wife was a hat.

The point I'm trying to dig down to is this: the world is not a collection of objects that we can interact with. The world is continuous. The identity of objects as distinct objects is not a fact of the world, but a fact about our understanding of the world. The pile of matter that constitutes Theseus's ship is just a pile of matter.

The problem of Theseus's ship is not one of the identity of objects: objects don't have an identity. The identity of objects is a perceptual thing, born from our own minds' organizing principle. Our brains are object oriented. The world can be understood in these terms, but as the Ship of Theseus problem shows, there are edge cases that can give us bizarre results when we start trying to analyze what about an object provides it with its identity.

God, I hope that made sense.

Safe Communications

Wed, 17 Jun 2009 00:17:21 GMT

Given the events transpiring in Iran, now's a good time for more people to fire up Tor, an "onion router". Essentially, it's a Peer-to-Peer anonymizer that helps keep your Internet traffic confidential. The more peers on the Tor network, the better the network performs and the more anonymous it is.

Tor also acts as a proxy, so if you or someone you know is living in a country where Internet traffic is restricted, using Tor allows them to bypass web censorship. With browser plugins to make it easy to use, anyone with only a small amount of technical know-how can get unfiltered web access whether their government or employer likes it or not.

You can download Tor, and quickly benefit from its security benefits, if you're concerned about staying anonymous online. If you want to help others bypass censorship, you must run in relay mode. It's easy to setup, and will only take you a few minutes. It will even do its best to talk to your router to configure services so that you don't have to.

By simply leaving a Tor relay node running on one computer on your network, you can help guarantee anonymous, unfiltered access to the Internet for anyone in the world. No muss, no fuss.

If someone lives in a country with web censorship, they may have a hard time getting access to the Tor install files. So, if you have some web space, consider mirroring the install files. Pass the link discreetly to those that might be interested in such things. I'm setting up my own mirror right now.

Riots in Iran

Sun, 14 Jun 2009 14:09:45 GMT

At this very moment, things are spiraling out of control in Iran. Frightened by a suddenly politically active youth bloc, the government did the only thing you can do if you're a repressive, theocratic, and generally vile government: it tampered with the election results. It didn't just tamper- "tampering" implies that they wanted to conceal their actions. They fabricated election results. They released election results that have as much grounding in reality as Lord of the Rings.

The message was clear: "Go away. We do not want you involved in the political process. Return to your cynicism and go away. Fuck. You."

Fuck you indeed. Since the announcement of the fabricated results, the Persians have taken to the streets, and they're still out there. Marching, rioting. They are mad as hell, and they're not going to take it anymore. As of this morning, protests are still moving through the cities.

It's too early to tell where this is going to go. The government is scrambling, raiding dorms, arresting anyone that could rally the people, cutting power, Internet, telephones, and now hunting satellite phones. But word is still getting out, people are still getting organized.

This may be the beginning of the end for the current regime, and good riddance to bad rubbish. For Persians and for the rest of the world, this violent upset could be the best thing to happen to the region, and it's been a long time coming. This could be the death of Iran's theocracy.

Which raises the question, where's the US news media? It's not that they're not covering the story- it's getting headlines, here and there. But they're focusing on other things. Last night, as people were taking to the rooftops in Iran. CNN's top story was the Six Flags Bankruptcy.

Not only was the Internet the best place to get up to the minute reports, it was the only place. Considering Iran's role in the US's foreign policy, this is patently ridiculous.

All this said, there's painfully little we can do. But some people in Iran are still online. Discuss the subject, and let the people of Iran know that the world supports them.

I'm the KOOL AID MAN!

Tue, 09 Jun 2009 22:57:01 GMT

//Oh yeah!

Would you like some money for doing nothing? No? Well fuck you then.

Sat, 06 Jun 2009 15:51:34 GMT

Minna and I haven't been too hot on buying a house. We're saving a big wad, and waiting for the right thing to come along. We're not going to stress about it, we're not going to angst over it. Our apartment is nice, cheap, and enough for anything we need.

In short, we're a Realtor's worst nightmare. They can't leverage us. "I've got another offer in, so you need to act quick!" No, no I don't. If I don't get this place, there'll be something else. Each place I've looked at is nicer than the last, and at a better price.

We looked at a place today, and we were feeling pretty good about it. We tagged along on someone else's showing, so we didn't meet the selling agent. In order to save a few bucks, Minna and I are willing to brave the process without a buyer's agent, if the seller's agent is willing to cut their commission from 6% down to 4-5%.

I thought this was pretty reasonable, and I've talked to some Realtors that were fine with such arrangements. This Realtor, on the other hand, gave me shit over it. "Oh, I'm way too busy to handle both sides of the transaction for a mere 4-5%."

Yeah, because we've eaten up so much of your day by getting you to show the house (oh, right, we didn't!). And, like a regular buyer's agent, we've dragged you to a million houses while looking for something we like (oh, right, we didn't!).

If we brought in our own buyer's agent, she'd get 3%. I'm offering her 1-2% on top of that all for the glorious chore of signing some papers. It just makes no sense to me- I'm offering you more money for doing pretty much nothing. It just boggles my mind. Even worse, she was such a raging bitch about it, that I don't even want her to get even the 3%- I don't want to do business with her. It's not that she was rude, or really lost her shit, she just started ranting at me well past the point where I was ready to drop the subject. I'm trying to cut you a deal, I'm trying to work out something were everybody wins. It's fine to say "No thanks," but she ranted at me about it well past the point where I cared anymore, and was just trying to end the phone call politely.

It's sad, because it was a nice place, but the condo fee was too steep anyway- $200/mo is just more than we could afford on top of taxes and mortgage. She had an offer in anyway (she claimed, and I believe it), so she can just hope that goes through, because I'm not going to counter offer, and if I see she's the selling agent on another property I'm interested in, I'm going to avoid it. It'd have to be something really special to get me to deal with her.

There's a reason Levitt equates NAR with the KKK (Freakonomics is an excellent read, by the way).

On the Order Of

Thu, 04 Jun 2009 21:02:33 GMT

Programmers, when they're really concerned with being efficient, will often start talking about "Big O" or "Order". There's a lot of interesting math behind this, but what it boils down to is a very simple concept- how many operations does it take to solve a certain problem, using a certain algorithm? I'm going to simplify things here, ignore the difference between so called "Big O" and "Little O" and not worry too much about the theoretical underpinnings. This also isn't the only metric used to evaluate how efficient an algorithm is, but it's a major one.

For example, if I hand you a deck of cards, and I tell you to find the Ace of Spades, how many cards would you have to look at in order to find it? Assume you start at the top, and keep drawing cards until you find the one you want. In the best case, the very first card you flip is the one you're looking for. That's on the order of one operation, written as O(1). But, that's the best case. What about the worst? If it were the last card you flipped, that would have taken 52 operations- O(52), or, to be really programmery, O(n), where "n" is the number of cards in the deck. By being general, we can extrapolate this to any collection of cards- be it a full deck, less than a full deck, or even a collection of something else entirely- like an address book.

When talking about the "O" of an algorithm, programmers don't like to worry about constants. We don't like "52"- that's far to specific. "n" is much more useful. But similarly, if there were an algorithm with O(2n+1), we're going to ignore the 2 and the +1 and just call in O(n). Doubling it once just isn't a big enough change to really count. It's details and small stuff, and if "n" is sufficiently large, it just doesn't matter.

Now, searching each individual card in the deck isn't terribly clever, but if the deck is shuffled, that's really all you can do. Picking out randomly, going from top to bottom, bottom to top, it just doesn't matter. But if the deck were sorted- now that could be useful. Think about the phone book- it's sorted, and you don't have to search every single name to find the one you're looking for.

If you were writing a computer program to find a name in the phone book, you'd start by going to the name in the middle. If it's the name you're looking for, great, you're done. If not, does it come before or after the name you're looking for? Once you know that- you've just eliminated half of the names. Take the remaining half, and go to the middle. Repeat the previous process until you find the name you want. Each operation (name you look at) eliminates half of the remaining names. Searching a sorted list is very effeceint, and in this case, it's on the order of "lg n". (lg is the log base 2, or "what power do I have to raise 2 to to get "n"). This, by the way, is called "binary search" and is the goto searching algorithm to use in any search.

In our naive search, where we just checked every element, it's O(n). In a deck of 52 cards, that's 52 operations. Not that bad, but what if I had a list of 1,024 items? That's 1,024 comparisons. But if we used the binary search, it's O(lg n)- 10 comparisons. (2¹⁰ = 1,024). That's a savings of 1,014 comparisons, in the worst case.

Programmers tend to break down algorithms into a few major categories based on their Big O.

Stuff that's O(lg n) is very fast stuff. They scale really efficiently to big sets of data (1,000 elements is 10 operations, a million elements is 20 operations, a billion elements is only 30 operations!).

O(n) isn't as liked, and most O(n) algorithms can be turned into O(lg n) if you're very clever and find ways to cheat (for example, if you need to search an unsorted list, it'll always be O(n), but if you sort it first, even though sorting is expensive, you'll make it up if you search a lot).

Sorting is really expensive, in comparison. The best sorting algorithms are O(n*lg n). That means, for a set of a thousand elements, it would take 10,000 operations to sort. Ugh, but some problems just can't be made any easier. O(n*lg n) is the best you'll get for sorting algorithms, and there are whole classes of problems that fall into this category.

Worst is O(n²), or even larger exponents. This is the demon of programmers. For example, the first sorting algorithm that most CompSci students learn is the Bubble Sort. It's very simple to understand and implement, but it's an O(n²) sort. But, to sort 1,000 elements, you have to do 1,000,000 operations. Algorithms that run in "polynomial" time are ones that programmers hate. It's worth noting that there are a lot of problems that, as far as we know, can't be solved any faster. There's a whole family of problems that are called "NP Complete", which, right now, can only be solved in polynomial time. Whether or not it's possible to find a better way to solve these problems is an open question.

This was brought up today because I'm working on Gravitone, an iPhone instrument that uses gravity to generate music. I took a very naive approach to the problem, and simply cycle through all the masses and objects in the world and apply gravity to every other object. My algorithm is O(n*m), where n is the number of orbs in play, and m is the number of masses. Sort of polynomial, and if I were to try and scale it up so that the orbs could effect each other, it'd be O(n²). Ugh, polynomial time? That's expensive. Or is it?

I did some research in gravity simulation, and found that the best algorithms out there run in O(n*lg n). The one I looked at specifically was called the "Barnes-Hut algorithm", and it's very clever, and something I'll probably use for a different application. It's also fairly complex to implement, and requires a lot of memory, despite being very fast (memory/speed are a common trade off).

It's faster, but should I use it in this application? No!

In my application, I've capped it at 15 orbs in play and 5 masses in play, which means it's going to take 60 operations.

That's 20 objects total, and in the Barnes-Hut algorithm, 20 * lg 20 ≈ 60 operations.

For very small data sets, sometimes, really expensive seeming algorithms are okay. Now, the Barnes-Hut algorithm would be better- it would scale better, and it would allow me to have the different orbs attract each other, and the masses- it'd be a much more compelling simulation, but that's not the point of the application. I don't need the power offered by Barnes-Hut.

All that said, I think making a Barnes-Hut simulation on the iPhone would be kinda neat. Maybe a game, or a different instrument.

Antivaccination Deaths

Mon, 04 May 2009 19:04:42 GMT

An infant died of whooping cough in Australia recently. She was too young to be immunized, but if the adults she came in contact with had been, she would have benefited from herd immunity, and would still be alive.

Out of curiosity, I ran some numbers on vaccinations.

Let's be very generous. Let's posit that this utterly unsubstantiated and implausible link between autism and vaccination exists.

What's the rate of incidence? There's 6 in 1,000 people that have some sort of autism spectrum disorder. 2 in 1,000 have true autism. We'll work with that number. Let's assume that every member of this 0.2% is properly diagnosed and the disease was caused by a vaccination.

I repeat: these assumptions are very generous. Even if vaccines did cause autism (they don't), we know that there are other causes as well. These are very generous assumptions to make.

Now, let's look an measles. In an vaccinated person in a developed country, 3 in 1,000 people die- 0.3% fatality rate. In underdeveloped nations, it's closer to 280 per thousand. In immunocompromised patients, like AIDS victims or cancer patients, it's in the same neighborhood- about 300 in 1,000.

And that's just measles. And that's just deaths- we're not counting complications like corneal scarring- yes, measles can blind you.

So, even if we grant the most generous possible claims made by the anti-vaxxers, their arguments don't stand up. Measles, alone, is a more credible threat than vaccine induced autism, even if every autism case was caused by a vaccine. Even if we take the absurd claims at face value, the argument doesn't hold up.

Oh, let's keep going. Whooping cough kills 600,000 people a year of the 10-90 million it infects. Why the big range? It occurs mostly in third world countries where it's hard to get good statistics. Let's pick in the middle- say, 60 million cases. That's a 1% fatality rate. Heck, even if we go out to 90 million, we're still looking at a 0.6% fatality rate- which is the same rate of autism spectrum disorder in the population.

Between measles and whooping cough, we're talking a 0.9% fatality rate. Wanna start adding diseases? Polio isn't extinct, you know.

Ignoring the non-existent autism link, there are real risks to vaccines. The CDC has some data, but it should be perfectly clear: vaccines are less dangerous than the disease they prevent.

Functional Programming in .NET

Sat, 25 Apr 2009 17:46:08 GMT

I love functional programming. I'm going to present on doing FP in VB.Net in August- this is a feature new to .NET 3.5. I was very excited about the ability to do functional programming in VB.Net. And some of the key features I really wanted, currying and closures, are there.

But the limitations... they almost make it not worth the trouble. Most obviously, VB.Net doesn't support true lambdas. In a true lambda environment, I could do something like this:

f = Function(x as integer) if (x % 2 = 0) then return x / 2 else ... some other code ...

In true lambdas, you can put any code you like inside of your function. C# and F# allow this. VB.Net does not- VB.Net only allows expressions (you can't do ifs or loops or anything like that).

Still, there's a lot you can do with expressions, so that's not too bad. Since you get closures, you can work around that without too much trouble.

But then you start running into the bizarre things. I wanted to do a patterned call. In many functional languages, you can create functions like this: f(1) = 1; f(2) = 2; f(x) = f(x - 1) + f(x - 2);. Calling f(13) will print out the 13th number it the Fibonacci sequence.

Now, I accept that VB.Net wouldn't have an architecture like that built in- it's a somewhat obscure functional trick. But I was hoping I could roll my own. My first attempt at it was to come up with a compiler macro- oops! VB.Net doesn't support pre-proccesor macros. C# does, of course.

Well, okay, what about Attributes? .NET has the ability to define metadata on code, that you can "reflect" on to change runtime behavior. I could do something like this:

Module 1
  '"fib" is the function name, the second parameter is the pattern- if it returns true, execute this
  'operation, otherwise, go find another operation named "fib" to execute.
  <PatternedFunction("fib",function(x as Integer) x = 1 or x = 2)> Function f1(x as Integer)
    Return x
  End Function

  <PatternedFunction("fib",function(x as Integer) true)> Function f2(x as Integer)
    Return PatternedCall("fib")(x - 1) + PatternedCall("fib")(x - 2)
  End Function
End Module

Not as concise as I would like, but hey, it works, right? Wrong. Turns out, since Attributes are evaluated at compile time, you can only pass constant expressions into them. Since a function may possibly contain a closure (even though this one does not), you can't ever treat a function like a constant.

I'm being an FP snob, and I know it. My reason for wanting to do this is less because I have a specific need and more because I want to. I get frustrated when a language implements a potentially awesome feature in a half assed way, but as I think about it, "Potentially awesome, practically useless" describes VB.Net to a "T". If I had my druthers, we'd be a C# shop if we were doing Microsoft at all- I think a big portion of our business would be streamlined by a real RAD language, like Ruby or Python.

All of my complaints would be patched if VB.Net supported compiler macros. I'm stunned that it doesn't- it's not exactly the hardest thing on earth to implement; your average C compiler has had a macro pre-processor since before I was born. C# has one. And here's the real kick in the teeth: most macro engines are language agnostic, so there's no reason they couldn't have wired the C# engine onto VB.Net.

Basically, this is an exercise in driving home the flaws of the language that pays my bills.

Gravitone

Tue, 21 Apr 2009 16:15:35 GMT

Here's a demo of my current project- an iPhone app called "Gravitone".

Beer, Pop Art... pop top beer.

Mon, 13 Apr 2009 02:27:36 GMT

Ever since moving to Pittsburgh, I've kinda been avoiding the Warhol Gallery. I don't like Warhol. I don't generally care for pop art (although this is pretty cool).

But, a friend volunteers/works there and managed to score tickets to a beer tasting. Hey, free beer? I'll take a look at any art gallery if free beer is involved.

The local brewery they chose, Stoudt's, didn't have much to offer that I was blown away by. Good, but not spectacular stuff. Very Belgian style, even in their "American" beers. But that's not the real point of this post.

After enjoying a generous helping of beer, we took a stroll through the gallery. A lot of it, unsurprisingly, was dedicated to Warhol himself. There were some other modern installations too. And it was all, generally, crap. As I'm walking through there, seeing wall after wall of celebrity snapshots and pop culture ephemera, "recontextualized" as art, it really struck me:

If pop art were a person, it would be the gum-snapping coworker in the cube over from you with some ear-wormy ringtone on the too-loud-cellphone that natters on and on and on about Branjelina and Brestica or whatever while gossiping about how great the next "Sex and the City" movie is going to be.

That's my take away from the Warhol gallery. A large amount of pop art focuses on taking the inane, vapid and annoying and "recontextualizing" it as art. The result is art that is inane, vapid and annoying.

Now, after venting my spleen, let me cushion the blow with a hint of perspective. A large part of my reaction is that it simply hasn't aged well. The reason people appreciate Warhol is that he altered the definition of what constitutes art. That's not entirely a good thing, and he was certainly standing on the shoulders of giants, like Duchamp. But teleporting myself back to the 50s and 60s, seeing giant Elvises shooting at me from a wall would be jarring, and that's obviously the desired effect.

One solace in the Warhol gallery: Warhol hated Pittsburgh. Reviled it. So, the fact that the town has a little shrine to him and has named a bridge after him stands out as a little "fuck you" to Warhol, and I can live with that.

More Stanford iPhone Homework

Mon, 13 Apr 2009 01:59:37 GMT

Not content to have exceeded the assignment's requirements by adding a graphical layer, I went a few steps farther: I added support for animation and multitouch gestures. The latest version of the source lives here, and it's chock full of comments this time.

Once again, it's under a Share-Alike-CC-license. Don't hand it in as your homework.

iPhone Programming : CS193P

Sat, 11 Apr 2009 15:31:52 GMT

Stanford University is publishing video and assignments for their iPhone programming course online. I've been following it and doing the assignments, and man... I miss compsci classes. I've been having so much fun doing this. For those that recall me in college, I was a lazy underachiever in most of my classes- but not the programming ones. In those, I always took the assignment and exceeded the parameters. I would show other students how to do the assignments. I'd add little flourishes.

This has been such a breath of fresh air. By day, I slog through tedious code written in tedious languages to do tedious tasks. By comparison, programming on the iPhone is downright sexy. It's fun, it's fast.

But more than that, I'm enjoying my remote college experience. My brain is getting a gentle stretching, and I really like that. Of course, it's very gentle- the course material goes at a painfully slow pace and is treading over the basics of OOP with a leaden step. But then I pick up the homework assignments, and run past the requirements and show off, and I don't care how dull the lectures get.

Not to say I get nothing from the lectures. I finally "get" Objective-C memory management. ObjC has an approach that's someplace between Java-style Garbage Collection and C style malloc/free.

In any case, I've got my first non-trivial iPhone application done. The business logic is pretty trivial- do some stuff with polygon shapes- but the UI has drawing and animations, which is well beyond the goals for the current homework assignment. If you have an Intel Mac, you can download the SDK from Apple (free signup required) and run it if you like. The linked code is distributed under a CC-share-alike license: HelloPoly code.

If anyone is dumbshit enough to try and hand this in to the class, they're going to get owned, because it's pretty obviously not what the assignment called for.