Noodlings

Over the weekend, Gus Mueller turned me on to the LLVM/Clang static analyzer. And just in time, too, as I was polishing up my 2.2 release (which went up earlier today).

It’s an offshoot of the LLVM and Clang projects (read the respective pages on what they are if you don’t know already). The static analyzer analyzes your code and looks for problems, focusing mainly on memory allocation patterns, in this case, including Objective-C/Cocoa (retain/release/autorelease) and CF (CFRetain/CFRelease) semantics.

Take this contrived example for instance:

  id foo()
  {
      NSArray       *array = [[NSArray alloc] init];

      if ([array count] > 0)
      {
        return nil;
      }
      return [array autorelease];
  }

The example above will get you a report like this (it generates html):

Drilling down you get this (still in html):

(click to enlarge)

Here you can see it pointing out [1] where the object was allocated [2] the branch it took and [3] the point where you leaked it. Pretty neat. It tries to follow every possible branch finding paths where you may have leaked an object. It also finds what it calls “dead stores” (when you assign a value to a variable but never use it again) and missing dealloc methods

As the project page says, it is very early in development. You’ll find that it does turn up a lot of false positives, especially with the missing deallocs. False positives for memory leaks seem to occur when you release something in a different scope than where you created it. For instance, I have this chunk of Apple code that wraps CFRelease() with it’s own function that checks for NULL first. The checker complained about this every time. Nonetheless, it did turn up some real leaks for me.

Aside from reducing the number of false positives, I’d also like to see the entries grouped by source file (it’s annoying jumping around between files) as well as some way to bring up the original source file by clicking on its name in the source code view. You will also see multiple entries for the same leak when the code traverses multiple paths that end up with the same leak which can be annoying.

In any case, I recommend downloading it and giving it a try. I’m not sure how thorough it is (i.e. whether it can supplant running your program through MallocDebug/Instruments/leaks) but it makes a great additional tool to add to your arsenal. Chances are it will look at some code path that you don’t test. Oh, and a couple tips:

• Make sure you do a clean build on your project first. The checker only runs on files that would normally be compiled (it sits in as your compiler). If your project is already built, then no files will be compiled/analyzed.
• Use the -V option, which will pop open a browser with the analysis page when done. Normally, it sticks the files somewhere under /tmp but only shows the actual path when you start the run. Needless to say, that bit of text scrolls off pretty quickly.
• While the tool does come up with false positives, you’ll find that sometimes it finds something subtle that you may blow off as a false positive on first glance. Make sure you understand what it is flagging, even if it ends up being wrong.

I haven’t used it with a garbage collected program so I don’t know if it uses different techniques in such a case or is just plain unnecessary. Maybe the dead store detection becomes more important. Reports from anyone using this with GC are welcome.

Yes, it’s finally out. Hazel 2.2 is what I consider the “power user” release. It adds advanced features such as pattern matching and custom tokens (basically, a more accessible form of regular expression matching and substitution, for you programmer types out there), inline scripts and ways for AppleScripts to control the rule flow. There are a bunch of smaller things tucked away in there, some of them subtle in their own ways. Make sure to read the release notes.

Thanks to all the beta testers who found all the bugs there were to find (you guys did find them all, right?) and all the users who have sent in the great comments that motivate me to keep working on this thing. Download it and give it a spin.

As for the future, I’m thinking of 3.0 though I’m not sure what will be in it yet or when it will happen. I also have been mulling over other projects so we’ll see. In the meantime, I look forward to your comments.

It occurred to me that it’s been about a month since I did the MacUpdate bundle in April. Now that things have settled down I figured I’d share my experiences with it.

Now, there has been some controversy concerning the bundles, boiling down to whether it is a good deal for the developers. After all this, I can’t say that the issue has been fully resolved in my mind but I’ll try to at least clarify the real issues at stake. I want this to be useful to other devs who are considering participating in bundle promotions without resorting to any demagoguery.

Why Did I Do it?

Hazel was included in the bundle with 9 other apps, the main one being Parallels. This was the anchor app that would be the big draw for most users. For me, this was a key thing. Not so much because of the potential sales as much as being associated with some larger name apps. Some people may have considered the bundle a bit boring as a lot of the apps were a bit “mainstream.” But then again, we’re talking apps that you actually see on the shelf in a store. It’s a whole level of distribution and exposure than online. What many people online don’t realize is that a vast segment of the potential user base does not scour the net for software reviews. They’ve never heard of any of your favorite Mac sites. They don’t know and don’t care about the latest Mac scandals, memes and fads. They go buy their software at an Apple store, or maybe on Amazon. Our only chance to get noticed by these users is in the oddball case when they click on that “Mac OS X Software” item in the Apple menu. The point is that for us smaller ISVs without a physical boxed product, it’s an untapped market for us. So, to be able to be associated with a couple apps from that “realm” I felt was an opportunity.

MacUpdate did run full-page spreads in MacWorld and MacLife, not something I would have been able to do myself. There is something nice about seeing your icon next to some bigger names in print. I don’t know how effective they were. The problem with print, of course, is that it’s hard to track. But again, I felt that the association would be helpful.

Now, I’m not going to give specific details concerning the money but I will say that it was a percentage-type deal. The more copies sold, the more money I got. Contrast this with a flat amount that does not vary no matter how well it sells. That said, the amount I got per bundle is way below what my product costs. So why do it? Some reasons:

1. Volume. They will sell over an order of magnitude more copies than you will during that time period. The hope is that the volume makes up for the huge discount. The hope is more money in total.
2. Not everyone is buying the bundle for your app. Especially if you are one of the smaller fish, you are probably piggybacking on the anchor app. One way to look at it is that you are getting part of someone else’s sale.
3. Exposure. Yes, that nebulous thing that is bandied about. It’s hard to make any concrete claims on this one so I think it’s best to not base your decision on this factor alone. You hope your product gets more recognition in the long run but it’s hard to measure that.
4. Userbase. By building up a large userbase, you have more people to get upgrade revenue from when the next big version comes out. Not having done a paid upgrade yet, I don’t know if this is all that it’s cracked up to be. Anyone with firsthand experience with this is invited to comment.

And of course, what’s bad about doing it?

1. Cheapening your software. The notion here is that if you are selling your software at rock-bottom prices, people get the perception that it is not worth much. I feel this is valid but I also don’t think doing it every now and then is a big problem. I think the issue is if your product is sold at a discount often enough that people will start to expect it, waiting for the next promo to buy it. Like exposure, it is hard to quantify and so I have a hard time basing any argument on this alone.
2. Support. Yes, taking on thousands of new customers in a short amount of time will probably result in an increased support load.
3. Allowing promoters to exploit you. This is more of an ego/sense of justice issue. Fact is, these bundles are pulling in a lot of money. It is unclear whether the developers are getting a big enough slice of that pie. No one likes to be ripped off.

What Happened?

How did it go? Well, the bundle sold 15K copies. From what I can tell, that seems ok. Part of me expected more considering that a big name app (Parallels) was anchoring the thing. Also, it seemed like the unlocking thresholds backfired resulting in lags in the sales rate at certain points in the promotion. Personally, I’m not big on gimmicks but I leave the promotion to the promoters. I did sense a general fatigue amongst consumers with these bundles, though.

As far as the money issue goes, I got a decent chunk. If you look at it solely from a per-copy basis, then yes, it sucked. But the way I look at it is this: My revenue per month is at some amount A. I sell in a burst and rake in something like 3-4x A. Afterwards, revenue goes back to A. Now the last bit is important as it implies, to me, that I didn’t cannibalize (i.e. sold at a discount to a lot of people that would have purchased it anyways at full price) or otherwise negatively impact sales.

Now, while my sales were fine after the promotion, they didn’t shoot up (as some people may expect). This would seem to indicate the lack of effectiveness of the “exposure” element. It’s only been a month so we’ll see how it goes but at the moment, it’s like it never happened.

The question here now is what was lost. I’ll start with the more tangible cost which is support. Now, I had to deal with a support nightmare because the integration of my licensing with MacUpdate was less than ideal. I had to deal with a ton of emails with getting the program registered. Now, I’ll admit my licensing scheme was the odd-man out but I don’t get these issues with my own store. So, in this instance, the support cost to me was a bit high. That said, it was very annoying because I felt it could have been easily prevented or rectified. Outside of the registration snafu, support was not too bad. A moderate burst during the promotion and that’s it.

The other thing to consider is opportunity cost. This is the cannibalization I referred to earlier. Were you losing money to people that would have bought at full price? It’s not a question that can be definitively answered but as I mentioned, I use sales after the promotion as a gauge for this type of thing. If there’s a sales dip right afterwards, that implies to me that a lot of people that were going to buy your product anyways bought earlier to get the deal. In my case, sales were not negatively affected so I do not consider it a significant issue in this case.

Was It Worth It and Will I Do It Again?

I’d say it was worth it, but not in the way people would expect. It was worth it to me in that I got a nice check afterwards without having my sales adversely affected. As for exposure, there may be better ways to get people to know about your app without practically giving it to them. I don’t have a paid upgrade planned out yet so that was not even a consideration for me. For me to do it again, I think I would have to be offered a good deal, percentage-wise. My sense is that these promotions are more effective in the short term than long term.

Now, that doesn’t mean that it is totally ineffective in creating exposure for your app. I have seen signs that people have been turned on to Hazel who had never heard of it before. The issue here is that it has not translated into increased sales (at least so far). Maybe it will help in the long term but since it’s near impossible to quantify, it’s not a prime motivation for me when doing these types of promotions.

Did I Get Ripped Off?

To address the big controversy, no, I do not think I was ripped off though I could’ve done better. Let me start off by saying that I feel that promoters do provide a valuable service. I’ve seen some developers band together with their own bundles with less than stellar results. Fact is, not anyone can just throw their stuff up and expect to sell any large volume. There is skill and work involved in getting people’s attention. Doing it yourself is, well, just doing it yourself. If you already have the pull to get tons of people to pay attention to you, then you probably don’t need to do special promotions for your apps.

Now, I haven’t seen the books for any of these promotions plus the deals probably vary greatly depending on who does them so it may be unfair to lump them all together. But it seems to me that the promoters are getting quite a large chunk and that they can afford to cut the devs a better deal. I’ve seen arguments along the lines of “How can you criticize them? They are getting sales for developers/exposure for their software/donating money to charity/yadda yadda.” This is a logical fallacy. Yes, maybe they are doing some good things, but that doesn’t mean it justifies the bad things. There is nothing that has indicated to me that they can’t do all the good things they already do while providing a more equitable split. Again, I haven’t seen the books but my sense is that there is some leeway there.

I want to make it clear, though, that I have no complaints about my deal with MacUpdate. In the end, it was my deal to negotiate. For me, this was an experiment and knowing what I do now will help in assessing future promotions.

And of course, this is just my experience and your mileage may vary. Especially if you are just starting out, it’s possible the exposure element will be more helpful. I’m not sure how useful the exposure is for the more well-known apps but I suspect that they, having more leverage, negotiated higher percentages. In the end, you have to assess whether it fits with where you and your app are, marketwise.

Yes, it’s that time of year. I do have to say that I was on the fence about going this year. There didn’t seem to be anything new on the OS X front (this was before the recent rumors of 10.6) and I’m apparently one of the few people not terribly excited about iPhone development.

Nonetheless, I regretted not going to MacWorld and I felt that I would regret not going to WWDC. I did contemplate going to SF just to crash the parties. Then I thought that this will be my chance to hit the sessions for those APIs that I don’t normally use. I figure that the devs in the OS X sessions would be easier to get at since everybody will be in the iPhone sessions. Plus I always find the labs valuable.

So, if you see a wide-eyed developer who is peeing his pants over being able to write a countdown app on his iPhone, that’s not me. Instead, I’ll be the grumpy guy walking around trying to corner one of Apple’s filesystem engineers.

This past month I’ve finally gotten some time focus on development and it’s paid off. Hazel 2.2 beta is out. Go to the forums (signup required) for details on how to get it.

The big feature is pattern based matching and substitution. It’s regular expressions for regular people. I still consider it somewhat of a power-user feature but I think I’ve managed to make it accessible to non-programmers. I know some of the programmer-types out there wanted regexes but, really, life is too short for people to have to decipher something like ^(.*)\(\p{N}+\)\.(?:\p{L}|\p{N})+$.

Other features include being able to edit/include AppleScripts and shell scripts inline. This will allow for self-contained rule sets that you can distribute. More date matching operators especially geared towards GTD-like workflows. A couple other more subtle things here and there.

But most of all, this is a momentous occasion because I have finally done it. I have fixed bug #1 in my bug database. Entered almost two years ago, it kept getting carried over to each release’s bug list time and time again. What is bug #1, you may ask? Why, it’s none other than “multiple selection in the rule list”. It’s been one of those rarely requested (maybe 2 or 3 people since Hazel’s original release), low-priority bugs that kept getting overlooked. Well, no longer. I went in and fixed it once and for all (hopefully). Now it looks like bug #4 is the next oldest bug, which, at this rate, I should have closed out in a couple years.

So, if you are fine with running unstable pre-release software that then I invite you to kick the tires. All I ask that you keep all details of the beta to the forums or in emails to me.

Some months ago I had arranged to sponsor Daring Fireball for sometime in April. Then a certain promotion came up that overlapped with it. Not being big on overlapping promotions I bumped it until later. Then I bumped it again. Then another bump. Then a jump to the left and a step to the…whoops, wrong card.

But this week, finally, I am sponsoring Daring Fireball’s RSS feed. There’s even a promo coupon. I’m not going to tell you what it is as that would defeat the purpose of the promotion. Subscribe to the feed and look for yourself. If it’s not showing up for you, wait towards the end of the week when it will get posted again.

And on a side note, I suggest hanging around the forums (signup required for the beta forum in particular) as there may be some beta news by the end of the week.

It’s been over a month since I’ve been with Slicehost so I figure it’s enough time to make an assessment. Especially now that the MacUpdate promo is over, I actually have a sense of how well things hold up under load.

For those who don’t know, Slicehost is a hosting provider. What sets them apart from shared hosting providers is that they provide you with what they call a “slice” (other similar providers may call it a virtual private server or VPS). What this is is a virtualized server of your own. From your perspective, it’s like getting a dedicated server. You choose what OS you want (which right now consists of different Linux distributions) and you get root access so you can do whatever you want.

It differs from shared hosting in that your slice is like it’s own machine. It gets a guaranteed amount of memory and CPU so even if your neighbor is a hog, it won’t affect your slice. Because of the way that the Xen virtualization works, it is impossible to oversell on capacity.

Compared to getting a dedicated machine, it’s much cheaper and you aren’t tied to specific hardware. Blown power supply? Not your problem. I don’t know exactly what they do in this case, but I imagine they can move your slice to different hardware as needed.

Set up

You can read my original report on getting set up. You are expected to set up and administer the slice yourself. If you have the inclination and need a high level of control, then this is probably for you.

It only took me a weekend from getting my slice to having everything migrated over. Of course, being the tweaky type that I am, I spent some days playing with it and optimizing it. One of the benefits and dangers of having full control.

Upgrading

When I launched the new store, there was a problem. Ruby on Rails is a memory hog and as a result, I needed to upgrade to a bigger slice. Fortunately, Slicehost automates all that. Just log into the management console and request the larger slice. It takes a little while for the slice to get prepped but during that time your slice is still up. The downtime for the reboot was short (less than a minute) and that was it. The fact that it’s automated is a big deal to me as it means I can do it on the fly without doing a drawn out back and forth with a support person.

Traffic

As you may or may not know, Hazel was included in the recent MacUpdate bundle. Before the launch, I upgraded my slice again to 1024M in anticipation of the load. Turns out, this was unnecessary. The 1024 slice never broke a sweat. The load went up briefly to 0.4 once. Apache connections stayed below the upper limits of what a smaller slice would have been able to handle. Traffic was about 150K requests a day at its peak. I don’t really have a frame of reference for that except that it’s a good bit more than I usually get. In short, the 512 slice would have been able to handle it fine. The slice has performed better than with my previous providers and I haven’t noticed any slowdowns or downtime (except for when I restart things for maintenance). With the promotion over, I’ve downgraded my slice and things are still running smoothly.

After all this, I’ve only contacted support twice. Once in the beginning just to say hi and once today for an issue that ended up being an Ubuntu thing. In both cases, I received responses within the hour. Granted, I’m doing a lot of the things that support at other hosting services would do for you. It’s a trade-off between effort and control and I’m at that point where I need more of the latter. For the things I really care about, keeping the machine and network reliable, there has been nothing to report, and that is how it should be.

It’s too bad most providers price on bandwidth and storage space. I would have happily paid more per month if I could get higher availability and reliability (with the ability to run RoR – sorry Pair). Of course, with everyone claiming 99.999999% availability, it’s hard to differentiate oneself on this front so providers seem to just pile on the bandwidth/space like extra gravy hiding the bad meat.

I feel like VPSes are the future of hosting. The amount of computing power that you can cram into a 1U rack space is far more than most of us need or want to pay for. But virtualize it and divvy it up and you have a great scalable model for doing dedicated hosting. It’s probably greener too but I’ll let the hippies make a determination on that.

As you may or may not know, Hazel is packaged as a preference pane. While you do all the configuration via the preference pane interface, the actual work of running your rules is done by background processes. The commandline tool that actually runs the rules is stored inside the pref pane bundle. I’ve kept it in the Resources part of the bundle for no good reason except that’s the catch all for all your bundle stuff. But then I thought, “why not stick it in the MacOS dir. That’s where executables are supposed to go, right?”

So, I went into XCode and created a new “Copy Files” phase to copy the built executables into the “executables” dir (which ends up being the MacOS dir in your bundle). I modified the code that launches the program to look in the right place (NSBundle’s -pathForAuxiliaryExecutable: for those of you keeping score at home). It seemed to work. My bundle was now a tad tidier with things in their proper place.

Later on, I was testing something else and noticed that when Hazel was running certain rules, the pref pane icon would show up in the dock and then disappear when the background program was done. Strange enough that a pref pane icon would appear in the dock; stranger that it was being triggered by a non-GUI program. More testing revealed that it happened when Hazel was executing AppleScripts. I tried isolating the AppleScript parts. I checked all sorts of paths. I couldn’t figure out what was causing it. The Pope was in town so maybe he was messing things up somehow.

I checked previous versions to isolate when the bug was introduced and discovered it was only in the last set of changes. Moving the executables back to Resources fixed it.

Problem solved but I had no idea why. After muttering about it in IRC, Mike Ash casually throws out the explanation: “the binary is hitting the window server, the window server notices that it’s in an executable directory of a bundle, sticks it in the Dock.”

Of course, it all sounds so simple when you hear the answer. Calling the AppleScript created the window server connection which then prompted it to throw the icon in the dock as if it were a regular app. Not sure how long it would have taken me to figure that out on my own.

The lesson here is that when faced with a crazy bug, ask Mike Ash. Really, though, don’t do that. He’ll crash his glider into me next chance he gets. The real lesson is that you should be careful about putting executables for background programs in the MacOS directory of your bundle.

Ok, so I can’t pin this one on the Pope. All I know is that I don’t get these types of problems when the Dalai Lama is in town.

If you haven’t seen the ads or announcements yet, Hazel is in the MacUpdate bundle that starts today. 10 great apps for $65.

Bundle mates include:

Parallels Desktop
Banner Zest
Sound Studio
DVD Remaster
Typinator
StoryMill
Leap
MenuCalendarClock
Art Text

This is the first bundle I’ve been a part of so I’m not sure what to expect. I’ve beefed up the server so hopefully the site will stay responsive throughout the promotion. When it’s all said and done, I’ll post my thoughts on participating. In the meantime, grab it while it’s hot.

In getting Potion Store integrated with my site, I had to learn Ruby on Rails. RoR has always been one of those things I’ve been curious about so I saw it as an opportunity to dive in. I present here a first impression. Note that these are impressions. They represent how I perceived things coming to it for the first time. While chances are that these perceptions are inaccurate, there’s something to be said for seeing how things appear to fresh eyes.

Ruby is an interesting language. It appears to be very flexible and dynamic, allowing you to do a bunch of really neat things that you can’t in most other languages. Note that these features aren’t really unique to Ruby. My sense is that Ruby is not a distinctive language in itself so much as a mostly good mix of other languages. I say “mostly” because I’m not sold on the syntax. It reminds me too much of Perl. It’s like they created a great new flavor of ice cream and then mixed in glass and razor blades.

My advice to those picking up Rails for the first time, get the Rails book and read the first few chapters before diving in. I found the online tutorials kinda annoying and not as good at explaining the structure and conventions that Rails relies on. Rails’ strongpoint is streamlining common tasks and functionality but to do so, it expects things to be laid out in a certain way. The other thing that takes time to figure out is the “magic” that Rails performs at certain points. One of the great things about Rails is how certain things are automatic. Very little glue needs to be coded and I feel that’s a wonderful thing. On the downside, sometimes this magic makes it hard to figure out what’s going on. For instance, some classes had naming patterns for their methods that you had to find out about in the documentation. I found the quality of the documentation to be a bit uneven which didn’t help in some cases.

I feel most of the problems with Rails are less about the design and more about the implementation. For instance, Rails is a memory hog. And because Rails is single-threaded, you need to have at least two of these things sitting there to get any sort of concurrency. When I deployed my store, I had to upgrade my SliceHost slice as the 256M slice wasn’t cutting it (remember I’m also running mysql and PHP stuff). Also, I felt deployment was a bit clumsy, as I had to set up a cluster of mongrel instances to run my app with apache acting as a proxy. While I see that there is a mod_ruby for apache out there, no one I know recommended it. Nonetheless, I felt like I had to deal with more moving parts than was necessary.

In a nutshell, Rails: great, Ruby: enh. If you are going to buy a book on it, get the Rails book first. It has an appendix on Ruby that should be sufficient for most things. If you want to go deeper with the Ruby side of things, you can get the Ruby book as well but personally, I could have done without it.

Overall, I like the platform but at the same time, I can see that it’s relatively immature and has some ways to go. I’m pretty much tied to RoR for now since it’s what the store is written in but that’s not a bad thing. That said, I also am not so amazed that I am going to join the Ruby cult.

Nonetheless, RoR feels like how a web framework should and less like a bloated platform where you end up installing a hundred frameworks requiring just as many config files to be edited to solve non-existent problems that are justified because someone gave it a fancy technical name along with a contrived design pattern to fix it (if you need a hint as to what I’m referring to: “starts with a J”). RoR seems to be able to identify and address the actual problems people run into when creating web applications, all the while keeping it relatively simple, and that is refreshing to see.