Noodlings

I’ve spent the past couple hours going crazy over a bug. It’s the type of bug where it breaks except for when you look at it, in which case, it works all the time. Don’t believe me? Compile the following program:

#import <Foundation/Foundation.h>

int main(int argc, const char *argv[])
{
    NSAutoreleasePool     *pool;
    NSString              *path;
    NSDictionary          *dict;
  
    pool = [[NSAutoreleasePool alloc] init];
    path = [NSString stringWithUTF8String:argv[1]];
  
    dict = [[NSFileManager defaultManager] fileAttributesAtPath:path
                                         traverseLink:NO];

    // The busy flag will be NULL if you comment out the next line.
    //NSLog(@"DICT: %@", dict);

    NSLog(@"Creation: %@", [dict objectForKey:NSFileCreationDate]);
    NSLog(@"Busy: %@", [dict objectForKey:NSFileBusy]);
}

By the way, I know I’m not releasing the pool.

Now, take a file and set its busy flag. You can do this as follows:

/Developer/Tools/SetFile -a Z some file

Then run my little program with some file as an argument. Notice how the busy flag is null?

Well, let’s see what’s in the dictionary passed back to us to find out what’s going wrong here. Edit the program above to uncomment the NSLog statement. Run it again. Notice how the busy flag is now set?

It appears that the dictionary is half-initialized until you happen to call certain methods (-allKeys works as well as -description). I haven’t tested every key but it does seem to be peculiar to NSFileBusy.

For the time being, I’m going to switch to Carbon to do this but I’d love to hear theories (or even facts) on what’s going on here. Let the wild speculation begin.

Oh, and since I brought up Heisenberg, I can’t let an opportunity to include an artful rendering of a dead cat pass by so here:

Over the years you learn a few lessons and make a few rules. You swear to yourself that you’ll always follow them but over time you get lazy or you just forget. Especially when debugging, you get in that panicky state where all that goes out the window. Urgency beats out common sense.

Usually when you find a crasher, you end up going at it head-on, in a brute force manner. Run it through the debugger. Carpet bomb the code with log output. Bang your head against the desk. Rinse. Repeat.

The lesson here is obvious (maybe painfully so) but I don’t see it applied enough in practice. I find that it’s commonly forgotten in the heat of the moment and is only heeded as a last resort. Basically, it is this:

If it worked before and is now broken, go back and look at what has changed.

Simple enough. Machines are deterministic and most systems you code are as well. Code usually doesn’t degrade and break on its own; something must have changed to cause the problem. Go back and see if you can isolate the build where the thing broke. If you have QA people, you can parallelize and have them isolate the build while you debug. Now look at the commit logs, starting at that build and go back. Many times, this will be the “a-ha” moment you needed. Even if the changed code is not the actual cause, it’s usually a trigger or it somehow exposes the problem. Now debug, focusing on the areas of those changes.

Now, this doesn’t always work. Maybe the code just acts differently after a certain date for some odd reason (Y2K anyone?) or there’s a hardware glitch or your program is getting input it never received before. But these cases are the exception. I have repeatedly regretted not checking the change logs but not once regretted doing it.

So, now that you know the rule, how do you make sure you follow it next time you’re looking down the barrel of a EXC_BAD_ACCESS error? Try this: set a timer for, say, 10 minutes and then start debugging. When the timer goes off, if you do not have a good sense of what’s going wrong, then stop and check the commit logs. How do you remember to set the timer? Actually, I don’t have good answer for that. Maybe you can write a hook into CrashReporter to start it for you. Or maybe you hire a magical pixie that appears whenever your program crashes and reminds you. Or you combine the two and have CrashReporter somehow wake up the magical pixie who will then remind you to set the timer. Then, just work in a bowling ball and a cuckoo clock and you’re golden.

This issue came up twice in the past couple weeks thus triggering my rule that if something comes up more than once then others may be interested in it. So, here’s a little tip that may save you some frustration.

When using regular expressions with Xcode’s search, the docs mention that XCode uses the ICU library. Naturally, you’d think it would use ICU’s syntax for specifying backreferences in the replace string, which would be to use variables consisting of a dollar sign ($) followed by the number of the capture group being referenced.

Of course, if that were the case, I probably wouldn’t be writing this tip. Using $ for backreferences turns “Search & Replace” into “Search & Destroy”. The syntax is to use backslash (\) instead of dollar sign. In short, \1 instead of $1.

Type the following in an Xcode editor window: “$1 works in replace strings.”

Now perform the following search & replace:

Ok, maybe this example makes things more confusing. If in doubt, just remember: $ = bad, \ = good in Xcode Find panel.

As to why Xcode does not use the ICU syntax in the replace strings, beats me. If anyone has a simple explanation, send it my way though I also welcome apocryphal anecdotes and crackpot conspiracy theories (extra points if you can convincingly implicate the Trilateral Commission).

This has come up a couple times in the past couple months leading me to believe that there are others out there who might benefit from this. I’m overdue for an post anyways.

The question: How do you draw shapes with holes in them? (i.e. How do I draw a donut?)

Short answer: Winding rules.

So what’s a winding rule? A winding rule is a way to determine whether a point is inside or outside a shape, useful for when you want to fill a shape. Quartz (like Postscript) has two different winding rules: even-odd and non-zero.

Non-zero is the default. How does it work? Take the point in question. Follow a line from that point extending out to infinity. Now, maintain a count. Increment the count every time you cross the path with the path going from left to right. Decrement whenever you cross a path going from right to left. In the end, if the count is zero, you are outside the shape, otherwise you are inside.

In the diagram above, the final count is zero. Note that this is a compound path (remember, paths do not have to be contiguous). Our imaginary ray crosses the path twice but does so with the paths going in opposite directions. This means that the point in the middle of both circles is considered outside the shape. Voilà , a donut.

With even-odd, it’s a bit simpler. Draw a line out from the point but just increment every time you cross the path (regardless of direction). If the resulting count is even, you are outside, otherwise you are inside.

Except for the labels on the axes, the pic below is actual output from a Cocoa program showing different combinations. The top two shapes have the outer and inner circles specified going in the same direction. The bottom two have the inner cirlce going in the opposite direction as the outer. The shapes on the left are drawn with non-zero winding while the two on the right are drawn with even-odd. Paths are stroked in white.

Notice how the direction of the paths does not matter for even-odd. Also notice how in the case of non-zero with the paths going in the same direction (top-left donut), the inner circle is considered a part of the shape’s interior. This shows that non-zero gives you more control (you can control what parts of the shape are considered inside by adjusting the direction of the paths). It is also more tedious to deal with for the more simple cases. To draw donuts using non-zero, I did the outer circle, created another path for the inner circle, reversed it and added it to the outer circle’s path. Using even-odd, all I had to do is set the winding rule on the path. For more complex cases, the control of non-zero winding may be required but for simple shapes like donuts, use even-odd. Why you’d ever want to draw anything else besides donuts is beyond me; donuts are delicious. While this knowledge can be applied to other less worthy shapes, I cannot be held responsible for the jeers you will undoubtedly receive for even daring to allow to cross one’s mind the slightest notion of contemplating considering drawing anything non-donut-like.

I’m too lazy busy to clean up the code for you to download but play with it yourself. It can all be done via NSBezierPath (key methods are -bezierPathByReversingPath and -setWindingRule:). Also, the winding rules can apply to clipping as indicated by the following pic:

For extra credit, play with self-intersecting paths.

Enjoy.

Unless you installed your operating system from floppies, chances are that you have a bunch of programs running on your computer at any one time. There are enough things that can go wrong in any one particular program and when you add them into the same soup, the interactions between them create more scenarios where things can go awry. The operating system has ways of preventing programs from mucking with each other via protected memory and resource limits but it’s not that surprising when some program will interfere with another.

It’s something I have to deal with a lot with Hazel. Since Hazel runs in the background and manipulates files it does not have full control over, there’s a lot to be watch out for. I have to coordinate with browsers to make sure not to move the file while it’s still being downloaded. I try and detect when the user is copying something with the Finder (which for some reason uses ‘brok’ as the file type instead of the ‘bzy ‘ or ‘bzy?’ types which I believe was the convention from the older MacOS days). I run my background processes with lower priority and even use low priority IO. Try this test. Take a large file and copy it. Turn off Hazel’s trash management options, move one of the files to the trash and use Finder’s “Secure Empty Trash”. Observe how usable your machine is while this is happening (for me, the answer is “barely”). Now, turn on Hazel’s trash management, setting the trash size threshold to something lower than the file size. Set Hazel to throw away oversized files immediately and to do so securely. Now, throw away the other file. You can tell something is happening but at least from all my tests, my machine is more responsive.

So, it’s a bit frustrating when you go through the trouble of making your software a good citizen when someone else’s software goes out of their way to interfere with yours.

A while back, I got a report from a customer that Hazel was not working. Looking at the logs, the customer noticed that it would stop working whenever he ran Logic Pro. When he quit Logic, Hazel would magically work again. At first, I didn’t believe it. It seemed too weird but I downloaded a trial of Logic Express and sure enough, I experienced the exact same behavior. After a bit of poking around I discovered what was happening. Logic Pro/Express stops all launchd jobs. Hazel uses launchd to start its background processes so it was a bit disconcerting to see another program, especially one from Apple, disabling yours on purpose, albeit indirectly. At least Logic is nice enough to start the jobs again when it quits.

Now, I’m sure the Logic team is probably doing this to ensure a level of performance but scouring the Apple lists and the web at large turned up nothing about this behavior. Maybe a tech note somewhere would have been nice. It’s unclear if any of Apple’s other pro apps exhibit this behavior but at least now I know what to look for.

Should this be a preference in Logic? Maybe a warning when it detects that the user has their own launchd jobs? I know it’s not common for software out there to depend on launchd but it is pretty annoying for those of us it does affect. I could write my own daemon using kqueue (which launchd uses to monitor file changes). Logic will not interfere with it then, but part of me feels that this would make Hazel impolite. Logic is doing what it’s doing for a reason and to work around its intentions goes against Hazel’s good manners. So for the time being, I’ll just have to accept that the housekeeper must go on break whenever the cops come in to commandeer the house.

Many of you probably know about this already but for those who don’t, here’s a nifty little tidbit for you developers.

NSColor has all those methods to retrieve system colors. Wouldn’t it be nice to see what they actually are? Bring up a color panel (doesn’t matter which app), select the color list icon (it should be the 3rd icon) up top. In the “List” pop-up, select “Developer”.

What you get is a list of colors with NSColor’s method names. In addition, it seems to update if you change your color scheme in System Preferences.

Not that this clears up all questions. Still not sure where those purple “knob” colors appear.

For the past couple weeks on and off (mostly off), I’ve been working on this gradient code. In my previous post, I mentioned my attempts at vectorizing it which resulted in no performance gain in the end. Here, I recount a tale of writing a generic gradient method in a NSBezierPath category. Yes, a tedious story, but this is a blog after all, plus you stand to get some actually-useful, free source code in the end so bear with me. Or, like a kid pouring all the cereal out of the box to get the glow-in-the-dark prize inside, you could, through the power of the scrollbar, skip all this and just grab the code.

So yes, I wanted to write a category to provide a method to fill the path with a gradient. Of course, you need to provide a start and end color but I also thought, why not also provide an angle? I didn’t need to do gradients at arbitrary angles (the only gradients I need right now are vertical) but when writing this stuff, I like to make it generic. When Apple starts using -23.728° angle gradients all over the place, I’ll be ready.

Now, the issue is figuring out the starting and ending points for the gradient given an angle. You want the gradient to fully fill the path. Let’s take filling a rectangle at a specific angle:

In the figure, with an angle α and a starting point A, you’d want the gradient to end at B to maintain the angle and also fill in the opposite corner of the rectangle. Recalling the trig and geometry from the deep recesses of my mind, I came up with the following equation (using known quantities w, h and α):

x = sqrt(h2 + w2) * cos(α - atan(h/w)) * cos(α)

I was a little disappointed that this didn’t reduce down to something a bit more tidy (makes you appreciate E = mc²) but it did seem to work. The problem: not every path is a rectangle. While using the above does cover whatever path may be bound by that box, it is not optimal. Another diagram is in order.

In this example, there’s slop at the start and end. Imagine pulling a squeegee from point A to point B. There’s a gap before you hit the shape starting from A and then a gap after the shape before you hit B. How does this affect the gradient? Why is Mr. Sun sad? See below:

All these diagrams were done in OmniGraffle (except for Mr. Sun which obviously required more sophisticated software). You can see that the “L” shape on the left has a truncated gradient (the black is somewhere where the upper right corner should be). As a user, I’d expect that if I had specified a gradient from white to black, that I would get that range within the shape, otherwise, I would have specified gray as the darker color. It appears OmniGraffle is shading according to the bounding box. Here’s another picture:

The first shape I freehanded with the correct orientation. The second is the “L” shape from the previous diagram rotated. Notice the difference in gradients. It seems to me the user should not need to care how the shape was originally created. The gradient should be white to black in both cases. By the way, I’m not picking on OmniGraffle; I was using it for these diagrams and decided to test it’s behavior in this regard since I had it running. My guess is that this is the common behavior for software implementing gradients with rotation.

If someone specifies an angle and start and end colors, then the final result should be at that angle and have those start and end colors visible in the gradient and not cut off, dag gummit! I feel in the ideal case, you should really be filling the shape with a gradient, not just providing a window to what ever portion of the gradient happens to be lined up under the shape.

The problem here is that we are using the bounding box in the original coordinate system. What would be ideal is to calculate the bounding box in a rotated coordinate system so we can get the minimum span along the axis of the gradient (the line going through A to B in the first two diagrams).

In the case above, we want the bounding box on the right.

My initial approach was to apply the path to the current state, transform the coordinate space to the rotated space and get the bounding box there. The problem with this is that I was getting the bounding box of the bounding box in the original coordinate system. In a sense, I was getting what I originally calculated above. The solution was to transform the path and not the coordinate system.

After some futzing and some stupid mistakes which sent me down some dead-ends, I got it working. Rotate the path, calculate the bounding box and set the start and end points to the edge of the bounding box that corresponds to the gradient axis, then do an inverse transform back on those two points to get them back into the original coordinate system.

What does this get me? Well, for me personally, not much as I mentioned before I only needed to do a vertical gradient. But you, gentle reader, get to enjoy the fruits of my labor. This code is provided under MIT license. If you use it then just mention me and the Noodlesoft site wherever you put your attributions. And of course, if you have any bug reports, fixes, suggestions or whatever, send them my way so I can address/incorporate them.

Noodle Gradient Test + NSBezierPath-NoodleGradient category

I was writing some gradient code and while writing the gradient function it occurred to me that it could be vectorized. I was doing the same operation in a loop to two sets of 4 floats (the red, green, blue and alpha values that need to be calculated for the gradient). The 4 floats fit perfectly into a 128 bit vector. My reason for doing this was not because of any bottleneck observed in Shark; it was just a flimsy excuse to play with Altivec for the first time. Nonetheless, if there was a noticeable performance increase, all the better.

In between a wedding, entertaining friends who were staying with me and getting a beta release of Hazel out the door, I read a little on Altivec and cooked up a little test program. You can download the source below. It’s PowerPC with Altivec only. I was sloppy with the #ifdefs but compiling the test without Altivec support is a bit pointless. I don’t have an Intel machine so I didn’t write an SSE version. For you Intel readers at home, feel free to add the appropriate SSE code (and let me know what you come up with). You’ll find my guess at what the SSE version is supposed to look like commented out in the code (but don’t trust it; I don’t exactly know what I’m doing here).

What does the test show? Well, at least on the G4 and G5 machines I have available, performance is roughly the same.

Of course, there are two ways to interpret this:

My test sucks
(obligatory remarks: “I didn’t know they taught programming at clown college.”, “My [insert feeble relative] can code better than you”)

A very likely possibility. This is my first foray into Altivec and I just started delving into Core Graphics gradient functions last week. As far as my Altivec code goes, I am still quite fuzzy on whether I should have used something like vec_ld or vec_splat to turn the multiplier into a vector though I’m pretty sure that using the float[]/vFloat union fulfills the byte alignment requirements (though it’s possible that it’s slower). I’m guessing it’s the difference between sticking it in a register and having it in memory. If any SIMD experts could educate me on this, I’d appreciate it.

Vectorization is not effective in this scenario

I’m sure those much more experienced with Altivec can explain this without resorting to a tactic that we like to call “making stuff up”. Me, I am going make stuff up. So here goes: I’m only computing one vector at a time and I’m doing all these scalar to vector (and back) conversions. My guess is that the overhead outweighs the reduced instruction count in the computation. If the CoreGraphics functions were structured such that it gave you all the values at once (or at least in chunks) instead of at each step of the gradient, then I could see an argument for vectorization made here.

So for now the result is inconclusive until I can get someone who knows what they’re doing to either verify or refute my test. Try it for yourself and let me know (especially if your results differ from mine). And please point out any deficiencies in my implementation (though comments on coding style can go to /dev/hell).

GradientTest (PowerPC with Altivec only)

An issue I’ve been dealing with is displaying a file path to the user. While this may be strongly discouraged (anyone have a cite from official Apple documents?), you have to admit that having a compact visual representation is useful at times.

With OS X being a collision of sorts between UNIX and the older Mac OS, there are two ways most programmers would conceive of a file path. The UNIX way would use slashes (/) with no drive at the root. In the Mac OS way, you’d have a drive at the root and colons (:) to separate the parts of the path. It was my impression that colons were on the way out but at the same time UNIX paths did not seem appropriate for regular users.

Trying to find examples in OS X itself, you find that inklings of both appear depending on where you look. For instance, doing “Get Info” on a file in Finder gives you the UNIX path. Command-clicking on the title of the Finder window gives you a pop up of all the ancestors of the currently shown file in the more Mac OS style (it shows the drive). The open and save panels have the same thing in the pop-up up top. No colons though.

Things get a bit more interesting when English is not your primary language. Set Spanish as your primary language in System Preferences and look at your sidebar. Notice anything different? Common folders are now translated. “Music” is now “Música”. “Movies” is now “Películas”. You no longer have a Desktop, you have an Escritorio. Now, this isn’t the problem (for programmers) that you think it is in terms of having to localize references to these paths. Just jump into Terminal or even do a “Get Info” on the file and you see the same UNIX path you’d see if everything were in English. These translations only exist to the user. That said, it would be nice when displaying a path to use the one the user would most likely understand so UNIX paths are out.

Now, let’s look at NSFileManager. There are two methods for getting a display version of a path:

– displayNameAtPath:
– componentsToDisplayForPath:

The odd thing here is that you cannot get a string with the whole path. -displayNameAtPath: only returns a translation of the last path component. -componentsToDisplayForPath: returns an array of the components. In both cases, they skirt the issue of what to use as a path separator.

In the current version of Hazel, I punted and just used what was shown in “Get Info”, which is the UNIX path (BTW, to show the full path in Hazel, just hover your mouse over a folder). Recently, though I caught Finder in the act of displaying a path.

Try this: drag a file into that space in the Finder window between the toolbar buttons and the search field. For those that don’t know about this feature, it’s basically a place where you can stick files that you can quickly access later. For you NeXTSTEP people, it’s a poor man’s shelf.

Now hover your mouse over the file on the “shelf”. There it is. Not a slash. Not a colon. The path separator is a triangle pointing right (▸). That’s unicode character 0x25B8, for those keeping score at home. You can see this somewhat echoed in Finder when you do a search. When you click on one of the results, you get that nifty path widget at the bottom with different triangles, but triangles nonetheless.

So is this standard? Dunno, but it’s good enough for my purposes and I’ve stuck it in the next version of Hazel (I’ve also added a contextual “Reveal in Finder” menu item for those that want to view the folder in context in Finder). If you see file paths popping up anywhere else, let me know.

[IMPORTANT ADDENDUM: I have received requests to be even more emphatic in discouraging use of these icons. Using undocumented API does create a legacy that makes it hard for the Apple engineers to move forward with the API. So, in addition to the apocalyptic vision presented in the original post below, I, along with my cohorts, will start calling you “stinkypants” so, DO NOT USE THESE IMAGES IN A SHIPPING APP. I leave this article here out of archaeological interest but I am willing to remove it for the greater good of the Republic.]

To keep with the icon theme, let’s look into another source of icons.

Under Cocoa, you can also get access to certain system images via NSImage’s +imageNamed: method. Give it a string and it gives you an image registered with that name. For system images, the catch is that only a handful of those names are documented. Here are the ones I could find in the docs:

• NSAscendingSortIndicator
• NSDescendingSortIndicator
• NSApplicationIcon
• DRBurnIcon
• DREraseIcon

The docs also state that +imageNamed: searches the Appkit Framework. Digging around, I’ve come up with the following:

• NSDefaultApplicationIcon
• NSDeadKeyMenuImage
• NSGrayResizeCorner
• NSHelpCursor
• NSMysteryDocument
• NSMultipleFiles
• NSOpacitySlider
• NSUtilityInactivePattern
• NSUtilityKeyPattern
• SpellingDot
• NSTriangleNormalRight
• NSTrianglePressedRight
• NSTriangleNormalDown
• NSTrianglePressedDown

As with the Icon Manager icons, some of these are not currently used (old NeXTies should find the NSMutipleFiles image familiar). While most of these are prefixed with “NS” making them seem official, they are not documented explicitly, only indirectly via the allusion to images in the Appkit Framework.

It’s unclear whether one can count these images sticking around from release to release but my guess “no”. Usage of these images may result in plagues of locusts (in the form of people in three-piece suits with “Esquire” tacked to the end of their names) descending upon your home as it is simultaneously invaded by fire-breathing hippos who proceed to trample your worldly possessions and imbue your commemorative plate collection with a strange odor making them unsuitable for sale on Ebay, the worst of it being that your insurance will not cover any of it. Or maybe all that happens is that milk starts tasting funny. Try it out and let me know.

But even if you don’t use them in a shipping app, they can be useful as a temporary stand-in until you or a graphic artist makes a knock-off.

You can preview these images in InterfaceBuilder by setting them on a control (just type in the name). At least on my machine, IB shows a broken image when you enter any of the above but cutting and pasting the control seems to make the image appear. Maybe at some point I’ll add these to the System Icon Viewer.

By the way, if anyone knows of any named images that I missed, send them my way.

Enjoy.