Noodlings

[This was the topic of my presentation at the NYC Cocoaheads meeting last night. I thought it would be nice to also post on the topic here.]

[I’ve received email from Ken Ferry. See addendum at the bottom]

NSImage is a troublesome class. Over the years, it’s been misunderstood and abused. I think much of this is because of a lack of conceptual clarity in the docs and examples and the API itself can be confusing and misleading. Add to this having to mix with CGImage and CIImage and you can end up with a confused mess.

The way I like to think of NSImage is that it’s a semantic image. When you look at icons, they are made up of several versions at different resolutions. Technically, it’s 4 or 5 images but NSImage wraps those all up into one notion of an image. Semantically, it is an icon of, say, a house but underneath it’s made up of several actual images of a house. The main reason for these different versions is that some graphics do not scale well and it helps to have hand tuned versions, especially for the smaller sizes. You can also do things like omit certain features in the smaller sizes to simplify the graphics and make the icon more recognizable.

One key misunderstanding is the notion that NSImage has actual image data. While there are parts of the API that deal with a cached bitmap (more on this below), NSImage itself is not based on image data. It is best to think of NSImage as a mediator between the image data in the various representations and the drawing context.

Loading an image from a file and drawing it is usually straightforward. If there are multiple representations, NSImage will figure out the correct one to use when drawing it. Most of this is automatic and is an important key to understanding how to use NSImage. As you will see further on, bypassing this mechanism leads to many issues with size and resolution mismatches when processing NSImages.

I’ve created a little program to help illustrate the points in this article. Since it also includes a new reusable class, I’ve included it in my NoodleKit repo. I suggest checking it out. Just compile and run the ImageLab target (you may need to set the active executable in XCode in the project menu). The rest of this article will be referring to it so I recommend you run it while reading the rest.

When you launch the app, it will load the test image. It’s just a colored circle. Now resize the window. You’ll see that the color changes as you resize. The icon itself is made up of four different sized images and to make it clear which representation is being used, I made the circle different colors for each one. Whenever the color changes when you resize, NSImage is switching to a different representation based on the size. Here’s what the different reps look like in Preview:

Size is another confusing aspect of NSImage. One thing to remember is that size is not pixels. It represents a coordinate space. One way to think of it is that NSImage’s size is like NSView’s frame while NSImageRep’s size is like NSView’s bounds. For NSImage, its size is a suggested size to draw the image in the user coordinate space. If possible, it’s best to explicitly specify the destination rect.

Let’s take the case of drawing some custom graphics on top of an existing icon with several representations, like drawing a badge over your app icon. A common way to do this is to lock focus on the NSImage, do your drawing, unlock focus and then draw the resulting image. What -lockFocus actually does is allow you to draw into a cache. This has probably lead to a lot of the misunderstanding that NSImage holds image data. Unfortunately, there are issues with this approach. Mainly, because there is no context about where this image will be drawn so the resulting image is tied to a specific resolution. Also, you are editing a cache so none of this image data is reflected in the original image reps and actually, from poking around, it’s actually destructive in that it may end up removing any other representations.

In our case here we are modifying an icon with different sized representations, what we end up doing is locking the resulting image into the size that happened to be set on the original NSImage. In many cases, you may not know how and where this icon will be used but if any scaling is involved, you may end up having an icon with the wrong version being displayed. In the example program, select “Modified (lock focus)” in the pop-up. Here it picks the 64×64 version (as indicated by the green circle) which becomes a problem when you scale the image up as shown on the left in the image below.

The one on the right is a version which uses an  NSImageRep subclass (select “Modified (custom image rep)” in the pop-up). Notice that it picks the appropriate size as you resize the window. Why is this the case? It’s because NSImage doesn’t ask the image rep to draw until the NSImage itself is drawn. At that point, you actually have information about the destination, including how big it will actually appear on screen. NSImage is able to use this context to determine the right match between different sized images and the resolution of the output context. The same goes for any drawing code.

Subclassing NSImageRep is quite easy; you just need to override the -draw method. I’ve made it even easier by providing NoodleCustomImageRep which takes a block, allowing you to create images without creating a new subclass.

Say, though, you are drawing an image that should scale without pixellation, like drawing a square. Surely, you can just lock focus on an NSImage and draw a square and scale that as needed? Well, Mr Smarty Pants, take a look at the example program. Select “Drawn (lock focus)” in the pop-up.

Here, you’ll see odd fuzziness around the edges. What’s going on here? It’s not anti-aliasing but instead the graphics context where the image is being drawn has image interpolation turned on. As a result, the AppKit is trying to interpolate the image from it’s original size to a much bigger size.

How do you fix this? You could try turning off image interpolation in the destination graphics context but this isn’t always possible or desirable. The better solution is just like before: use a custom image rep to do the drawing (select “Drawn (custom image rep)” from the pop-up). Since the drawing occurs at drawing time, instead of image creation time, it knows about the context it is drawing into and therefore can provide your drawing code with a context at the correct resolution. The crisp square on the right speaks for itself.

Let’s take another example. Say we want to take an existing image and run a Core Image filter on it. Somehow you have to convert your NSImage into a CIImage. This usually entails a game of connecting up various methods that fit together until you got a CIImage. A common way to do this is:

ciImage = [CIImage imageWithData:[nsImage TIFFRepresentation]];

This dumps the whole image (all of its representations) into TIFF format which is in turn, re-parsed back into data. Now, CIImage is pixel based so it ends up picking one of the representations. It’s not documented which representation is picked since there’s no way to specify the context where it will be drawn so there’s a chance it won’t be the right one. Select “Core Image (TIFF Rep)” in the pop-up and you get something like the image on the left (or maybe you won’t; read on):

Now, it doesn’t look too bad here. It took the highest res representation and used that. That said, technically, it’s not correct. The version on the right (select “Core Image (custom rep)”) shows the correct image rep being used. Also, my experience has shown that the representation chosen by the -imageWithData: method can be different on different hardware and that it ignores the size set on the original NSImage so you may not be so lucky depending on what machine your code runs on.

Fortunately, Snow Leopard introduced a new method: -CGImageForProposedRect:context:hints:. As mentioned before, when you draw an NSImage into a context, it will automatically pick the right representation for that context. This method does basically the same thing but without the drawing part. Instead it returns a CGImage which you can then use to create your CIImage:

cgImage = [nsImage CGImageForProposedRect:&rect context:[NSGraphicsContext currentContext] hints:nil];
ciImage = [CIImage imageWithCGImage:cgImage];

Keep in mind, though, that it the image returned might not be the exact size you wanted. This becomes more of an issue when you are combining multiple images together in a CIImage pipeline and you need them all to be the same size. You can adjust for this by using CIImage’s -imageByApplyingTransform:.

In addition to the above method, Snow Leopard introduced -bestRepresentationForRect:context:hints: which does a similar thing but returns an image rep instead. Depending on your needs, you can use one or the other to tap NSImage’s image matching logic.

Finally, a note about performance. NSImage does keep a cache based on the drawing context. This helps for when you repeatedly draw the same image at the same size over and over. If you end up sharing an NSImage across different contexts, you’ll find that you are defeating the cache. For these cases, you should be copying the NSImages. Remember that NSImages are just mediators between image data and drawing context. NSImageReps are the actual image sources and, starting with 10.6, reps like NSBitmapImageRep do copy-on-write making it inexpensive to copy NSImages and their reps.

In the example app, there’s a field which shows the time taken to display the image. You’ll notice that when you resize the window, the cases which use a custom image rep are slower as it has to recache whereas the lockFocus cases don’t since the image is static. If this becomes an issue, you can turn off caching or use a fixed resolution image during a live resize. Another more subtle piece of business is performance when drawing from the cache. If you click the “Redisplay” button, it will cause the image to be displayed again. Since you aren’t changing the size, the cached version can be used. Notice how the versions using the custom image rep are usually a smidgen faster than the lockFocus versions. I suspect what is happening is when you lockFocus on the image, you lock the cache into a specific version and size. As a result, if you are drawing at any other size, it has to scale the cached image every time. With a custom image rep, it’s cached at the exact size so the cache can be used as is.

What are the lessons here?

1. When defining your image, you should use an NSImageRep subclass and override -draw. If you don’t want to create a whole subclass just to create an image, use my NoodleCustomImageRep (included in NoodleKit) which allows you to pass in a drawing block. Using an image rep gives your drawing code better contextual information than you would get just drawing in a -lockFocus.
2. If you follow point #1, then you can let NSImage make the decision of which representation to use. Use  one of the drawing methods, -CGImageForProposeRect:... or -bestRepresentationForRect:... and you’ll get the best sized representation for the job. Do not assume, though, that this representation will be the actual size you want. When drawing, it also helps to specify the rect to draw into.
3. Avoid using -lockFocus. It doesn’t produce the correct image in different contexts and can be destructive in terms of kicking out the other reps in the NSImage. While still ok in specific circumstances, you have to know what you are doing.
4. If using the same NSImage in different contexts, copy it. In 10.6 onwards, this is an inexpensive operation as bitmap data is copy-on-write. Copying NSImages is is also a good idea in case some decides to use -lockFocus on the image (see #3).

If you have access to it, I highly recommend watching the video for Ken Ferry’s session at WWDC 2009: Session 111: NSImage in Snow Leopard (you may need a developer account to view/download it). Much of this is derived from that presentation and it has even more interesting bits about NSImage than what I’ve presented here.

And in case you missed it above, you can find NoodleKit (which contains NoodleCustomImageRep as well as the example program) here.

Addendum (Apr. 18, 2011):

I received an email from Ken Ferry himself pointing out a couple things. Mainly, that as of 10.6, lock focusing doesn’t draw into the cache anymore. It creates a representation whose context is suited for the main display. It is still good for images which are meant for that context. Also as the NSImage context will maintain any size-to-pixel ratio that is on the main display, it can still be used in this situation should resolution independence come into play. It’s not much different than before in this respect but it’s not as volatile as a cache, which I believe is the key point here.

That said, all the caveats above about using -lockFocus still hold true. It’s not so great for when the image needs to be scaled or if you have representations you want to keep (it does remove all other reps when you lock focus). Also, because the representation is tied to the machine, it’s not very suitable for persisting.

In my last post, I introduced a little adapter class called NoodleGlue. It’s a simple class that wraps around a block allowing you to use it as the target and action for APIs that require it. It also optionally can take an extra block that is executed when the glue object is deallocated. This allows you to do any cleanup. In my NSTimer category from last time, I used this cleanup ability to allow the object to unregister from any notifications. Being a category, I couldn’t hook into the -invalidate or -dealloc methods of NSTimer so I created a glue object with a clean up block which I then set as an associated object on the timer. As a result, when the timer was deallocated, so would my glue object and my cleanup block would be triggered.

In more general terms, what does this mean? With my glue object plus the associative references feature, you can inject your code into any object’s deallocation sequence. It’s like being able to be notified if any object is freed. I’ve wrapped this up in an NSObject category with a new method, -addCleanupBlock:, which allows you to add a block to be invoked when the object is deallocated. It’s probably more of a proof of concept than something you’d want to use in production code but you can find it in the latest version of NoodleKit (it’s tacked onto the end of the NoodleGlue class).

Now, this sounds pretty cool on paper but this may be one of those cases of a solution looking for a problem. Not to be deterred, I’ve come up with some contrived cases that might actually be useful.

Add extra cleanup for extra stuff you attached to an object

It’s how I used it in my NSTimer category. You can unregister notifications and invalidate objects which need more than just a -release call.

Zero out references

One of the cool things about weak references in a GC environment is that they nil themselves out when the object is collected. You can simulate this behavior when using retain/release by setting up a cleanup block to nil out your reference to the object in question.

Tracing/debugging deallocations

Sure, you can do this in the debugger or Instruments but in some cases you want to trigger more involved code that you can’t do in gdb. Also, in a multithreaded program where the timing affects whether the bug shows up or not, this allows you to track deallocations without setting a breakpoint in the debugger which might otherwise change the timing.

Test if objects are actually deallocated (unit tests)

Credit goes to Guy English for this one. If you need to do a unit test where you want to test your memory management, you can “do the glue” to test that objects actually get deallocated. The problem here is if it’s not working (i.e. object doesn’t get deallocated), your block won’t get called so any assertions you put in there will just silently not happen. So, have your block set a variable and at the end of the test, do an STAssertTrue() on the variable.

Keep in mind that there are a couple of important caveats:

1. You cannot retain the object you are “observing” in the block itself as this will create a retain loop and the observed object (as well as the glue) will never be dealloc’ed. Note that just referencing an object retains it unless you assign the object to a variable declared with __block. This has already been done for you in my code as the cleanup block is sent a non-retaining reference to the object being freed so use that reference instead.

	[self addCleanupBlock:
		^(id object)
		{
			// Do not reference "self"
			[object doSomething];
		}];

2. It’s not defined when in the deallocation process the associated object is freed. As a result, you can’t rely on any state of the observed object as it may be the case that all its ivars have already been released. But, you can have the block “capture” variables and the block will retain and subsequently release them. For instance, if you want to log the “name” of an object as it’s deallocated, you can reference the name in the block and it will be retained. But be careful though, if you reference an ivar in the object directly, you end up retaining that object. So, if the observed object is creating the block and you access its ivar in that block, you end up implicitly retaining the observed object, which is in violation of point 1 above. In those cases, assign the ivar to a local variable and use that (in this case, do not use __block in the variable declaration as you want the block to retain the object.

	NSString	*localName = _name;	// _name is an ivar
	[self addCleanupBlock:
		^(id object)
		{
			// Do not reference _name here as that will implicitly retain self
			NSLog(@"Object %@ has been deallocated.", localName);
		}];

I suggest reading up on the memory management issues with blocks as described here. Joachim Bengtsson has a great article on blocks (that link goes to the section relevant to this article but I recommend reading the whole thing) and there is always Mike Ash’s reliable series of articles on the subject, which I have consulted many a time myself.

It’s been a long while. Part of it is that I’ve been busy working on Hazel 3. And since I’m not interested in writing a book, I’m finding it hard to be motivated to keep posting here. Nonetheless, every so often I have to let something squirt out.

Today, we are going to talk about NSTimer. One thing you may or may not have noticed is that in certain cases where time is suspended (putting your machine to sleep) or altered (changing the clock), NSTimer has a tendency to try and compensate. For instance, say you set a timer to fire in an hour. You close the lid on your MBP and come back 30 minutes later. You’ll see that the fire date for the timer has adjusted to take into account the time it was asleep. Here’s a quickie diagram for those not quite following:

Now, this is great if you wanted a timer to fire an hour later in the machine’s conception of time. But there are times when you want it to fire on the actual date you set on it. Typical example is setting a timer for a calendar appointment. That time is an absolute point in the user’s timeline and you don’t want that timer to shift to compensate for the machine’s timeline.
The basic fix here is when one of these time-altering events occurs, you reset the fire time on the timer to the original time that was set on it when it was created. Luckily, there are notifications for when the machine wakes from sleep as well as when the system clock is changed (this one being new in 10.6). Naturally, one would think to make an NSTimer subclass as the most straightforward way to do this. Unfortunately, it seems that it doesn’t work. Even if you get over the hurdle that NSTimer is actually abstract (like a class cluster), you have to add the timer to the run loop to schedule it. While NSRunLoop seems to accept the NSTimer subclass, it doesn’t ever fire. I don’t know if NSRunLoop is looking for the NSTimer class specifically or if it’s calling some private method that I need to override. If someone else has had any luck doing this, drop me a line.

The alternative is a category. The problem is that we need at least one ivar to store the original fire date. Normally, you can’t add ivars in a category but in this glorious post-10.6 age, we can, via associative references. It’s basically just a dictionary where you can store variables associated with an object. Sure, you could have implemented that yourself but the nice thing is that it handles memory management as well. When the object is dealloc’ed/collected, depending on the memory management you specify, it can also release/collect any associated objects.

So, NSTimer category it is. It supplies the method (among others) +scheduledTimerWithAbsoluteFireDate:target:selector:userInfo: which creates and returns an autoreleased and scheduled timer that will fire on the given date regardless of any time fluctuations/lapses. If all of a sudden the fire date has passed (like if the machine was asleep during the fire date or if the system clock was set ahead), it will fire immediately. Note that calling -setFireDate: won’t quite work in the sense that if the timer ever has to be reset from a time shift, it will be set to the date used when creating the timer, not the one that you set it to afterwards. This is an implementation limitation as I am using a category and can’t override methods (at least not without doing some method-swizzling which is not something I want to deal with). In cases where you need to change the fire date, I’d suggest just creating a new NSTimer.

I’ve also included a test harness where you can see it in action compared against a timer running in normal (non-absolute) mode. When run normally, both timers should fire at the same time but you can test things out by changing the system clock or putting your machine to sleep. You’ll see that the regular timer’s fire date will start to drift out while the “absolute” timer will stick with the time originally set.

I also went ahead and added methods to have NSTimer to use blocks instead of a target/selector combo. Note sure why this wasn’t in there already but I thought it’d be useful.

And that’s not all! Included is my NoodleGlue class. It’s a simple little class that just wraps a block (plus another block for cleanup, if you need it). It’s useful for cases where you want to set a target and selector for some object to use but don’t want to create a new class or method for it. Check out the source code for the NSTimer category to see how its used both with NSTimer (to implement the block API) as well as NSNotificationCenter. In the latter case, there is a blocks-based API but I had special memory management requirements that couldn’t be done with the existing API.

Needless to say, this extension only works on 10.6+. It’s in the latest version of NoodleKit and as a result I’ve made NoodleKit require 10.6 as well. Just build and run the TimerLab target. Fixes and suggestions welcome.

[Update 8:35pm EDT] I neglected to push the code to the github repository. Sorry about that. Everything should be there now.

I was just turned on to a post from Jesper of Waffle Software entitled How to Mimic the Artwork Column in Cocoa. There he approaches the problem of implementing cells in an NSTableView that can span multiple rows. This spurred on Jacob Xiao at Like Thought to tackle the problem in a different manner. I suggest reading both of the above to get a sense of the problem and the possible approaches.

With NSTableView still on my mind from my last diversion, this got me thinking about the problem and I decided to try my own hand at it. I preferred Jacob’s approach as I think it should be all contained within NSTableView if at all possible. I liked the notion of “slicing” up the cell instead of using another view. Playing with it, I came up with a generalized solution. Here are some of the interesting aspects:

• It can support any NSCell subclass. I do this by using a special wrapper cell. It draws the “full” version of the cell then draws out the slices for each row into the tableview.
• It determines the spans by finding contiguous rows that have the same object value for the column in question.
• The above two points are exploited to do some caching such that the “full” cell is only drawn once and the parts composited in place as needed.
• If the first column is set to span rows, it will only draw the grid for the last row in a span.
• You can designate multiple columns to do this. In the example project, if you click on the “Album” column, it will toggle between span and non-span mode. The “Artwork” column doesn’t do this because it looks cruddy in non-span mode. I haven’t tested it (see update below) but theoretically, the spans do not have to line up between different columns.

To use it, for any columns you want to have this row-spanning behavior, set the table column class to NoodleRowSpanningTableColumn. Also, it might help to get rid of any intercell spacing in your tableview (at least in the vertical direction). There are probably a few bugs. I do know that reordering columns may cause some grid drawing funniness but for the most part, it seems to work pretty well. Let me know if you find otherwise.

You can check the code for other implementation tidbits. I’ve included a self-contained project below but I’ll eventually polish it a bit and add it to my NoodleKit repository where it can join the rest of my goodies.

Many thanks for Jesper and Jacob for the inspiration and laying down the groundwork for this.

Update (Oct. 21, 2009): I observed odd behavior in that the program would get into a re-drawing frenzy eating up a bit of CPU. This was non-blocking in that I was still able to manipulate the UI (which is why I didn’t notice it at first). I finally figured out that it was the -drawGridInClipRect implementation. I was drawing the grid selectively by changing the grid mask and then calling super on subregions. Unfortunately, changing the grid mask queued up a redraw which started a loop (albeit one that didn’t tie up the event loop). Seeing as I needed to fix the grid-drawing logic anyways, I reworked the whole thing.

The new grid code should now be able to handle multiple spanning columns rearranged in whatever way you want. Here’s contrived example of having multiple spanning columns that don’t necessarily line up:

I also made some other fixes and improvements so it’s worth re-downloading the project even if you already did so before (I bumped this version to 0.37).

As for integration with the rest of NoodleKit, I still need to decide whether I want to merge this with my sticky row tableview to make some über-tableview. Problem is is that I can’t think of how the two features would interact so maybe it’s best to keep them separate. Thoughts on this are welcome.

Update (Oct. 23, 2009): I’ve just updated the project to version 0.68. There was a crasher when you would click on a non-span row and then move into a spanning cell. Also, the span cells were highlighting when you clicked on a non-span row. Both of these have been fixed. Thanks for Jesper for finding the crasher and Jacob for helping with the fix.

Update (Oct. 25, 2009): I’ve consolidated this functionality along with my previous sticky row header code into a single NoodleTableView class. You can find it in the NoodleKit repository. Future updates to this will be posted there from here on out.

Download NoodleRowSpanningTableViewTest.zip
[This project is no longer being updated here. You can get the latest in the NoodleKit repo]

I’ve finally gotten around to consolidating most of the bits of code I’ve posted on this blog over the years and put it in a repository. While the real NoodleKit is a much more extensive and cohesive toolkit that I use internally, this one will serve as a place where I put the odd scraps that I decide to open source. I guess I could’ve called it NoodleScraps but it doesn’t quite sound as nice.

If you’ve been following my blog for a while then there’s not much new here, codewise. There are a tweaks here and there plus the stuff should be 64-bit ready so it might be worth re-downloading just for that. It will be the place where I put future code so you should keep an eye on it.

The project is structured to build a framework but also has an examples directory (with corresponding targets) to demonstrate the use of the different classes. The current notion is to keep things compatible with 10.5. And, as usual, all code is released under the MIT license.

NoodleKit (hosted at github)

Enjoy.

Once again, I find myself with some cool code with nowhere to put it. And, yet once again, you get the reap the benefits.

UITableView on the iPhone has the neat feature where section headers stick at the top after the actual row for it has scrolled off. For instance, in Contacts, if you scroll down, the row for section “A” stays visible so that you know what section you’re in if the section is too large for the screen.

Unfortunately, someone asked how something like this could be done on desktop Cocoa. I say “unfortunately” because I was within earshot and it resulted in me embarking on a journey peppered with annoying run-ins with NSTableView’s idiosyncrasies. Nonetheless, I persevered and came up with something hopefully someone will find useful. As was the case before, I don’t have a use for it myself at the moment, but I feel it would be a shame to let the code go to waste.

I am presenting NoodleStickyRowTableView. I call the section headers “sticky rows” as one can specify any row to stick up top for whatever reason, though by default, it does it for group rows.

The bad stuff:

NSTableView makes it very difficult to cache a row visually. -drawRow:clipRect: should be able to do it but the problem is that it’s too smart for it’s own good. If the row is not visible, it will not draw anything. Add on top of that the impossibility of getting it to draw the special background the group rows use. I tried all sorts of techniques but none were general or reliable enough. My email on the cocoa-dev list was, not surprisingly, ignored. As a result, in the default implementation, I use my own look for the rows and then use a fade-in transition since the sticky row will look different from the actual one. You can see it in the video below where it has a ghost-like effect.

The good stuff:

The good news is that I managed to work it into an NSTableView category. You can think of it as an abstract category if you wish. In your subclass, at the very minimum, override -drawRect: and call -drawStickyRowHeader after calling super. From that one call you will get all the functionality.

Why do it this way? Well, ease of use for one. It makes it so that you can integrate it into your own NSTableView subclass without having to copy and paste large swaths of code or put it all sorts of odd hooks. Secondly, the changes get inherited by NSOutlineView so the functionality is available there and in its subclasses as well.

While I could use the new associated objects functionality in Snow Leopard, I opted for Leopard-compatiblity. This resulted in me coming up with a couple tricks to get around the lack of ivars, including the rarely used -viewWithTag: method. Check the “read me” file and code for details.

There are two different transitions. The default is the fade-in transition as shown here:

As mentioned before, since I couldn’t get the group rows to draw properly and consistently, I use a more generic look for the row. The transition makes it less jarring but it is a compromise of sorts. Not bad when you consider that you get all this by calling just one extra method in your -drawRect:.

But what if you want something more like the iPhone’s version? In your subclass, you can override -stickyRowHeaderTransition to return no transition and then do your own drawing of the rows so that they look the same between the regular and sticky versions. The NoodleIPhoneTableView included in the project shows how this is done with the result being something on par with UITableView:

I am releasing this under MIT license as usual. Use it however you want as long as you give me credit somewhere. Please send in any feedback, suggestions, bugs, etc but keep in mind that this is not something I use in my projects so I probably will not be actively maintaining it beyond a certain point. I’ve made a point of making the the API easy to use and to extend so have fun with it.

Download NoodleStickyRowTableViewTest.zip (version 0.18639)

Update (Sep. 29, 2009): I have included this class in my NoodleKit repository so you should check there for future updates.

[Warning: this article is for entertainment purposes only. Any harm you do to your code or any offense you incur on the sensibilities of other programmers is your responsibility.]

Some time ago, Guy English posted a great and twisted article on key value coding (KVC). He comes up with a nice hack to do stuff with KVC that was never intended. Of course, there are some neat ways you can use KVC without indulging in the craziness that Guy English delves into (and I think he’d be the first to admit that it’s not meant for primetime). The main point remains, which is that using KVC with NSArrays is a great way to avoid coding loops over their contents. You have an array of “Person” objects and want an array of their names (accessible via a -name method)? Just do…

newArray = [array valueForKey:@"name"];

Now, the keys do not necessarily have to be properties of the object. There’s nothing stopping you from calling other methods on your objects that return other objects, provided they don’t require any arguments. For instance, if starting out with an array of strings…

newArray = [array valueForKey:@"lowercaseString"]

…will generate a new array of lowercase versions of the original strings. Actually, the methods don’t even have to return objects. Want to convert an array of numbers in string form to numbers? Try the following:


array = [NSArray arrayWithObjects:@"1", @"2", @"3", @"4", nil];

newArray = [array valueForKey:@"intValue"];


This gives you an array of NSNumbers. KVC auto-boxes scalar types as needed. It’s a great way to do conversions/transformations of arrays of objects in one fell swoop.

Suppose you want to do a deep(er) copy of an array of objects (not only copy the array, but each of its objects as well):

newArray = [array valueForKey:@"copy"];

There is the issue, though, that the copies in the array now have a retain count of 1 and will be leaked unless extra care is taken. But no need write a loop to autorelease the objects. Instead, we can use keypaths to create chains:

newArray = [array valueForKeyPath:@"copy.autorelease"];

Retain it now and release it later or just let it get autoreleased. Either way, everything cleans up nicely.

And if you have a bunch of NSMutableCopying-conformant, immutable objects and want to convert them to their mutable counterparts:

newArray = [array valueForKeyPath:@"mutableCopy.autorelease"];

That gives us an array of autoreleased mutable copies.

Sure, not super mind-blowing but hopefully something above may save you some keystrokes. If you have your own KVC tricks, post them here in the comments.

[Update 2009-06-30]

This all started out with an conversation I had with Guy about wacky KVC stuff. The original title of this post was “KVC Abuse” but after a few edits, it got lost. I added the warning at the top but I’ll make it clear here: this is an abuse of KVC. This whole post is an indulgence.

I avoided a whole discussion of implementing HOM (Higher Order Messaging) type functionality in Objective-C but here’s a method you can use in an NSArray category:

- (NSArray *)map:(SEL)selector
{
	NSMutableArray		*result;
	
	result = [NSMutableArray arrayWithCapacity:[self count]];
	for (id object in self)
	{
		[result addObject:[object performSelector:selector]];
	}
	return result;
}

You don’t get the auto-boxing or the keypath stuff but the end result is still succinct and convenient for the basic case:


newArray = [array map:@selector(lowercaseString)];


Of course, my own version that I use has a more verbose method name (-transformedObjectsUsingSelector:) but no matter how you slice it, it will generate less bile from other developers.

Flipped coordinate systems always seem to confuse people, myself included. After working things out with pen and paper or handpuppets, I always feel like they’re not quite correct so I thought I’d resolve things here. If anything, I can refer back to it whenever I forget this stuff (which happens every couple years). First a refresher in case you need it: Cocoa Drawing Guide: Flipped Coordinate Systems.

The main thing to keep in mind is that while a flipped coordinate system is basically translating the origin to the top left and flipping things vertically, semantically it doesn’t mean that everything is upside-down. Images, text and other elements are supposed to render right-side-up. The flipped coordinates should affect where those elements are placed, not how they are rendered. Being flipped is a higher level notion and is separate from the
CTM. While setting something as flipped will flip the CTM, modifying the CTM without setting the graphics context as flipped results in everything being entirely upside-down, which is different. Many of the Cocoa classes and functions take flipped coordinate systems into account and will draw right-side-up for you but it gets tricky with images.

There are two sets of methods in NSImage to render it in a graphics context: the -composite... methods and the -draw... methods. I’ve put together an interactive example app to help illustrate the differences. It might help you to download it now so you can follow along. The figures included in this article are from that app

The -composite... methods actually seem to work in the sense that the image is always right-side-up. The problem is that it draws “upward” from the draw point regardless of the coordinate system. The result is you have to calculate the compositing point at the upper left corner of the image instead of lower left. This is the result of these methods not taking the CTM into account in terms of scaling and rotation. That also means that if you scale your view up, the images won’t scale with it. While you can use this to your advantage in some cases (like rendering resize handles or labels which you want to stay the same size regardless of zoom), it’s usually not the right way to do it.

View scaled at 2x. Notice how the composited version does not scale.

The -draw... methods on the other hand follow the CTM properly. That also means that if the view is flipped, so is the image, so you need to adjust accordingly. So while these methods obey the CTM, they do not take into account the flipped flag of the context which would be the cue to draw things right-side-up.

The draw... routines follow the CTM, but not the flipped flag.

Now, to complicate things even further, NSImage’s themselves can be flipped. The reason for this isn’t to make the images upside-down. It’s there to provide a flipped coordinate system for whenever you draw into it (i.e. lockFocus on it, draw, then unlockFocus). It’s useful for when you want to tell a flipped view to draw into an NSImage, for instance. It’s basically like a flipped view; you don’t expect a flipped view to draw upside down, no matter which view you set as its superview. A subtlety to be aware of is that flipping an image loaded from a bitmap does not make too much conceptual sense (you are changing the coordinate system after the content has already been “drawn”) but it does have the practical effect of flipping the image vertically, which seems to be an implementation detail. Yes, flipping the image will work to “correct” orientation problems in most cases but, depending on where your image gets its data (for instance if it has an NSCustomImageRep like the example app – see below) and whatever implementation-specific details lurk in NSImage, you may end up with undesired or inconsistent results.

As mentioned above, I’ve put together a little interactive example app (Leopard-only) to show how the different methods behave. In addition, I’ve written methods in an NSImage category (-drawAdjusted...) which will render the image correctly regardless of the flipped status of the coordinate system it draws into. As suggested in Apple’s docs, it does a transform reversing the flip, draws the image, then reverts the coordinate system back.

The image itself is drawn by code, not loaded from a bitmap. The reason for this is that I also wanted to illustrate using flipped images. It draws an arrow and some text to indicate the proper orientation. When flipped, the drawing code is exactly the same in that no coordinates are recalculated. The text content is changed to compensate for the new orientation. Notice how the text renders right side up no matter the flip state of the image; an indication that the NSString drawing methods are “flip-aware.” Also, it shows how to check the graphics context to get its flipped status so you can make your own drawing routines flip-aware as well.

Unfortunately, not everyone draws flipped images correctly. One place in particular is NSView’s/NSWindow’s -dragImage:at:offset:event:pasteboard:source:slideBack: method which will draw flipped images upside-down. Since you can’t control how the image is drawn, you can instead draw your flipped image into another non-flipped image and pass that in. I’ve added a method to the NSImage category to do this and you can check out the result in the example app (you can drag the image out of the views though only the last one has the corrected version).

And what if you actually want to draw everything upside-down, you irrepressible nut? Well, apply your own transforms using NSAffineTransform or CGAffineTransform. Just remember to concat the transform and not set it (a good general rule when using affine transforms). As long as you don’t tell any classes that you’re flipped, it should work out.

Hopefully this didn’t make things even more confusing (and also, hopefully, my interpretation of all this is correct). If you are still lost then just follow these rules:

1. Do not set images as flipped unless you know what you are doing.
2. Use the -drawAdjusted... methods in my category (or similar technique) to do all your image drawing.
3. If you didn’t listen to rule #1 and you have a flipped image and it is showing up upside-down even when following rule #2, then use the -unflippedImage method in my category to get an unflipped version of the image and use that instead.
4. Never go in against a Sicilian when death is on the line.

That should handle most cases you run into. And trust me on rule #4.

In this installment of The Invisible Interface, we are going to look at stealing preferences. What is stealing preferences? Simply enough, it’s using the preferences of some other app instead of having your own for a particular feature. The point of this is to avoid having to provide a separate interface for settings when the user has already made their choices known somewhere else.

The key here is finding cases where your functionality is more centrally used somewhere else. I’m going to use Hazel as an example but hopefully these will illustrate the point well enough for you to look for where you can apply it yourself.

Spring-Loaded Folders

For those that don’t know, Finder has a feature called spring-loaded folders. What this does is when you drag a file over a folder, after a delay, it will flash and then open that folder so you can drill deeper. It allows you to navigate the folder tree without having to let go of the file you are dragging.

Hazel implements spring-loaded folders as well. It works when you want to move/copy rules in between folders. Hovering the dragged rule over a folder in the list on the left will cause the view to switch to the rules for the hovered-over folder allowing you to drag back into the rule list and place the rule where you want it. [On a side note, implementing this uncovered a bug in NSTableView (at least on Tiger; have not checked with Leopard) resulting in me doing my own implementation of NSTableView’s drag and drop.]

Under Finder’s “General” tab, you’ll see a checkbox and slider to configure these settings. Your first thought may be to provide a similar UI in your app. But why? Does the user really care about tweaking this for each app it appears in? It seems like whatever setting works in Finder will be fine wherever else it is used so why not just use Finder’s preference?

Commonly Used Folders

In Hazel, when you specify a destination folder for some actions (like move and copy), there is a pop-up of folders. You’ll notice that there’s a list of common folders at the end of the pop-up menu. If you look a bit closer, you may notice that these are the same folders in the sidebar of your Finder windows. Those folders are common destinations for files so it’s a good list for Hazel to use. By grabbing that list from Finder, Hazel avoids any sort of extra maintenance/interface for managing that list.

AppleScript Editor

In 2.2, Hazel introduced inline editing of scripts. You are provided with a mini-AppleScript editor right in Hazel’s rule interface. Now, there are potentially different things you can tweak to make the editor suit your needs, such as line wrapping, tab widths and whether to use the script assistant. But if you poke around Hazel’s UI, you’ll see that there’s no interface to set these. That’s because Hazel steals these preferences from Script Editor. If someone is serious enough about editing AppleScript that they care about these settings, there’s a good chance they have Script Editor installed and already set these preferences. By using its preferences, there is a consistency of user experience between the two editors.

Of course, you can’t do this everywhere. It’s best suited when the functionality is primarily used elsewhere and you are echoing it in your own app. The apps Apple ships with the system are an easy mark since you can usually rely on them being installed and they tend to be the places where common functionality is defined. Overall, the result is a less tweaky and cluttered interface and a more seamless experience with the rest of the system.

Doing the Heist

You can grab other apps’ preferences using either CoreFoundation or Cocoa. With Cocoa, NSUserDefaults is your go-to guy. -persistentDomainForName: does what you want. Give it a bundle ID and in return, you get a dictionary of preferences. What would’ve made more sense is something like +userDefaultsForName: which would return an NSUserDefaults instance, but hey, it’s not like Apple is hiring me to do API design. With CoreFoundation, you can use CFPreferencesCopyAppValue() to pick individual preferences. Again, a bundle ID is needed.

And I can’t leave without placating the more pedantic among you that have to point out the potential dangers of doing this. Therefore, I must note that there is some risk in doing this as most apps do not document their preference settings and they can change at any time. Having default values of your own for these settings should minimize the risk, at least buying you time until you can re-work things to use the new schema. That said, if this is for some critical/primary functionality in your app, it might behoove you to have your own settings for it. As they say, invest only what you can afford to lose, or, to milk the stealing metaphor, don’t do the crime if you can’t do the time. And while we’re at it, just say “no” to drugs, kids.

Video games are on the forefront of what kinds of rich interactions people can have with computers. In the past decade, there’s been a push for more and more immersive virtual environments resulting in more advanced APIs and hardware to provide things such as super-fast 3D rendering. In recent years, OS X has leveraged these advances in the predominantly 2D world of user interfaces, often in brilliant ways as seen with QuartzGL, CoreAnimation and CoreImage.

In video games, it’s quite common to exploit stereo output or even better, surround sound, to provide positional audio cues. Just as graphics can simulate a 3D space, so can sounds be placed positionally in the same space. If you, super-genetically-modified-mutant-soldier, are running around on the virtual battlefield and there is some big-bad-alien-Nazi-demon-zombie dude shooting at you from the side, you will hear it coming from that direction and react accordingly. Directional audio cues can supplement visual cues or even supplant them if visual ones cannot be shown (i.e. something requiring attention outside your field of view).

On OS X, sound is used rather sparingly in the interface, which is probably a good thing. But for those cases where it’s use is warranted, why not take advantage of technology available? Just as animation can be used to guide the user’s focus, why not sound? OS X does ship with OpenAL, which is to sound what OpenGL is to graphics, providing a way to render sounds in a 3D space.

I’ve put together a quick proof of concept app (download link near the end of the article). Move the window around the screen and click the button to make a sound. Based on the window’s position, the sound will appear to come from the different sides, which, for the most part is left/right, most sound output systems not being designed to articulate things in the up/down direction. The program itself basically maps the window position to a point in the 3D sound space. Right now, it doesn’t really use the z-axis (the axis that goes into your screen) but conceivably you can do things like make the sound appear further away based on window ordering. Try using headphones if the effect is not as apparent using speakers.

There is a significant technical issue, though. You can’t really know the actual physical dimensions and layout of a user’s screens. In addition, the position of the speakers relative to the screens is also not known. While you can get screen resolutions and relative positions of the screens, these are mostly hints at the actual layout. In my demo program, it is assumed that the screens are relatively close to each other forming one gigantic screen. It is also assumed that the speakers produce a soundstage roughly centered on the primary display (the one with the menubar). It assumes a model like this (the circle is the user and the thin slabs are the monitors, from a top-down view):

In reality, it’s probably more likely the user would have a setup like the following:

But who knows, it could possibly be something like this:

The point here is that the effectiveness of this is dependent on the user’s setup. A particular idealized model would have to be chosen that hopefully works well enough for most people. While pinpoint accuracy is not really feasible, it probably isn’t required either. Human hearing is imprecise, otherwise ventriloquists would never be able to pick up a paycheck. Just an indication of left, right or center is probably enough for these purposes.

Where would this be useful? Well, this all came up yesterday when I received an IM (via Adium). I had my IM windows split up across two screens so I had to scan around a bit to find out which window had the new message. Though the window was on the screen to the left, the audio alert made me look at the main screen since the sound was centered straight ahead. It would be great to see an idea like this implemented in Adium and I’ve filed a feature request with them for their consideration. It’s ticket #11292 so you don’t go and submit a duplicate request.

It would be interesting to see more use of this in user interfaces out there. I don’t want to encourage people to add sounds to their apps if they weren’t already using them but for those that are, it’s something to consider. Overall, the effect is quite subtle but with some tweaking, it can be quite effective.

The link to download the demo program is below. Sorry, no source is provided this time. The code is a hacked together mess of stuff copied and pasted from an Apple example as I have never used OpenAL before. This can probably also be implemented in CoreAudio by adjusting the balance between the channels. If you are considering implementing something like this, email me and I’d be happy to discuss details as long as they don’t involve audio APIs since, well, I don’t know them particularly well.

Download PositionalAudioAlertTest.zip (Leopard only)

Thanks to Mike Ashe and Chris Liscio for advice on CoreAudio, which I ended up not needing as Daniel Jalkut suggested I use OpenAL instead which made things easier.