Noodlings

I was just turned on to a post from Jesper of Waffle Software entitled How to Mimic the Artwork Column in Cocoa. There he approaches the problem of implementing cells in an NSTableView that can span multiple rows. This spurred on Jacob Xiao at Like Thought to tackle the problem in a different manner. I suggest reading both of the above to get a sense of the problem and the possible approaches.

With NSTableView still on my mind from my last diversion, this got me thinking about the problem and I decided to try my own hand at it. I preferred Jacob’s approach as I think it should be all contained within NSTableView if at all possible. I liked the notion of “slicing” up the cell instead of using another view. Playing with it, I came up with a generalized solution. Here are some of the interesting aspects:

• It can support any NSCell subclass. I do this by using a special wrapper cell. It draws the “full” version of the cell then draws out the slices for each row into the tableview.
• It determines the spans by finding contiguous rows that have the same object value for the column in question.
• The above two points are exploited to do some caching such that the “full” cell is only drawn once and the parts composited in place as needed.
• If the first column is set to span rows, it will only draw the grid for the last row in a span.
• You can designate multiple columns to do this. In the example project, if you click on the “Album” column, it will toggle between span and non-span mode. The “Artwork” column doesn’t do this because it looks cruddy in non-span mode. I haven’t tested it (see update below) but theoretically, the spans do not have to line up between different columns.

To use it, for any columns you want to have this row-spanning behavior, set the table column class to NoodleRowSpanningTableColumn. Also, it might help to get rid of any intercell spacing in your tableview (at least in the vertical direction). There are probably a few bugs. I do know that reordering columns may cause some grid drawing funniness but for the most part, it seems to work pretty well. Let me know if you find otherwise.

You can check the code for other implementation tidbits. I’ve included a self-contained project below but I’ll eventually polish it a bit and add it to my NoodleKit repository where it can join the rest of my goodies.

Many thanks for Jesper and Jacob for the inspiration and laying down the groundwork for this.

Update (Oct. 21, 2009): I observed odd behavior in that the program would get into a re-drawing frenzy eating up a bit of CPU. This was non-blocking in that I was still able to manipulate the UI (which is why I didn’t notice it at first). I finally figured out that it was the -drawGridInClipRect implementation. I was drawing the grid selectively by changing the grid mask and then calling super on subregions. Unfortunately, changing the grid mask queued up a redraw which started a loop (albeit one that didn’t tie up the event loop). Seeing as I needed to fix the grid-drawing logic anyways, I reworked the whole thing.

The new grid code should now be able to handle multiple spanning columns rearranged in whatever way you want. Here’s contrived example of having multiple spanning columns that don’t necessarily line up:

I also made some other fixes and improvements so it’s worth re-downloading the project even if you already did so before (I bumped this version to 0.37).

As for integration with the rest of NoodleKit, I still need to decide whether I want to merge this with my sticky row tableview to make some über-tableview. Problem is is that I can’t think of how the two features would interact so maybe it’s best to keep them separate. Thoughts on this are welcome.

Update (Oct. 23, 2009): I’ve just updated the project to version 0.68. There was a crasher when you would click on a non-span row and then move into a spanning cell. Also, the span cells were highlighting when you clicked on a non-span row. Both of these have been fixed. Thanks for Jesper for finding the crasher and Jacob for helping with the fix.

Update (Oct. 25, 2009): I’ve consolidated this functionality along with my previous sticky row header code into a single NoodleTableView class. You can find it in the NoodleKit repository. Future updates to this will be posted there from here on out.

Download NoodleRowSpanningTableViewTest.zip
[This project is no longer being updated here. You can get the latest in the NoodleKit repo]

I’ve finally gotten around to consolidating most of the bits of code I’ve posted on this blog over the years and put it in a repository. While the real NoodleKit is a much more extensive and cohesive toolkit that I use internally, this one will serve as a place where I put the odd scraps that I decide to open source. I guess I could’ve called it NoodleScraps but it doesn’t quite sound as nice.

If you’ve been following my blog for a while then there’s not much new here, codewise. There are a tweaks here and there plus the stuff should be 64-bit ready so it might be worth re-downloading just for that. It will be the place where I put future code so you should keep an eye on it.

The project is structured to build a framework but also has an examples directory (with corresponding targets) to demonstrate the use of the different classes. The current notion is to keep things compatible with 10.5. And, as usual, all code is released under the MIT license.

NoodleKit (hosted at github)

Enjoy.

Once again, I find myself with some cool code with nowhere to put it. And, yet once again, you get the reap the benefits.

UITableView on the iPhone has the neat feature where section headers stick at the top after the actual row for it has scrolled off. For instance, in Contacts, if you scroll down, the row for section “A” stays visible so that you know what section you’re in if the section is too large for the screen.

Unfortunately, someone asked how something like this could be done on desktop Cocoa. I say “unfortunately” because I was within earshot and it resulted in me embarking on a journey peppered with annoying run-ins with NSTableView’s idiosyncrasies. Nonetheless, I persevered and came up with something hopefully someone will find useful. As was the case before, I don’t have a use for it myself at the moment, but I feel it would be a shame to let the code go to waste.

I am presenting NoodleStickyRowTableView. I call the section headers “sticky rows” as one can specify any row to stick up top for whatever reason, though by default, it does it for group rows.

The bad stuff:

NSTableView makes it very difficult to cache a row visually. -drawRow:clipRect: should be able to do it but the problem is that it’s too smart for it’s own good. If the row is not visible, it will not draw anything. Add on top of that the impossibility of getting it to draw the special background the group rows use. I tried all sorts of techniques but none were general or reliable enough. My email on the cocoa-dev list was, not surprisingly, ignored. As a result, in the default implementation, I use my own look for the rows and then use a fade-in transition since the sticky row will look different from the actual one. You can see it in the video below where it has a ghost-like effect.

The good stuff:

The good news is that I managed to work it into an NSTableView category. You can think of it as an abstract category if you wish. In your subclass, at the very minimum, override -drawRect: and call -drawStickyRowHeader after calling super. From that one call you will get all the functionality.

Why do it this way? Well, ease of use for one. It makes it so that you can integrate it into your own NSTableView subclass without having to copy and paste large swaths of code or put it all sorts of odd hooks. Secondly, the changes get inherited by NSOutlineView so the functionality is available there and in its subclasses as well.

While I could use the new associated objects functionality in Snow Leopard, I opted for Leopard-compatiblity. This resulted in me coming up with a couple tricks to get around the lack of ivars, including the rarely used -viewWithTag: method. Check the “read me” file and code for details.

There are two different transitions. The default is the fade-in transition as shown here:

As mentioned before, since I couldn’t get the group rows to draw properly and consistently, I use a more generic look for the row. The transition makes it less jarring but it is a compromise of sorts. Not bad when you consider that you get all this by calling just one extra method in your -drawRect:.

But what if you want something more like the iPhone’s version? In your subclass, you can override -stickyRowHeaderTransition to return no transition and then do your own drawing of the rows so that they look the same between the regular and sticky versions. The NoodleIPhoneTableView included in the project shows how this is done with the result being something on par with UITableView:

I am releasing this under MIT license as usual. Use it however you want as long as you give me credit somewhere. Please send in any feedback, suggestions, bugs, etc but keep in mind that this is not something I use in my projects so I probably will not be actively maintaining it beyond a certain point. I’ve made a point of making the the API easy to use and to extend so have fun with it.

Download NoodleStickyRowTableViewTest.zip (version 0.18639)

Update (Sep. 29, 2009): I have included this class in my NoodleKit repository so you should check there for future updates.

In this installment of The Invisible Interface, we are going to look at stealing preferences. What is stealing preferences? Simply enough, it’s using the preferences of some other app instead of having your own for a particular feature. The point of this is to avoid having to provide a separate interface for settings when the user has already made their choices known somewhere else.

The key here is finding cases where your functionality is more centrally used somewhere else. I’m going to use Hazel as an example but hopefully these will illustrate the point well enough for you to look for where you can apply it yourself.

Spring-Loaded Folders

For those that don’t know, Finder has a feature called spring-loaded folders. What this does is when you drag a file over a folder, after a delay, it will flash and then open that folder so you can drill deeper. It allows you to navigate the folder tree without having to let go of the file you are dragging.

Hazel implements spring-loaded folders as well. It works when you want to move/copy rules in between folders. Hovering the dragged rule over a folder in the list on the left will cause the view to switch to the rules for the hovered-over folder allowing you to drag back into the rule list and place the rule where you want it. [On a side note, implementing this uncovered a bug in NSTableView (at least on Tiger; have not checked with Leopard) resulting in me doing my own implementation of NSTableView's drag and drop.]

Under Finder’s “General” tab, you’ll see a checkbox and slider to configure these settings. Your first thought may be to provide a similar UI in your app. But why? Does the user really care about tweaking this for each app it appears in? It seems like whatever setting works in Finder will be fine wherever else it is used so why not just use Finder’s preference?

Commonly Used Folders

In Hazel, when you specify a destination folder for some actions (like move and copy), there is a pop-up of folders. You’ll notice that there’s a list of common folders at the end of the pop-up menu. If you look a bit closer, you may notice that these are the same folders in the sidebar of your Finder windows. Those folders are common destinations for files so it’s a good list for Hazel to use. By grabbing that list from Finder, Hazel avoids any sort of extra maintenance/interface for managing that list.

AppleScript Editor

In 2.2, Hazel introduced inline editing of scripts. You are provided with a mini-AppleScript editor right in Hazel’s rule interface. Now, there are potentially different things you can tweak to make the editor suit your needs, such as line wrapping, tab widths and whether to use the script assistant. But if you poke around Hazel’s UI, you’ll see that there’s no interface to set these. That’s because Hazel steals these preferences from Script Editor. If someone is serious enough about editing AppleScript that they care about these settings, there’s a good chance they have Script Editor installed and already set these preferences. By using its preferences, there is a consistency of user experience between the two editors.

Of course, you can’t do this everywhere. It’s best suited when the functionality is primarily used elsewhere and you are echoing it in your own app. The apps Apple ships with the system are an easy mark since you can usually rely on them being installed and they tend to be the places where common functionality is defined. Overall, the result is a less tweaky and cluttered interface and a more seamless experience with the rest of the system.

Doing the Heist

You can grab other apps’ preferences using either CoreFoundation or Cocoa. With Cocoa, NSUserDefaults is your go-to guy. -persistentDomainForName: does what you want. Give it a bundle ID and in return, you get a dictionary of preferences. What would’ve made more sense is something like +userDefaultsForName: which would return an NSUserDefaults instance, but hey, it’s not like Apple is hiring me to do API design. With CoreFoundation, you can use CFPreferencesCopyAppValue() to pick individual preferences. Again, a bundle ID is needed.

And I can’t leave without placating the more pedantic among you that have to point out the potential dangers of doing this. Therefore, I must note that there is some risk in doing this as most apps do not document their preference settings and they can change at any time. Having default values of your own for these settings should minimize the risk, at least buying you time until you can re-work things to use the new schema. That said, if this is for some critical/primary functionality in your app, it might behoove you to have your own settings for it. As they say, invest only what you can afford to lose, or, to milk the stealing metaphor, don’t do the crime if you can’t do the time. And while we’re at it, just say “no” to drugs, kids.

Video games are on the forefront of what kinds of rich interactions people can have with computers. In the past decade, there’s been a push for more and more immersive virtual environments resulting in more advanced APIs and hardware to provide things such as super-fast 3D rendering. In recent years, OS X has leveraged these advances in the predominantly 2D world of user interfaces, often in brilliant ways as seen with QuartzGL, CoreAnimation and CoreImage.

In video games, it’s quite common to exploit stereo output or even better, surround sound, to provide positional audio cues. Just as graphics can simulate a 3D space, so can sounds be placed positionally in the same space. If you, super-genetically-modified-mutant-soldier, are running around on the virtual battlefield and there is some big-bad-alien-Nazi-demon-zombie dude shooting at you from the side, you will hear it coming from that direction and react accordingly. Directional audio cues can supplement visual cues or even supplant them if visual ones cannot be shown (i.e. something requiring attention outside your field of view).

On OS X, sound is used rather sparingly in the interface, which is probably a good thing. But for those cases where it’s use is warranted, why not take advantage of technology available? Just as animation can be used to guide the user’s focus, why not sound? OS X does ship with OpenAL, which is to sound what OpenGL is to graphics, providing a way to render sounds in a 3D space.

I’ve put together a quick proof of concept app (download link near the end of the article). Move the window around the screen and click the button to make a sound. Based on the window’s position, the sound will appear to come from the different sides, which, for the most part is left/right, most sound output systems not being designed to articulate things in the up/down direction. The program itself basically maps the window position to a point in the 3D sound space. Right now, it doesn’t really use the z-axis (the axis that goes into your screen) but conceivably you can do things like make the sound appear further away based on window ordering. Try using headphones if the effect is not as apparent using speakers.

There is a significant technical issue, though. You can’t really know the actual physical dimensions and layout of a user’s screens. In addition, the position of the speakers relative to the screens is also not known. While you can get screen resolutions and relative positions of the screens, these are mostly hints at the actual layout. In my demo program, it is assumed that the screens are relatively close to each other forming one gigantic screen. It is also assumed that the speakers produce a soundstage roughly centered on the primary display (the one with the menubar). It assumes a model like this (the circle is the user and the thin slabs are the monitors, from a top-down view):

In reality, it’s probably more likely the user would have a setup like the following:

But who knows, it could possibly be something like this:

The point here is that the effectiveness of this is dependent on the user’s setup. A particular idealized model would have to be chosen that hopefully works well enough for most people. While pinpoint accuracy is not really feasible, it probably isn’t required either. Human hearing is imprecise, otherwise ventriloquists would never be able to pick up a paycheck. Just an indication of left, right or center is probably enough for these purposes.

Where would this be useful? Well, this all came up yesterday when I received an IM (via Adium). I had my IM windows split up across two screens so I had to scan around a bit to find out which window had the new message. Though the window was on the screen to the left, the audio alert made me look at the main screen since the sound was centered straight ahead. It would be great to see an idea like this implemented in Adium and I’ve filed a feature request with them for their consideration. It’s ticket #11292 so you don’t go and submit a duplicate request.

It would be interesting to see more use of this in user interfaces out there. I don’t want to encourage people to add sounds to their apps if they weren’t already using them but for those that are, it’s something to consider. Overall, the effect is quite subtle but with some tweaking, it can be quite effective.

The link to download the demo program is below. Sorry, no source is provided this time. The code is a hacked together mess of stuff copied and pasted from an Apple example as I have never used OpenAL before. This can probably also be implemented in CoreAudio by adjusting the balance between the channels. If you are considering implementing something like this, email me and I’d be happy to discuss details as long as they don’t involve audio APIs since, well, I don’t know them particularly well.

Download PositionalAudioAlertTest.zip (Leopard only)

Thanks to Mike Ashe and Chris Liscio for advice on CoreAudio, which I ended up not needing as Daniel Jalkut suggested I use OpenAL instead which made things easier.

This is something that I’ve thought about for some time so I thought I’d write a series on the topic of invisible interfaces. What is the invisible interface? When people think of a user interface, they think of something visual made up of windows and widgets. Even for a commandline program, it’s the arguments, the output and error messages. But what many people aren’t aware of are the choices the designer made and the logic the programmer codes that make decisions for you. An interface not only encompasses what the developer put into it, but also what the programmer specifically kept out. This benefits the user in a number of ways: a less cluttered interface, a simpler interaction paradigm and fewer steps to accomplish a task. Many of these things are too subtle to be noticed normally which is the beauty of it. Sometimes the best interface is the one you never know is there.

Let’s take for example the flush toilet. Yes, sorry if this example is a bit disgusting but I’ll try and keep it clean and it is a fitting example. Just bear with me here. So, where were we? Ah yes, the toilet. Simple interface. Push down on the lever, water is flushed down and it stops and refills the tank ready for the next flush. It doesn’t get much simpler than that (well, it can but more on that below). Notice how you don’t have to stop the flush. If the toilet is calibrated properly, it should have enough water to flush down whatever you may put in there.

Of course, from a performance/efficiency standpoint, it’s not optimal. You are flushing the same amount of water each time, whether there is liquid or solid matter to be disposed of. How does one work this new requirement (the need to save water) into the interface? In Europe (I have yet to see them stateside), there are toilets with a split button. Hit one side and a lesser amount of water is flushed whereas hitting the other side flushes down a full measure. There are usually markings to indicate 1 or 2 (one or two dots is what I’ve seen) so you can figure out which one is which. Now, the interface has become more complicated. Yes, in the grand scheme of things, it’s not rocket science, but humor me. Now a decision has been added. Do I hit the 1 or 2 button? The user is now required to give the device more information than they had to before. The question is, is complicating the interface worth the functional gain and also, is there a way to effect the same result without changing the interface at all?

How about auto-detecting the amount of water needed? Not only does this optimize the efficiency of the device, it also takes away a decision. Now, of course, whether this can be practically done is in question. It is unclear whether the technology to do this reliably exists and there are also issues of manufacturing, cost and maintenance that play into it. But the point is that from a pure interface standpoint, it would seem to be a better solution. It meets the new requirement while retaining the one button simplicity from before.

And to take it even further, it could sense when a flush is needed, alleviating the need of the button altogether. While these types of toilets are becoming more common in public restrooms, I haven’t heard of any demand for these in the home. Here, it’s possible that automatically doing something on the user’s behalf becomes unwelcome. I can imagine in your smaller bathroom at home, you are likely to trigger it accidentally by walking by it which can be startling. In a public setting, you probably don’t care if toilets are firing off left and right like the cannons in the 1812 Overture. On one hand, it could be just an issue of implementation; maybe the technology just isn’t accurate enough. On the other hand, it’s very possible that this is a feature (when to flush) that the user wants control over. Either way, it’s an issue that the designer must grapple with.

The point of all this is that there is some room for improvement in terms of simplifying interfaces when one strives to have their program/device do more for the user. The more your program does, the less the user has to. But one can also overstep their bounds to create something that may be seen as intrusive. It’s about defining the balance between what the user does and what the machine does, with an eye towards putting more on the machine’s side.

As I mentioned in the beginning, I’m intending this to be the start of a series. Don’t expect some well-thought out arc with this; it will probably just be an occasional article here and there. While part of me wouldn’t mind writing more about toilets (I haven’t even touched upon those wacky Japanese toilets¹), in the next installments, I’ll try and come up with examples more relevant to computer/human interaction.