Category: Programming


The Invisible Interface: APIs

May 25th, 2017 — 11:21am

Let’s talk about APIs. Like UIs, APIs provide an interface (hence the “I”) to a class of users. In this case, the users are programmers. Because of this, many times you have APIs that are cluttered and overly difficult to use since programmers can probably figure that stuff out. It’s unfortunate as many just see APIs as a way to expose functionality without any regard for design.

I feel a hallmark of a good UI is one that avoids presenting the user with unnecessary decisions. The more you can either filter or figure out on the user’s behalf, the better. I’ve touched upon this before: 1 2

I feel the same holds true for APIs. APIs should do more than just provide functionality. They should strive to do more for the user without burdening them with unnecessary details. Provide power with little effort on the user’s part. Let the programmer worry about the “what” and not about the “how”.

This leads me to my favorite API: NSArray

Yes, NSArray.

What is an NSArray? It’s an ordered collection of objects. Doesn’t sound terribly exciting but ask yourself this: what data structure does it implement? If you ask around, you’ll get a bunch of different answers but the real answer here is “It doesn’t matter.”

Now, you may be thinking that NSArray is insufficient to cover all the different cases where you’d need an ordered list. Surely, you have to use a skip list in this one case or a circular queue in this other one. Before going further, I highly recommend reading this oldie but goodie (and one of my favorite blog posts): Array.

As it turns out, NSArray is adaptive. It’s not implementing a single data structure but instead changes depending on usage. Every time I, or a colleague, has implemented their own special data structure, it has ended up performing no better, or sometimes even worse, than NSArray. Now, I’m sure you can come up with some special set of circumstances where you’d need to roll your own, but from my experiences with Cocoa over the years, I’d say that for 99% of the cases, NSArray is fine.

Another example of adaptive behavior is Introsort, which will use either quicksort or heapsort depending on the circumstances. Quicksort has great average performance but has a horrible worst case (O(n2)). Introsort is able to detect when things are headed towards that worst case and switches to heap sort instead (worst case: O(n log n)). No need for the user to evaluate the data to figure out which sort algorithm to use.

Now, sometimes things are inherently complex (threading comes to mind). It does take a bit of care and experience to make sure that instead of making something simple, you end up making it simplistic. Nonetheless, I feel that NSArray hits that sweet spot of providing what you need while at the same time cutting out what you really don’t.

Hopefully this shows that APIs can be so much more than a simple wrapper around some code. The use of proper abstraction as well as going the extra mile on the user’s behalf can go a long way towards making a merely functional API something wonderful.

 

Comment » | Cocoa, OS X, Programming

Getting Info From iTunes

May 19th, 2017 — 12:35pm

Things have settled down with my 4.1 release so I thought I’d tend to my neglected blog.

Today, I’m going to talk about getting info about your iTunes library, playlists and albums.

Now, the obvious way to do this would be through AppleScript/AppleEvents. The problem with this approach is that it requires iTunes to be running for it to work. To actually change stuff in your iTunes library, you will have to use AppleScript but for the purposes of this article, we are just looking at ways to get information about it.

The old way to do this was to parse the iTunes Library.xml file (located in ~/Music/iTunes). This file contains a dump of everything you need. Unfortunately, recent versions of iTunes do not generate this automatically; you have to enable it in its preferences (it’s in the “Advanced” tab).

Now, the notion of having to parse an XML file does sound painful but luckily it’s actually a property list so you can load it that way and access everything as NSArrays, NSDictionaries, NSStrings, etc.

With OS X 10.9 (yes, I know it’s macOS now but back then it was still OS X), Apple introduced the MediaLibrary API (ML). This provided a way of not just accessing the iTunes library, but the libraries of all of Apple’s other media apps (Photos, GarageBand, even Final Cut). No parsing involved but it is a fully asynchronous API so expect to deal with a lot of callbacks.

The last one (I don’t know exactly when it was released but it was the last one I discovered) is iTunesLibrary (ITLib) and it’s a bit easy to miss because, from what I can tell, it is not a part of the OS, but iTunes itself. You can find the framework under /Library/Frameworks (not /System/Library/Frameworks). It provides similar functionality to the MediaLibrary framework, but specific to iTunes.

With so many options, which do you use? For the most part, they return the same information though it depends on the version of iTunes. With some versions, one API may return some special playlist that the others don’t. Also, ML is asynchronous while the others are synchronous. I don’t see that as a big deal either way as you can convert one into the other with a tiny bit of work.

The big differentiating factor is performance. Here are the results for getting a list of playlists for my library:

XML: 1.664939s
ML: 18.792855s
ITLib: 0.844415

I don’t think my library is the biggest in the world but it is significant (about 2000 albums). As you can see, ML is pretty damn slow with ITLib being much, much, much faster. What is surprising to me is how much faster parsing the XML file is than ML. It is possible, though, that I’m holding it wrong.

So, ITLib is my recommended API for this. It’s what Hazel uses starting with 4.1 (previously, it was parsing the XML when it was available and falling back to ML). That said, there is one wrinkle.

The persistent IDs of entities in iTunes are hex strings. In the XML file, they are zero padded. In ML, though it returns strings, they are not zero padded (rdar://26624642 for you Apple folks watching from home). To add even more confusion to the mix, they are NSNumbers in ITLib.

If you are using any of these APIs in isolation then it’s probably not a big deal, but if you want to later manipulate that entity using AppleScript (like importing files into a playlist), you will need to provide the zero-padded hex string. Something to keep in mind.

Comment » | Cocoa, Hazel, OS X, Programming, Software

On Dynamism

May 23rd, 2016 — 7:15pm

I’ve kept my hat out of the discussions of Swift vs. Objective-C for several reasons, but the main one is that frankly, I’ve been busy. I ship a product, one that I would consider successful, and recently shipped a major release. What does that mean? It means that debating these types of issues and playing with the latest thing is a luxury. Because I have a shipping product with a lot of users, I can’t just change things unless I find that it adds value to my users. My job is to solve my users’ problems. Now that I have a little bit of time to do so, this is one lens I’m hoping to add to this discussion

In addition, years ago, when Java was introduced, I jumped on that bandwagon, leaving behind Objective-C. It started out writing replacements for Foundation and AppKit in Java and ended with writing large scale servers. I was on that wagon for a good 10 years before returning, so I wasn’t a tourist there. This is yet another lens I also want to add to the discussion as history seems to be repeating itself and many younger people seem to be missing out on this part.

But let’s start with the current issue about dynamism since that’s the hot topic.

One thing many people seem to overlook about the dynamism of Objective-C is that it enabled NeXT (and Apple) to provide better GUI tools. Using dynamism, they were able to make GUI building declarative in nature. Connect this to that. Call this method. All stored in a file that was (and still is) data, not code. Competitors at the time (and today) resorted to code generation which is fragile and, ironically, unsafe. Yes, you could have a more declarative file format, but implementing that in using a static language required a lot of hard-coding and switch statements. Not the elegance that many people claim to be moving towards.

I’m not saying that a language has to be purely dynamic but it shouldn’t be purely static either. It think it’s spurious not to credit a level of dynamism for the quality of apps on Apple platforms over the years, and to be pedantic, the NeXT ones as well – many of which were considered the best on any platform at the time. To deny that, I feel, shows a lack of understanding of what has made the platform great all these years.

Moving from a dynamic language to a more static one seems refreshing at first. I went through the exact same thing when switching to Java. Yes, I can hear some of you grumbling “Java” out there but the irony is that there are more similarities in the evolution between Java and Swift than you may want to admit.

Everything seemed so much “tighter” in Java. Interfaces (Java’s version of protocols) everywhere! No more pointers! Checked exceptions! It’s all so new and exciting. You might just have to take my word on this, but any new language that you really like will always seem to make you “more productive”. A colleague and I (plus maybe one, or one and a half, devs who were on the periphery) implemented our own version of Foundation and AppKit in what I would consider a pretty short amount of time. Other people at Lighthouse can correct me on this but my recollection was that it was on the scale of months, not years. You can read up more on it here.

Was it the language? At the time we might say yes but in hindsight, I’d say it was just that we were just super excited and into the project. I was working with an exceptional team and being a new platform, we were excited about making our mark on it. Yeah, there were bits about Java that were better than Objective-C. But there were bits that were worse. The point here, though, is that until the honeymoon is over and you are maintaining a project over several years, it’s hard to make real proclamations about productivity. Truth be told, I’d say the biggest impact on software quality and productivity is the quality of engineers regardless of any other factors.

Looking back on my years with Java, I find the biggest problems with the platform was this idea that everything can be done by the compiler. It sounds great on paper but when implemented, it resulted in all sorts of contortions to work around not letting any dynamism in. It resulted in a lot of glue being written that I wouldn’t have to write in a more dynamic platform. It resulted in cascading changes across code to make trivial modifications just to appease the compiler. Maybe in a future post I’ll go into specific language features that I found problematic and show parallel trends/features in Swift but let’s just say that less is more. It’s an aesthetic that I always felt that put Apple above the rest and something which I feel like in some ways is being abandoned.

Now, I’m sure many of you will just paint me as an old, stubborn Obj-C luddite. And it’s true on some level but that doesn’t mean you should discount my experience. Many of us old fogeys are working on products that have shipped for years. That means that (1) we are solving user problems (2) have immense experience with doing app development (3) really understanding the frameworks (4) have been maintaining a code base.

Let’s go through these points:

1. We are solving user problems.

We have real problems to solve for people outside of ourselves. We need to be practical to survive. Users don’t care about the tech underneath. We have to ship them solutions. This oftentimes requires rapid response. This also oftentimes involves working around Apple bugs, bugs that never get fixed and will only grow in number as Apple maintains their yearly release schedule (something to think about for proponents of the “final” keyword). Agility and flexibility are paramount here. Reality is not pretty. The user could care less how beautiful your code is if you aren’t solving their problems.

2. We have immense experience with doing app development

I want to emphasize the “app” part of that. A common argument for very strict type safety are for cases which aren’t apps. The predominant software in the Apple ecosystem is end user applications. Let’s try and not lose focus here and argue the case for categories of software that aren’t Apple’s forte. If you are writing some life-critical software on an Apple platform, god help you. Even with Swift, you are stuck with Xcode and Apple’s insistence on yearly new OS releases.

3. We really understand the frameworks

I would take many of the arguments for Swift more seriously if there weren’t strawman examples set up on the Obj-C side. You’ll have to dig up Brent Simmons’ posts on this, but when I see the kooky examples of horrible old Obj-C way, what it tells me is more that you don’t understand the frameworks. It resembles informercials where people are fumbling opening a jar and then are presented with some useless gadget to solve a non-existent problem. I’m sure things can be done better than they are now but providing bad examples indicating ignorance of what it is you are trying to replace doesn’t help your argument. And I have seen plenty of examples of new people replacing what they don’t understand. Please don’t be one of those people.

4. We have been maintaining a code base for a while

I have been writing Hazel for ten years. Many of us old-time app developers are in similar positions. Speaking with many other devs, most of whom write iOS apps, I’d be lucky to find those that have maintained a code base for a third of that time. Sure, there are parts of the code that are ugly, but that happens on any platform. Same thing happened on Java. No matter what you do, time is always against you. When I was working with Java, I found the technical debt grew in a different fashion. It was hard to change anything. This was because changes in stricter languages tended to cascade. Because everything has to declare everything about itself, changing code turns into a Facebook feed, except where you and everyone else are obligated to respond to every stupid pic and forward of Snopes-bait. With a more dynamic code base, I can say I have less code to maintain and that I can be more agile. Not that it’s without it’s own problems. But let’s not act like type-safety is a panacea.

And to give more of my personal perspective on the language debate, I’d love to have something better than Objective-C. But I’d also like to have something better than Swift. My personal pet peeve is that we are in the 21st century and we still have to give a shit about the storage type of numbers. I’d be happy to get rid of scalar primitives entirely.

Ultimately, though, I need something that solves my problems. My problems aren’t performance. They aren’t type-safety (maybe it’s just me, but I rarely have issues with it and when I do, the time it takes to fix it is far less than the time specifying everything to a tee everywhere else). They aren’t being able to write clever VDLs. For me, it’s writing apps that solve my users’ problems and getting them out in a timely fashion. As it stands now, Swift (at least pure-Swift, or even current Swift as a non-ABI-stable moving target) does not do that for me.

Now maybe these same problems can be solved in a static way but what I’m not seeing from the static-camp are (decent) solutions. What I’m seeing are either hand-waving or the same crufty code-generation, write tons of repetitive boilerplate type of solutions that I had hoped we had left behind in the 90s.

What I do see makes me worry that it’s not the experienced app-writers that are being heard. Unfortunately, many of us are too busy to sit in the various mailing lists and forums. Yes, ok, we lose. But we all lose if those creating the platform don’t draw from the experience of those who have built upon it successfully. As someone who has done both (write frameworks and apps) and seen what happens when the former ignore the latter (the rise and fall of Java), I’m hoping history doesn’t repeat itself.

That’s my somewhat meandering brain dump for the moment as I do have to get back to working on my next patch. Sure, these are just my opinions but my hope is that we can at least look for more balance in our approaches to solving actual problems.

36 comments » | Cocoa, OS X, Programming

NSSecureCoding

November 9th, 2015 — 1:30pm

I’ve had it on my list to convert all my NSCoding classes to use NSSecureCoding and this recent post reminded me so I figured now is as good a time as ever.

How do you go about using secure coding? Here are the Apple docs. Unfortunately, those docs don’t give the whole picture. There are a few details you need to be aware of.

When you subclass an object that supports NSSecureCoding, if you override -initWithCoder, you must also implement +supportsSecureCoding

Even if you don’t call -decodeObject:… in your -initWithCoder: you still have to implement +supportsSecureCoding  and return YES in your class, even if a superclass already did it.

When decoding collections, you must specify the classes of the objects it contains

This is actually documented elsewhere in Apple’s XPC docs. Suppose you have an NSArray. You can’t just say -decodeObjectOfClass:[NSArray class]…. That’s because when NSArray unarchives itself, it can’t specify the class of the objects within it, since they are typed as id  Therefore, you need to specify the classes of those objects for it. In this case, you would use -decodeObjectOfClasses:…, specifying the classes contained by the array as well as NSArray itself.

Some objects will be disabled after secure decoding

Objects like NSPredicate and NSSortDescriptor can take in key paths or selectors making them potentially unsafe. As a result, they are disabled after being securely decoded. To re-enable them, you have to call -allowEvaluation (presumably after doing some sort of check).

You have to turn on secure coding on the coder instances

Sounds a bit obvious except that most of the time you are using the class, not instance, methods to archive/unarchive. Those do not use secure coding nor are there similar class methods for secure coding. Instead, you’ll have to create an instance, set it to secure coding and encode/decode the root object yourself. Of course, you can bundle this up into convenience methods and stick it into a category like so:

 

@implementation NSKeyedArchiver (NoodleExtensions)

 

+ (NSData *)noodleSecurelyArchivedDataWithRootObject:(id)rootObject;

{

    NSMutableData   *data;

    NSKeyedArchiver *archiver;

    

    data = [NSMutableData data];

    archiver = [[NSKeyedArchiver alloc] initForWritingWithMutableData:data];

    [archiver setRequiresSecureCoding:YES];

    [archiver encodeObject:rootObject forKey:NSKeyedArchiveRootObjectKey];

    [archiver finishEncoding];

    

    return data;

}

 

@end

 

@implementation NSKeyedUnarchiver (NoodleExtensions)

 

+ (id)noodleSecurelyUnarchiveObjectOfClass:(Class)classObject withData:(NSData *)data

{

    NSKeyedUnarchiver       *unarchiver;

    

    unarchiver = [[NSKeyedUnarchiver alloc] initForReadingWithData:data];

    [unarchiver setRequiresSecureCoding:YES];

    return [unarchiver decodeObjectOfClass:classObject forKey:NSKeyedArchiveRootObjectKey];

}

 

+ (id)noodleSecurelyUnarchiveObjectOfClasses:(NSSet *)classes withData:(NSData *)data

{

    NSKeyedUnarchiver       *unarchiver;

    

    unarchiver = [[NSKeyedUnarchiver alloc] initForReadingWithData:data];

    [unarchiver setRequiresSecureCoding:YES];

    return [unarchiver decodeObjectOfClasses:classes forKey:NSKeyedArchiveRootObjectKey];

}

 

@end

 

Hopefully this will make your conversion to NSSecureCoding go a little smoother.

Comment » | Cocoa, Downloads, OS X, Programming

Unpredictable Date Formatting

May 27th, 2015 — 5:24pm

Daniel Jalkut posted about his recent issues with NSDateFormatter. Having run into the same issue last month (rdar:/20512790 for you Apple folks keeping score at home), I’ve found that the issue is more confusing than that.

For one, it’s not just the 12/24-hour clock affecting HH in date formats. It seems that in some circumstances, in some locales, literal characters, like period (.), will be changed to colon (:). Not only that, this happens (at least in the case I found it) depending on whether you use hh or HH in your pattern.

Observe this output from my test program:

LOCALE: en_UA

FORMAT: MMMM d, hh.mm.ss a

January 1, 13:00:00

FORMAT: MMMM d, HH.mm.ss a

January 1, 13.00.00 PM

“en_UA” is English as the language, but Ukraine as the region. It’s important to note that there are two components here, the language and region, encoded in the locale. This is important because Apple’s suggestion to use “US_en_POSIX” is incorrect. That only works if English is your language. To do it correctly for the user’s language (not every date format uses numerical months), you need to get the locale for their current language, grab the identifier, tag “-posix” to the end and get a new locale from that. I’m not sure if this type of scheme works for all languages (anyone have documentation to this effect?) so it’s possible this won’t work for some language out there. All in all, a bit of a pain and a bit fragile for something that worked properly before.

And before some of you bring up some argument about respecting the user’s preferences, in the case where I use it, the user is specifying the pattern in the UI and wants to specify a different pattern than the one that they used in their preferences. Also, this API is at a much lower level and should respect the literal pattern provided the developer. I’d suggest the current behavior only mangle the format string when you explicitly use +dateFormatFromTemplate:options:locale: leaving format strings that don’t use that method as-is. In the very least, provide a “literal” mode when it comes to interpreting the format string.

 

1 comment » | Cocoa, OS X, Programming

Delegates vs Blocks

May 15th, 2015 — 12:58pm

This was something that came up at last night’s Cocoaheads so I thought I’d share my thoughts on it here. I’m way overdue for a post anyways.

Both delegates and blocks allow for callbacks, a way to allow an API user to provide custom logic to what is otherwise a black box. With the advent of blocks, there’s a temptation by some to use them everywhere. But as with anytime you have a new toy (ok, not so new in this case but newer), you must resist the temptation to stop using old stuff when is still better suited for the situation.

So, when do you use delegates? Delegates are for when there will be a closer, long term relationship between objects. There is usually a well defined interface for the delegate to follow. The delegate itself is probably maintaining state and consolidating logic for this purpose. From the point of view of the object using the delegate, it is a one-to-one relationship (the delegate might not actually be monogamous in this relationship and be a delegate to several objects). You can rely on one object to know how to handle these cases and expect some level of self consistency.

How about block callbacks? These are best suited for more ad hoc cases. Usually these blocks are called immediately or up to a few times in the not too distant future. An example of the former is having a block called on the elements in an array, after which, control is returned to the caller. For the latter, completion or progress handlers are typical examples, where you initiate some longer running process and want to specify behavior for certain events. The reason blocks are good for this situation is that they can capture the immediate context and use that in the custom code. This is also why blocks are less well suited for longer term relationships as they end up creating undesired or unintended dependencies that are harder to extricate.

The rules are not cut and dry but hopefully you’ll see that there’s a need for both. It’s both fortunate and unfortunate that blocks came into the Cocoa world as late as they did. While it did allow for the creation of some solid and clean use of delegates, it also allowed for some APIs which should have used blocks from the beginning (basically anything which takes a user-supplied context parameter). Hopefully Apple will clean these up over time but for your own APIs, be mindful of the differences.

Comment » | Cocoa, OS X, Programming

Hardcoding Classes

May 15th, 2014 — 12:58pm

When referring to class object, you can refer to it directly by name. When creating an object, you can do:

[[Foo alloc] init]

There are cases though where you want it to be a bit more dynamic. Say you want a class method which creates an autoreleased instance. You might do something like:

+ (id)fooWithName:(NSString *)name
{
   return [[[Foo alloc] initWithName:name] autorelease];
}

The problem with this, of course, is that if you have a subclass, SubFoo, then +fooWithName: will return Foo objects, not SubFoo. To fix this, instead of hardcoding the class, refer to it dynamically:

+ (id)fooWithName:(NSString *)name
{
  return [[self alloc] initWithName:name] autorelease];
}

Since this is a class method, self refers to the class object. Now, no matter which subclass you are in, the proper type of instance will be created. And if you are calling class methods from other class methods in the same class:

+ (void)doBlah
{
  [self doBlech];
}

Hardcoding the class here would call that specific class’s version of the method, even in subclasses, which is usually undesirable.

If you are in an instance method, you can likewise do something like:

- (id)copyWithZone:(NSZone *)zone
{
    return [[[self class] alloc] initWithName:[self name]];
}

Now, this is all well and good. In general, it’s a good idea to to use [self class] instead of hardcoding the name. But let’s take this example:

bundle = [NSBundle bundleForClass:[self class]];

Seems ok. You want the bundle where this class comes from. Or do you? Suppose Foo is in a Framework and SubFoo is in an app using that framework. The above code will return a different bundle for SubFoo than it does for Foo. Depending on what you want to do with that bundle, that can be a bad thing. For instance, finding a nib. Unless it is expected that the subclasses provide their own nib, chances are, the nib you are looking for only exists in Foo’s bundle and the above will fail. The fix? Hardcode the class:

bundle = [NSBundle bundleForClass:[Foo class]]

Here’s an odd case that I ran into some time back. Suppose Foo implements a protocol that has an optional method -bingo. In SubFoo, you implement -bingo but you’re not sure if the superclass (Foo) implements -bingo itself as part of the protocol. If it does, we want to call super but if it doesn’t we can’t. The documentation is silent on the subject so we can’t rely on any particular behavior. How do we check it at run-time?

- (void)bingo
{
   ...
    if ([[[self class] superclass] instancesRespondToSelector:@selector(bingo)])
   {
       [super bingo];
   }
}

Besides the compiler warning that Foo may not implement -bingo, it looks like it works. If you are a SubFoo instance, it gets the Foo class and asks it if its instances can receive -bingo. Ah, but what happens when we have a subclass of SubFoo called ReallySubFoo. Let’s also say that Foo doesn’t implement -bingo. When the above code is called, from a ReallySubFoo instance, we get a SubFoo class which then gets asked if it responds to -bingo. It does because we implemented it in SubFoo. Problem is there is no super implementation of -bingo and we get a crash. Again, the fix is to hardcode the class:

- (void)bingo
{
  ...
   if ([Foo instancesRespondToSelector(bingo)])
   {
       [super bingo];
   }
}

In most cases, dynamically getting the class is the right thing to do but don’t do it indiscriminately. Take a moment to think about how it will play out in subclasses. You may be surprised.

Comment » | Cocoa, OS X, Programming

ConnectionKit

July 10th, 2013 — 10:58am

In Hazel 3.1, I added the ability to upload files. Underneath the hood, I used ConnectionKit 2 to do the heavy lifting. It has an API similar to NSFileManager, making it very easy to add FTP, SFTP and WebDAV support without having to deal with the pesky details of those protocols. There were other factors on why I chose it, such as the fact that not only is it actively developed, but done so by people who use it in their own shipping commercial products. And it didn’t hurt that it’s maintained by my friends at Karelia.

If you have used this feature in Hazel, you would have encountered this interface:

ckopenpanel1.png

It should look a bit familiar. It’s a homegrown, from scratch, implementation of NSOpenPanel made to work on top of ConnectionKit. I felt it important to provide a familiar interface instead of coming up with something new in this case. I did a bit of work to make sure it operated as closely to NSOpenPanel as was reasonable. It has multiple view types (icon, list, and column) and most of the controls you’d expect.

Of course, browsing a filesystem via various network protocols is a different beast so some things had to be done differently. The main thing is that the UI is asynchronous from the network stuff so no beachballing while waiting for responses from the server. The UI should be responsive at all times, even allowing you to cancel the panel at any point. In the upper right, there’s a reload button that changes to a progress indicator to indicate activity and you will see a loading message when trying to fetch the contents of directories that haven’t been loaded yet.

Visually, the biggest difference is the header up top noting which server is being browsed. Also, the sidebar is missing. There aren’t many standard locations that can be put there and I felt that favorite URLs should be managed elsewhere as it didn’t seem likely that users would want to hop from server to server by clicking different entries in the sidebar. The one common directory that I could think of was the home directory and so instead, I gave that its own button (note that not all servers/protocols have such a concept of a home directory and it also depends on what URL you use to connect to the server so YMMV).

Another omission is the search field. Since there’s no notion of Spotlight in any of these protocols, doing an exhaustive search of the remote filesystem would be resource intensive and not very friendly to the servers involved. While I may consider adding it back in to search the current directory, you can get much of the same effect by just typing out the name of the file. Just as in NSOpenPanel, it will end up selecting the entry starting with those characters. I also dropped the Arrange button and the Coverflow view, mainly because I didn’t think people actually used those things. Especially for Coverflow, it wouldn’t be particularly useful since we can’t get file previews without downloading the whole files themselves.

But besides all of that, I think you’ll find it to be a pretty decent facsimile of NSOpenPanel. You might be surprised by how much of NSOpenPanel’s behavior is replicated here, including some features many people don’t know that NSOpenPanel has.

And the resemblance is more than skin deep. CK2OpenPanel (the name of the NSOpenPanel implementation) has a nearly identical API to that of NSOpenPanel. It’s a mostly drop-in replacement for NSOpenPanel so programming to its API should be just as familiar. Why would you care? Well, you should care because I’ve contributed the code for CK2OpenPanel to be included in ConnectionKit. You can see all the code, warts and all. Play around with it. Use it in your own projects. Print it out and wallpaper your home with it. I felt it was the least I could do for all the work that’s been done on the backend. Also, I thought it would be nice if people adopted it so that there was a standard UI for this type of thing.

You can find everything on the ConnectionKit GitHub page. In particular, you want to check out the v2.x-beta branch. Note that there are separate targets/frameworks in the project for the back end (ConnectionKit) and front end (ConnectionKitUI), with the latter dependent on the former. Make sure to check out the README as it should answer a lot of your questions. And if you are interested in contributing to ConnectionKit, please do as there are a bunch of things that need work. S3 support would be nice, for instance.

Last but not least, a big thanks to Mike Abdullah at Karelia for working with me to provide the backend support for this. Rest assured that he will be receiving beers from me at the next conference we both attend.

Enjoy.

Comment » | Cocoa, Downloads, Hazel, OS X, Programming, Software, User Interface

[self note]

October 4th, 2012 — 11:20am

While working on the most recent patch to Hazel, I stumbled across some code that made me scratch my head. It seemed the code was unnecessary and was causing the bug I was trying to fix. I ended up removing that code while doing the fix thinking I must have been drunk/high/had a gun to my head when I wrote that code originally. Initial tests seemed to confirm it as things were working at first. A couple days later I discovered an older bug had resurfaced. Sure enough, I began to realize why I wrote that odd piece of code back when. I ended up backing out my recent changes and making the fix with the original code intact. I also made another change which was to comment the code explaining why the hell I wrote it that way in the first place.

[self now] != [self future]

One of the nice things about being a solo developer is that you don’t have to deal with the overhead involved when working on a team. You don’t have issues with miscommunication that you would when working with others. Or do you? Like it or not, you are still a team. Sure, you probably don’t need to communicate things with someone else about a current issue, but you, yourself, now, is not the same person as yourself a year from now. You’ll find many times that intentions are lost through the annals of time so documenting things as if there are other programmers on your team is a worthy exercise. Comment your code. Write descriptive commit messages. Edit your wiki. Your future self will thank you for it.

8 comments » | Debugging, Programming

Syntax Coloring For Fun And Profit

May 29th, 2012 — 9:32am

At the last NYC Cocoaheads meeting, I did a presentation entitled “Let’s Build a Text Editor: A Practical Introduction to the Cocoa Text System.” In it, I basically go through building a code editor from scratch, pointing out how to use the Cocoa text system along the way. I’m not going to reprise the whole thing here. The first part was an introduction to the basics of creating a document-based app and fitting the text system into that and the second part dealt with adding line numbering, which, if you’ve been paying attention at home, I covered here a while back. Instead, I’m going to focus on the third part, which is implementing syntax coloring.

One major aspect of syntax coloring is actually parsing the text. And guess what? I’m not going to cover that here either. There are books you should read on the topic. Like many people during their college years, I learned from the Dragon book though there are probably books more specific to parsing that you can find out there. Keep in mind, though, that you may not have to write as sophisticated a parser as you think. Sometimes, a lexical scanner is all that you need. Just parse the different elements you want to highlight (comments, strings, keywords, etc.) and ignore the syntactic structure. In some cases, this may be enough.

The focus of this article will be getting the Cocoa text system to do the coloring based on what your parser parses. Before we start, you should download the example project as this article refers to it throughout. I’ve created various snapshots at key points to show the progression. If you want, you can go through the various snapshots to see how the app was built up but for the purposes of this article, start at the snapshot called “Version 4”.

The Basic Approach

At the core of Cocoa text system is the NSTextStorage object. It stores the actual text as well as any formatting attributes. Being a subclass of NSMutableAttributedString, NSTextStorage allows you to apply different attributes to ranges of characters. For syntax highlighting, all we care about is the NSForegroundColorAttributeName attribute, which is the attribute that affects the font color.

If you look at the –parse: method in NoodleSyntaxHighlighter, you’ll see that it scans for C-style comments (/*...*/) and string literals ("..."). When it finds the range of a particular entity, it calls -addAttributes:range: on the NSTextStorage to annotate the characters with the appropriate color based on which entity we found. Note that this is how you modify any attributed string; it’s not specific to NSTextStorage. To make sure we keep up with the changes, we implement NSTextStorageDelegate protocol’s -textStorageDidProcessEditing: method. There’s an equivalent NSTextStorageDidProcessEditingNotification though I believe modifying the text storage is only allowed via the delegate method.

If you compile and run at this point, you’ll see that it pretty much works as advertised. We’re done, right? Not exactly. Try loading in a large-ish document. Around a couple thousand lines or so. Start typing really fast. It might depend on your machine, but you’ll find that the typing lags a bit. It seems that modifying the NSTextStorage is not quite as efficient as we’d like. Luckily, there’s a better way to do this.

Using Temporary Attributes

NSLayoutManager has a thing called temporary attributes. These are attributes just like attributes you use on an attributed string, except that they’re, well, temporary. They are more lightweight and designed specifically for cases like this, where the attributes are only used for display and aren’t something intrinsic to the document itself. It’s used for features like spellchecking and other places where you need to make temporary annotations to the text.

Imagine writing a rich text editor. You don’t want to make display-only changes to the NSTextStorage since those changes could end up being saved with the document. Sure, the code editor in this example doesn’t allow rich text but you can see where semantic division lies. The NSTextStorage is your “model” and you don’t want something like syntax highlighting, a “view” feature, to change the model.

In the example project, restore snapshot “Version 5”. This code isn’t much different. It’s adding attributes as before but instead of adding them to the NSTextStorage, they are added via the NSLayoutManager. Now, compile and run it and, if it wasn’t automatically restored from before, load up the large document you used earlier. You should find that it keeps up with your typing this time around.

Now you could stop here but I’d like to take it a step further. You have this nice parser and all. Surely, the ability to pick out different entities in the document is useful for more than just syntax coloring. Maybe you want to add more smarts to your editor. Maybe triple clicks will select the whole comment or string literal instead of just a paragraph/line. Maybe you want to know where code blocks begin and end for code folding. Or maybe you want special tool tips when hovering over different entities.

You can ask the layout manager for the temporary attributes at any character position. Just check for the color that’s set there and deduce which entity (comment or string) it is. Suppose, though, that the user can change the colors in the preferences. If entities are set to be the same color, this method won’t work.

Creating Your Own Custom Attributes

There’s nothing saying that you can’t create and add your own attributes to an attributed string. Sure, only ones defined by Cocoa will be used by the system for display but let’s decide for the moment to annotate our text semantically. Instead of setting the NSForegroundColorAttributeName, we set our own custom attribute to indicate the type of element we’ve found.

Restore snapshot “Version 6” in the example project. At the top of the NoodleSyntaxHighlighter implementation you’ll see that I’ve defined our own attribute (“noodleElementType”) and the two possible values (“comment” and “string”). The -parse: method is the same as before except instead of setting colors, we set our own custom attribute, marking regions of text as comments or strings.

That’s great but how do we get our colors? If you look at the NSLayoutManagerDelegate protocol, you’ll see a method called -layoutManager:shouldUseTemporaryAttributes:forDrawingToScreen:atCharacterIndex: effectiveRange:. What this method does is allow you to substitute different attributes for the ones that are there already. In our implementation, we check what element type we set during the parse phase, look up its color and return it as the NSForegroundColor. So, our NSTextStorage has semantic markup but when displayed, we can substitute in the proper display attributes to get it to color properly.

• • •

Now, there’s one thing I glossed over which is an obvious optimization. Notice how we re-parse the whole document on every change. When you get the text storage notification, you can query it for the range of characters that was changed via its -editedRange method. You can use this information to limit the range of the document you have to re-parse. What you can do is look at what element is there already (from your annotations from the last parse pass) and start parsing from the beginning of that element. You can even tighten it up further by limiting how far it has to parse. I leave implementing this as an exercise for the reader.

As you can see, doing the coloring part isn’t so bad though there are some slightly subtle details to keep in mind. The hard part is the parsing which I’ve conveniently (for myself) left for you to figure out on your own. Enjoy.

And in case you missed the link to the example project above: NoodleEdit.zip

2 comments » | Cocoa, OS X, Programming, User Interface

Back to top