Category: Programming


On Dynamism

May 23rd, 2016 — 7:15pm

I’ve kept my hat out of the discussions of Swift vs. Objective-C for several reasons, but the main one is that frankly, I’ve been busy. I ship a product, one that I would consider successful, and recently shipped a major release. What does that mean? It means that debating these types of issues and playing with the latest thing is a luxury. Because I have a shipping product with a lot of users, I can’t just change things unless I find that it adds value to my users. My job is to solve my users’ problems. Now that I have a little bit of time to do so, this is one lens I’m hoping to add to this discussion

In addition, years ago, when Java was introduced, I jumped on that bandwagon, leaving behind Objective-C. It started out writing replacements for Foundation and AppKit in Java and ended with writing large scale servers. I was on that wagon for a good 10 years before returning, so I wasn’t a tourist there. This is yet another lens I also want to add to the discussion as history seems to be repeating itself and many younger people seem to be missing out on this part.

But let’s start with the current issue about dynamism since that’s the hot topic.

One thing many people seem to overlook about the dynamism of Objective-C is that it enabled NeXT (and Apple) to provide better GUI tools. Using dynamism, they were able to make GUI building declarative in nature. Connect this to that. Call this method. All stored in a file that was (and still is) data, not code. Competitors at the time (and today) resorted to code generation which is fragile and, ironically, unsafe. Yes, you could have a more declarative file format, but implementing that in using a static language required a lot of hard-coding and switch statements. Not the elegance that many people claim to be moving towards.

I’m not saying that a language has to be purely dynamic but it shouldn’t be purely static either. It think it’s spurious not to credit a level of dynamism for the quality of apps on Apple platforms over the years, and to be pedantic, the NeXT ones as well – many of which were considered the best on any platform at the time. To deny that, I feel, shows a lack of understanding of what has made the platform great all these years.

Moving from a dynamic language to a more static one seems refreshing at first. I went through the exact same thing when switching to Java. Yes, I can hear some of you grumbling “Java” out there but the irony is that there are more similarities in the evolution between Java and Swift than you may want to admit.

Everything seemed so much “tighter” in Java. Interfaces (Java’s version of protocols) everywhere! No more pointers! Checked exceptions! It’s all so new and exciting. You might just have to take my word on this, but any new language that you really like will always seem to make you “more productive”. A colleague and I (plus maybe one, or one and a half, devs who were on the periphery) implemented our own version of Foundation and AppKit in what I would consider a pretty short amount of time. Other people at Lighthouse can correct me on this but my recollection was that it was on the scale of months, not years. You can read up more on it here.

Was it the language? At the time we might say yes but in hindsight, I’d say it was just that we were just super excited and into the project. I was working with an exceptional team and being a new platform, we were excited about making our mark on it. Yeah, there were bits about Java that were better than Objective-C. But there were bits that were worse. The point here, though, is that until the honeymoon is over and you are maintaining a project over several years, it’s hard to make real proclamations about productivity. Truth be told, I’d say the biggest impact on software quality and productivity is the quality of engineers regardless of any other factors.

Looking back on my years with Java, I find the biggest problems with the platform was this idea that everything can be done by the compiler. It sounds great on paper but when implemented, it resulted in all sorts of contortions to work around not letting any dynamism in. It resulted in a lot of glue being written that I wouldn’t have to write in a more dynamic platform. It resulted in cascading changes across code to make trivial modifications just to appease the compiler. Maybe in a future post I’ll go into specific language features that I found problematic and show parallel trends/features in Swift but let’s just say that less is more. It’s an aesthetic that I always felt that put Apple above the rest and something which I feel like in some ways is being abandoned.

Now, I’m sure many of you will just paint me as an old, stubborn Obj-C luddite. And it’s true on some level but that doesn’t mean you should discount my experience. Many of us old fogeys are working on products that have shipped for years. That means that (1) we are solving user problems (2) have immense experience with doing app development (3) really understanding the frameworks (4) have been maintaining a code base.

Let’s go through these points:

1. We are solving user problems.

We have real problems to solve for people outside of ourselves. We need to be practical to survive. Users don’t care about the tech underneath. We have to ship them solutions. This oftentimes requires rapid response. This also oftentimes involves working around Apple bugs, bugs that never get fixed and will only grow in number as Apple maintains their yearly release schedule (something to think about for proponents of the “final” keyword). Agility and flexibility are paramount here. Reality is not pretty. The user could care less how beautiful your code is if you aren’t solving their problems.

2. We have immense experience with doing app development

I want to emphasize the “app” part of that. A common argument for very strict type safety are for cases which aren’t apps. The predominant software in the Apple ecosystem is end user applications. Let’s try and not lose focus here and argue the case for categories of software that aren’t Apple’s forte. If you are writing some life-critical software on an Apple platform, god help you. Even with Swift, you are stuck with Xcode and Apple’s insistence on yearly new OS releases.

3. We really understand the frameworks

I would take many of the arguments for Swift more seriously if there weren’t strawman examples set up on the Obj-C side. You’ll have to dig up Brent Simmons’ posts on this, but when I see the kooky examples of horrible old Obj-C way, what it tells me is more that you don’t understand the frameworks. It resembles informercials where people are fumbling opening a jar and then are presented with some useless gadget to solve a non-existent problem. I’m sure things can be done better than they are now but providing bad examples indicating ignorance of what it is you are trying to replace doesn’t help your argument. And I have seen plenty of examples of new people replacing what they don’t understand. Please don’t be one of those people.

4. We have been maintaining a code base for a while

I have been writing Hazel for ten years. Many of us old-time app developers are in similar positions. Speaking with many other devs, most of whom write iOS apps, I’d be lucky to find those that have maintained a code base for a third of that time. Sure, there are parts of the code that are ugly, but that happens on any platform. Same thing happened on Java. No matter what you do, time is always against you. When I was working with Java, I found the technical debt grew in a different fashion. It was hard to change anything. This was because changes in stricter languages tended to cascade. Because everything has to declare everything about itself, changing code turns into a Facebook feed, except where you and everyone else are obligated to respond to every stupid pic and forward of Snopes-bait. With a more dynamic code base, I can say I have less code to maintain and that I can be more agile. Not that it’s without it’s own problems. But let’s not act like type-safety is a panacea.

And to give more of my personal perspective on the language debate, I’d love to have something better than Objective-C. But I’d also like to have something better than Swift. My personal pet peeve is that we are in the 21st century and we still have to give a shit about the storage type of numbers. I’d be happy to get rid of scalar primitives entirely.

Ultimately, though, I need something that solves my problems. My problems aren’t performance. They aren’t type-safety (maybe it’s just me, but I rarely have issues with it and when I do, the time it takes to fix it is far less than the time specifying everything to a tee everywhere else). They aren’t being able to write clever VDLs. For me, it’s writing apps that solve my users’ problems and getting them out in a timely fashion. As it stands now, Swift (at least pure-Swift, or even current Swift as a non-ABI-stable moving target) does not do that for me.

Now maybe these same problems can be solved in a static way but what I’m not seeing from the static-camp are (decent) solutions. What I’m seeing are either hand-waving or the same crufty code-generation, write tons of repetitive boilerplate type of solutions that I had hoped we had left behind in the 90s.

What I do see makes me worry that it’s not the experienced app-writers that are being heard. Unfortunately, many of us are too busy to sit in the various mailing lists and forums. Yes, ok, we lose. But we all lose if those creating the platform don’t draw from the experience of those who have built upon it successfully. As someone who has done both (write frameworks and apps) and seen what happens when the former ignore the latter (the rise and fall of Java), I’m hoping history doesn’t repeat itself.

That’s my somewhat meandering brain dump for the moment as I do have to get back to working on my next patch. Sure, these are just my opinions but my hope is that we can at least look for more balance in our approaches to solving actual problems.

6 comments » | Cocoa, OS X, Programming

NSSecureCoding

November 9th, 2015 — 1:30pm

I’ve had it on my list to convert all my NSCoding classes to use NSSecureCoding and this recent post reminded me so I figured now is as good a time as ever.

How do you go about using secure coding? Here are the Apple docs. Unfortunately, those docs don’t give the whole picture. There are a few details you need to be aware of.

When you subclass an object that supports NSSecureCoding, if you override -initWithCoder, you must also implement +supportsSecureCoding

Even if you don’t call -decodeObject:… in your -initWithCoder: you still have to implement +supportsSecureCoding  and return YES in your class, even if a superclass already did it.

When decoding collections, you must specify the classes of the objects it contains

This is actually documented elsewhere in Apple’s XPC docs. Suppose you have an NSArray. You can’t just say -decodeObjectOfClass:[NSArray class]…. That’s because when NSArray unarchives itself, it can’t specify the class of the objects within it, since they are typed as id  Therefore, you need to specify the classes of those objects for it. In this case, you would use -decodeObjectOfClasses:…, specifying the classes contained by the array as well as NSArray itself.

Some objects will be disabled after secure decoding

Objects like NSPredicate and NSSortDescriptor can take in key paths or selectors making them potentially unsafe. As a result, they are disabled after being securely decoded. To re-enable them, you have to call -allowEvaluation (presumably after doing some sort of check).

You have to turn on secure coding on the coder instances

Sounds a bit obvious except that most of the time you are using the class, not instance, methods to archive/unarchive. Those do not use secure coding nor are there similar class methods for secure coding. Instead, you’ll have to create an instance, set it to secure coding and encode/decode the root object yourself. Of course, you can bundle this up into convenience methods and stick it into a category like so:

 

@implementation NSKeyedArchiver (NoodleExtensions)

 

+ (NSData *)noodleSecurelyArchivedDataWithRootObject:(id)rootObject;

{

    NSMutableData   *data;

    NSKeyedArchiver *archiver;

    

    data = [NSMutableData data];

    archiver = [[NSKeyedArchiver alloc] initForWritingWithMutableData:data];

    [archiver setRequiresSecureCoding:YES];

    [archiver encodeObject:rootObject forKey:NSKeyedArchiveRootObjectKey];

    [archiver finishEncoding];

    

    return data;

}

 

@end

 

@implementation NSKeyedUnarchiver (NoodleExtensions)

 

+ (id)noodleSecurelyUnarchiveObjectOfClass:(Class)classObject withData:(NSData *)data

{

    NSKeyedUnarchiver       *unarchiver;

    

    unarchiver = [[NSKeyedUnarchiver alloc] initForReadingWithData:data];

    [unarchiver setRequiresSecureCoding:YES];

    return [unarchiver decodeObjectOfClass:classObject forKey:NSKeyedArchiveRootObjectKey];

}

 

+ (id)noodleSecurelyUnarchiveObjectOfClasses:(NSSet *)classes withData:(NSData *)data

{

    NSKeyedUnarchiver       *unarchiver;

    

    unarchiver = [[NSKeyedUnarchiver alloc] initForReadingWithData:data];

    [unarchiver setRequiresSecureCoding:YES];

    return [unarchiver decodeObjectOfClasses:classes forKey:NSKeyedArchiveRootObjectKey];

}

 

@end

 

Hopefully this will make your conversion to NSSecureCoding go a little smoother.

Comment » | Cocoa, Downloads, OS X, Programming

Unpredictable Date Formatting

May 27th, 2015 — 5:24pm

Daniel Jalkut posted about his recent issues with NSDateFormatter. Having run into the same issue last month (rdar:/20512790 for you Apple folks keeping score at home), I’ve found that the issue is more confusing than that.

For one, it’s not just the 12/24-hour clock affecting HH in date formats. It seems that in some circumstances, in some locales, literal characters, like period (.), will be changed to colon (:). Not only that, this happens (at least in the case I found it) depending on whether you use hh or HH in your pattern.

Observe this output from my test program:

LOCALE: en_UA

FORMAT: MMMM d, hh.mm.ss a

January 1, 13:00:00

FORMAT: MMMM d, HH.mm.ss a

January 1, 13.00.00 PM

“en_UA” is English as the language, but Ukraine as the region. It’s important to note that there are two components here, the language and region, encoded in the locale. This is important because Apple’s suggestion to use “US_en_POSIX” is incorrect. That only works if English is your language. To do it correctly for the user’s language (not every date format uses numerical months), you need to get the locale for their current language, grab the identifier, tag “-posix” to the end and get a new locale from that. I’m not sure if this type of scheme works for all languages (anyone have documentation to this effect?) so it’s possible this won’t work for some language out there. All in all, a bit of a pain and a bit fragile for something that worked properly before.

And before some of you bring up some argument about respecting the user’s preferences, in the case where I use it, the user is specifying the pattern in the UI and wants to specify a different pattern than the one that they used in their preferences. Also, this API is at a much lower level and should respect the literal pattern provided the developer. I’d suggest the current behavior only mangle the format string when you explicitly use +dateFormatFromTemplate:options:locale: leaving format strings that don’t use that method as-is. In the very least, provide a “literal” mode when it comes to interpreting the format string.

 

1 comment » | Cocoa, OS X, Programming

Delegates vs Blocks

May 15th, 2015 — 12:58pm

This was something that came up at last night’s Cocoaheads so I thought I’d share my thoughts on it here. I’m way overdue for a post anyways.

Both delegates and blocks allow for callbacks, a way to allow an API user to provide custom logic to what is otherwise a black box. With the advent of blocks, there’s a temptation by some to use them everywhere. But as with anytime you have a new toy (ok, not so new in this case but newer), you must resist the temptation to stop using old stuff when is still better suited for the situation.

So, when do you use delegates? Delegates are for when there will be a closer, long term relationship between objects. There is usually a well defined interface for the delegate to follow. The delegate itself is probably maintaining state and consolidating logic for this purpose. From the point of view of the object using the delegate, it is a one-to-one relationship (the delegate might not actually be monogamous in this relationship and be a delegate to several objects). You can rely on one object to know how to handle these cases and expect some level of self consistency.

How about block callbacks? These are best suited for more ad hoc cases. Usually these blocks are called immediately or up to a few times in the not too distant future. An example of the former is having a block called on the elements in an array, after which, control is returned to the caller. For the latter, completion or progress handlers are typical examples, where you initiate some longer running process and want to specify behavior for certain events. The reason blocks are good for this situation is that they can capture the immediate context and use that in the custom code. This is also why blocks are less well suited for longer term relationships as they end up creating undesired or unintended dependencies that are harder to extricate.

The rules are not cut and dry but hopefully you’ll see that there’s a need for both. It’s both fortunate and unfortunate that blocks came into the Cocoa world as late as they did. While it did allow for the creation of some solid and clean use of delegates, it also allowed for some APIs which should have used blocks from the beginning (basically anything which takes a user-supplied context parameter). Hopefully Apple will clean these up over time but for your own APIs, be mindful of the differences.

Comment » | Cocoa, OS X, Programming

Hardcoding Classes

May 15th, 2014 — 12:58pm

When referring to class object, you can refer to it directly by name. When creating an object, you can do:

[[Foo alloc] init]

There are cases though where you want it to be a bit more dynamic. Say you want a class method which creates an autoreleased instance. You might do something like:

+ (id)fooWithName:(NSString *)name
{
   return [[[Foo alloc] initWithName:name] autorelease];
}

The problem with this, of course, is that if you have a subclass, SubFoo, then +fooWithName: will return Foo objects, not SubFoo. To fix this, instead of hardcoding the class, refer to it dynamically:

+ (id)fooWithName:(NSString *)name
{
  return [[self alloc] initWithName:name] autorelease];
}

Since this is a class method, self refers to the class object. Now, no matter which subclass you are in, the proper type of instance will be created. And if you are calling class methods from other class methods in the same class:

+ (void)doBlah
{
  [self doBlech];
}

Hardcoding the class here would call that specific class’s version of the method, even in subclasses, which is usually undesirable.

If you are in an instance method, you can likewise do something like:

- (id)copyWithZone:(NSZone *)zone
{
    return [[[self class] alloc] initWithName:[self name]];
}

Now, this is all well and good. In general, it’s a good idea to to use [self class] instead of hardcoding the name. But let’s take this example:

bundle = [NSBundle bundleForClass:[self class]];

Seems ok. You want the bundle where this class comes from. Or do you? Suppose Foo is in a Framework and SubFoo is in an app using that framework. The above code will return a different bundle for SubFoo than it does for Foo. Depending on what you want to do with that bundle, that can be a bad thing. For instance, finding a nib. Unless it is expected that the subclasses provide their own nib, chances are, the nib you are looking for only exists in Foo’s bundle and the above will fail. The fix? Hardcode the class:

bundle = [NSBundle bundleForClass:[Foo class]]

Here’s an odd case that I ran into some time back. Suppose Foo implements a protocol that has an optional method -bingo. In SubFoo, you implement -bingo but you’re not sure if the superclass (Foo) implements -bingo itself as part of the protocol. If it does, we want to call super but if it doesn’t we can’t. The documentation is silent on the subject so we can’t rely on any particular behavior. How do we check it at run-time?

- (void)bingo
{
   ...
    if ([[[self class] superclass] instancesRespondToSelector:@selector(bingo)])
   {
       [super bingo];
   }
}

Besides the compiler warning that Foo may not implement -bingo, it looks like it works. If you are a SubFoo instance, it gets the Foo class and asks it if its instances can receive -bingo. Ah, but what happens when we have a subclass of SubFoo called ReallySubFoo. Let’s also say that Foo doesn’t implement -bingo. When the above code is called, from a ReallySubFoo instance, we get a SubFoo class which then gets asked if it responds to -bingo. It does because we implemented it in SubFoo. Problem is there is no super implementation of -bingo and we get a crash. Again, the fix is to hardcode the class:

- (void)bingo
{
  ...
   if ([Foo instancesRespondToSelector(bingo)])
   {
       [super bingo];
   }
}

In most cases, dynamically getting the class is the right thing to do but don’t do it indiscriminately. Take a moment to think about how it will play out in subclasses. You may be surprised.

Comment » | Cocoa, OS X, Programming

ConnectionKit

July 10th, 2013 — 10:58am

In Hazel 3.1, I added the ability to upload files. Underneath the hood, I used ConnectionKit 2 to do the heavy lifting. It has an API similar to NSFileManager, making it very easy to add FTP, SFTP and WebDAV support without having to deal with the pesky details of those protocols. There were other factors on why I chose it, such as the fact that not only is it actively developed, but done so by people who use it in their own shipping commercial products. And it didn’t hurt that it’s maintained by my friends at Karelia.

If you have used this feature in Hazel, you would have encountered this interface:

ckopenpanel1.png

It should look a bit familiar. It’s a homegrown, from scratch, implementation of NSOpenPanel made to work on top of ConnectionKit. I felt it important to provide a familiar interface instead of coming up with something new in this case. I did a bit of work to make sure it operated as closely to NSOpenPanel as was reasonable. It has multiple view types (icon, list, and column) and most of the controls you’d expect.

Of course, browsing a filesystem via various network protocols is a different beast so some things had to be done differently. The main thing is that the UI is asynchronous from the network stuff so no beachballing while waiting for responses from the server. The UI should be responsive at all times, even allowing you to cancel the panel at any point. In the upper right, there’s a reload button that changes to a progress indicator to indicate activity and you will see a loading message when trying to fetch the contents of directories that haven’t been loaded yet.

Visually, the biggest difference is the header up top noting which server is being browsed. Also, the sidebar is missing. There aren’t many standard locations that can be put there and I felt that favorite URLs should be managed elsewhere as it didn’t seem likely that users would want to hop from server to server by clicking different entries in the sidebar. The one common directory that I could think of was the home directory and so instead, I gave that its own button (note that not all servers/protocols have such a concept of a home directory and it also depends on what URL you use to connect to the server so YMMV).

Another omission is the search field. Since there’s no notion of Spotlight in any of these protocols, doing an exhaustive search of the remote filesystem would be resource intensive and not very friendly to the servers involved. While I may consider adding it back in to search the current directory, you can get much of the same effect by just typing out the name of the file. Just as in NSOpenPanel, it will end up selecting the entry starting with those characters. I also dropped the Arrange button and the Coverflow view, mainly because I didn’t think people actually used those things. Especially for Coverflow, it wouldn’t be particularly useful since we can’t get file previews without downloading the whole files themselves.

But besides all of that, I think you’ll find it to be a pretty decent facsimile of NSOpenPanel. You might be surprised by how much of NSOpenPanel’s behavior is replicated here, including some features many people don’t know that NSOpenPanel has.

And the resemblance is more than skin deep. CK2OpenPanel (the name of the NSOpenPanel implementation) has a nearly identical API to that of NSOpenPanel. It’s a mostly drop-in replacement for NSOpenPanel so programming to its API should be just as familiar. Why would you care? Well, you should care because I’ve contributed the code for CK2OpenPanel to be included in ConnectionKit. You can see all the code, warts and all. Play around with it. Use it in your own projects. Print it out and wallpaper your home with it. I felt it was the least I could do for all the work that’s been done on the backend. Also, I thought it would be nice if people adopted it so that there was a standard UI for this type of thing.

You can find everything on the ConnectionKit GitHub page. In particular, you want to check out the v2.x-beta branch. Note that there are separate targets/frameworks in the project for the back end (ConnectionKit) and front end (ConnectionKitUI), with the latter dependent on the former. Make sure to check out the README as it should answer a lot of your questions. And if you are interested in contributing to ConnectionKit, please do as there are a bunch of things that need work. S3 support would be nice, for instance.

Last but not least, a big thanks to Mike Abdullah at Karelia for working with me to provide the backend support for this. Rest assured that he will be receiving beers from me at the next conference we both attend.

Enjoy.

Comment » | Cocoa, Downloads, Hazel, OS X, Programming, Software, User Interface

[self note]

October 4th, 2012 — 11:20am

While working on the most recent patch to Hazel, I stumbled across some code that made me scratch my head. It seemed the code was unnecessary and was causing the bug I was trying to fix. I ended up removing that code while doing the fix thinking I must have been drunk/high/had a gun to my head when I wrote that code originally. Initial tests seemed to confirm it as things were working at first. A couple days later I discovered an older bug had resurfaced. Sure enough, I began to realize why I wrote that odd piece of code back when. I ended up backing out my recent changes and making the fix with the original code intact. I also made another change which was to comment the code explaining why the hell I wrote it that way in the first place.

[self now] != [self future]

One of the nice things about being a solo developer is that you don’t have to deal with the overhead involved when working on a team. You don’t have issues with miscommunication that you would when working with others. Or do you? Like it or not, you are still a team. Sure, you probably don’t need to communicate things with someone else about a current issue, but you, yourself, now, is not the same person as yourself a year from now. You’ll find many times that intentions are lost through the annals of time so documenting things as if there are other programmers on your team is a worthy exercise. Comment your code. Write descriptive commit messages. Edit your wiki. Your future self will thank you for it.

8 comments » | Debugging, Programming

Syntax Coloring For Fun And Profit

May 29th, 2012 — 9:32am

At the last NYC Cocoaheads meeting, I did a presentation entitled “Let’s Build a Text Editor: A Practical Introduction to the Cocoa Text System.” In it, I basically go through building a code editor from scratch, pointing out how to use the Cocoa text system along the way. I’m not going to reprise the whole thing here. The first part was an introduction to the basics of creating a document-based app and fitting the text system into that and the second part dealt with adding line numbering, which, if you’ve been paying attention at home, I covered here a while back. Instead, I’m going to focus on the third part, which is implementing syntax coloring.

One major aspect of syntax coloring is actually parsing the text. And guess what? I’m not going to cover that here either. There are books you should read on the topic. Like many people during their college years, I learned from the Dragon book though there are probably books more specific to parsing that you can find out there. Keep in mind, though, that you may not have to write as sophisticated a parser as you think. Sometimes, a lexical scanner is all that you need. Just parse the different elements you want to highlight (comments, strings, keywords, etc.) and ignore the syntactic structure. In some cases, this may be enough.

The focus of this article will be getting the Cocoa text system to do the coloring based on what your parser parses. Before we start, you should download the example project as this article refers to it throughout. I’ve created various snapshots at key points to show the progression. If you want, you can go through the various snapshots to see how the app was built up but for the purposes of this article, start at the snapshot called “Version 4”.

The Basic Approach

At the core of Cocoa text system is the NSTextStorage object. It stores the actual text as well as any formatting attributes. Being a subclass of NSMutableAttributedString, NSTextStorage allows you to apply different attributes to ranges of characters. For syntax highlighting, all we care about is the NSForegroundColorAttributeName attribute, which is the attribute that affects the font color.

If you look at the –parse: method in NoodleSyntaxHighlighter, you’ll see that it scans for C-style comments (/*...*/) and string literals ("..."). When it finds the range of a particular entity, it calls -addAttributes:range: on the NSTextStorage to annotate the characters with the appropriate color based on which entity we found. Note that this is how you modify any attributed string; it’s not specific to NSTextStorage. To make sure we keep up with the changes, we implement NSTextStorageDelegate protocol’s -textStorageDidProcessEditing: method. There’s an equivalent NSTextStorageDidProcessEditingNotification though I believe modifying the text storage is only allowed via the delegate method.

If you compile and run at this point, you’ll see that it pretty much works as advertised. We’re done, right? Not exactly. Try loading in a large-ish document. Around a couple thousand lines or so. Start typing really fast. It might depend on your machine, but you’ll find that the typing lags a bit. It seems that modifying the NSTextStorage is not quite as efficient as we’d like. Luckily, there’s a better way to do this.

Using Temporary Attributes

NSLayoutManager has a thing called temporary attributes. These are attributes just like attributes you use on an attributed string, except that they’re, well, temporary. They are more lightweight and designed specifically for cases like this, where the attributes are only used for display and aren’t something intrinsic to the document itself. It’s used for features like spellchecking and other places where you need to make temporary annotations to the text.

Imagine writing a rich text editor. You don’t want to make display-only changes to the NSTextStorage since those changes could end up being saved with the document. Sure, the code editor in this example doesn’t allow rich text but you can see where semantic division lies. The NSTextStorage is your “model” and you don’t want something like syntax highlighting, a “view” feature, to change the model.

In the example project, restore snapshot “Version 5”. This code isn’t much different. It’s adding attributes as before but instead of adding them to the NSTextStorage, they are added via the NSLayoutManager. Now, compile and run it and, if it wasn’t automatically restored from before, load up the large document you used earlier. You should find that it keeps up with your typing this time around.

Now you could stop here but I’d like to take it a step further. You have this nice parser and all. Surely, the ability to pick out different entities in the document is useful for more than just syntax coloring. Maybe you want to add more smarts to your editor. Maybe triple clicks will select the whole comment or string literal instead of just a paragraph/line. Maybe you want to know where code blocks begin and end for code folding. Or maybe you want special tool tips when hovering over different entities.

You can ask the layout manager for the temporary attributes at any character position. Just check for the color that’s set there and deduce which entity (comment or string) it is. Suppose, though, that the user can change the colors in the preferences. If entities are set to be the same color, this method won’t work.

Creating Your Own Custom Attributes

There’s nothing saying that you can’t create and add your own attributes to an attributed string. Sure, only ones defined by Cocoa will be used by the system for display but let’s decide for the moment to annotate our text semantically. Instead of setting the NSForegroundColorAttributeName, we set our own custom attribute to indicate the type of element we’ve found.

Restore snapshot “Version 6” in the example project. At the top of the NoodleSyntaxHighlighter implementation you’ll see that I’ve defined our own attribute (“noodleElementType”) and the two possible values (“comment” and “string”). The -parse: method is the same as before except instead of setting colors, we set our own custom attribute, marking regions of text as comments or strings.

That’s great but how do we get our colors? If you look at the NSLayoutManagerDelegate protocol, you’ll see a method called -layoutManager:shouldUseTemporaryAttributes:forDrawingToScreen:atCharacterIndex: effectiveRange:. What this method does is allow you to substitute different attributes for the ones that are there already. In our implementation, we check what element type we set during the parse phase, look up its color and return it as the NSForegroundColor. So, our NSTextStorage has semantic markup but when displayed, we can substitute in the proper display attributes to get it to color properly.

• • •

Now, there’s one thing I glossed over which is an obvious optimization. Notice how we re-parse the whole document on every change. When you get the text storage notification, you can query it for the range of characters that was changed via its -editedRange method. You can use this information to limit the range of the document you have to re-parse. What you can do is look at what element is there already (from your annotations from the last parse pass) and start parsing from the beginning of that element. You can even tighten it up further by limiting how far it has to parse. I leave implementing this as an exercise for the reader.

As you can see, doing the coloring part isn’t so bad though there are some slightly subtle details to keep in mind. The hard part is the parsing which I’ve conveniently (for myself) left for you to figure out on your own. Enjoy.

And in case you missed the link to the example project above: NoodleEdit.zip

2 comments » | Cocoa, OS X, Programming, User Interface

The Proper Care and Feeding of NSImage

April 15th, 2011 — 10:17am

[This was the topic of my presentation at the NYC Cocoaheads meeting last night. I thought it would be nice to also post on the topic here.]

[I’ve received email from Ken Ferry. See addendum at the bottom]

NSImage is a troublesome class. Over the years, it’s been misunderstood and abused. I think much of this is because of a lack of conceptual clarity in the docs and examples and the API itself can be confusing and misleading. Add to this having to mix with CGImage and CIImage and you can end up with a confused mess.

The way I like to think of NSImage is that it’s a semantic image. When you look at icons, they are made up of several versions at different resolutions. Technically, it’s 4 or 5 images but NSImage wraps those all up into one notion of an image. Semantically, it is an icon of, say, a house but underneath it’s made up of several actual images of a house. The main reason for these different versions is that some graphics do not scale well and it helps to have hand tuned versions, especially for the smaller sizes. You can also do things like omit certain features in the smaller sizes to simplify the graphics and make the icon more recognizable.

One key misunderstanding is the notion that NSImage has actual image data. While there are parts of the API that deal with a cached bitmap (more on this below), NSImage itself is not based on image data. It is best to think of NSImage as a mediator between the image data in the various representations and the drawing context.

Structure3

Loading an image from a file and drawing it is usually straightforward. If there are multiple representations, NSImage will figure out the correct one to use when drawing it. Most of this is automatic and is an important key to understanding how to use NSImage. As you will see further on, bypassing this mechanism leads to many issues with size and resolution mismatches when processing NSImages.

I’ve created a little program to help illustrate the points in this article. Since it also includes a new reusable class, I’ve included it in my NoodleKit repo. I suggest checking it out. Just compile and run the ImageLab target (you may need to set the active executable in XCode in the project menu). The rest of this article will be referring to it so I recommend you run it while reading the rest.

When you launch the app, it will load the test image. It’s just a colored circle. Now resize the window. You’ll see that the color changes as you resize. The icon itself is made up of four different sized images and to make it clear which representation is being used, I made the circle different colors for each one. Whenever the color changes when you resize, NSImage is switching to a different representation based on the size. Here’s what the different reps look like in Preview:

original

Size is another confusing aspect of NSImage. One thing to remember is that size is not pixels. It represents a coordinate space. One way to think of it is that NSImage’s size is like NSView’s frame while NSImageRep’s size is like NSView’s bounds. For NSImage, its size is a suggested size to draw the image in the user coordinate space. If possible, it’s best to explicitly specify the destination rect.

Let’s take the case of drawing some custom graphics on top of an existing icon with several representations, like drawing a badge over your app icon. A common way to do this is to lock focus on the NSImage, do your drawing, unlock focus and then draw the resulting image. What -lockFocus actually does is allow you to draw into a cache. This has probably lead to a lot of the misunderstanding that NSImage holds image data. Unfortunately, there are issues with this approach. Mainly, because there is no context about where this image will be drawn so the resulting image is tied to a specific resolution. Also, you are editing a cache so none of this image data is reflected in the original image reps and actually, from poking around, it’s actually destructive in that it may end up removing any other representations.

In our case here we are modifying an icon with different sized representations, what we end up doing is locking the resulting image into the size that happened to be set on the original NSImage. In many cases, you may not know how and where this icon will be used but if any scaling is involved, you may end up having an icon with the wrong version being displayed. In the example program, select “Modified (lock focus)” in the pop-up. Here it picks the 64×64 version (as indicated by the green circle) which becomes a problem when you scale the image up as shown on the left in the image below.

lockFocus

The one on the right is a version which uses an  NSImageRep subclass (select “Modified (custom image rep)” in the pop-up). Notice that it picks the appropriate size as you resize the window. Why is this the case? It’s because NSImage doesn’t ask the image rep to draw until the NSImage itself is drawn. At that point, you actually have information about the destination, including how big it will actually appear on screen. NSImage is able to use this context to determine the right match between different sized images and the resolution of the output context. The same goes for any drawing code.

Subclassing NSImageRep is quite easy; you just need to override the -draw method. I’ve made it even easier by providing NoodleCustomImageRep which takes a block, allowing you to create images without creating a new subclass.

Say, though, you are drawing an image that should scale without pixellation, like drawing a square. Surely, you can just lock focus on an NSImage and draw a square and scale that as needed? Well, Mr Smarty Pants, take a look at the example program. Select “Drawn (lock focus)” in the pop-up.

square

 

Here, you’ll see odd fuzziness around the edges. What’s going on here? It’s not anti-aliasing but instead the graphics context where the image is being drawn has image interpolation turned on. As a result, the AppKit is trying to interpolate the image from it’s original size to a much bigger size.

How do you fix this? You could try turning off image interpolation in the destination graphics context but this isn’t always possible or desirable. The better solution is just like before: use a custom image rep to do the drawing (select “Drawn (custom image rep)” from the pop-up). Since the drawing occurs at drawing time, instead of image creation time, it knows about the context it is drawing into and therefore can provide your drawing code with a context at the correct resolution. The crisp square on the right speaks for itself.

Let’s take another example. Say we want to take an existing image and run a Core Image filter on it. Somehow you have to convert your NSImage into a CIImage. This usually entails a game of connecting up various methods that fit together until you got a CIImage. A common way to do this is:

ciImage = [CIImage imageWithData:[nsImage TIFFRepresentation]];

This dumps the whole image (all of its representations) into TIFF format which is in turn, re-parsed back into data. Now, CIImage is pixel based so it ends up picking one of the representations. It’s not documented which representation is picked since there’s no way to specify the context where it will be drawn so there’s a chance it won’t be the right one. Select “Core Image (TIFF Rep)” in the pop-up and you get something like the image on the left (or maybe you won’t; read on):

coreImage

Now, it doesn’t look too bad here. It took the highest res representation and used that. That said, technically, it’s not correct. The version on the right (select “Core Image (custom rep)”) shows the correct image rep being used. Also, my experience has shown that the representation chosen by the -imageWithData: method can be different on different hardware and that it ignores the size set on the original NSImage so you may not be so lucky depending on what machine your code runs on.

Fortunately, Snow Leopard introduced a new method: -CGImageForProposedRect:context:hints:. As mentioned before, when you draw an NSImage into a context, it will automatically pick the right representation for that context. This method does basically the same thing but without the drawing part. Instead it returns a CGImage which you can then use to create your CIImage:

cgImage = [nsImage CGImageForProposedRect:&rect context:[NSGraphicsContext currentContext] hints:nil];
ciImage = [CIImage imageWithCGImage:cgImage];

Keep in mind, though, that it the image returned might not be the exact size you wanted. This becomes more of an issue when you are combining multiple images together in a CIImage pipeline and you need them all to be the same size. You can adjust for this by using CIImage’s -imageByApplyingTransform:.

In addition to the above method, Snow Leopard introduced -bestRepresentationForRect:context:hints: which does a similar thing but returns an image rep instead. Depending on your needs, you can use one or the other to tap NSImage’s image matching logic.

Finally, a note about performance. NSImage does keep a cache based on the drawing context. This helps for when you repeatedly draw the same image at the same size over and over. If you end up sharing an NSImage across different contexts, you’ll find that you are defeating the cache. For these cases, you should be copying the NSImages. Remember that NSImages are just mediators between image data and drawing context. NSImageReps are the actual image sources and, starting with 10.6, reps like NSBitmapImageRep do copy-on-write making it inexpensive to copy NSImages and their reps.

In the example app, there’s a field which shows the time taken to display the image. You’ll notice that when you resize the window, the cases which use a custom image rep are slower as it has to recache whereas the lockFocus cases don’t since the image is static. If this becomes an issue, you can turn off caching or use a fixed resolution image during a live resize. Another more subtle piece of business is performance when drawing from the cache. If you click the “Redisplay” button, it will cause the image to be displayed again. Since you aren’t changing the size, the cached version can be used. Notice how the versions using the custom image rep are usually a smidgen faster than the lockFocus versions. I suspect what is happening is when you lockFocus on the image, you lock the cache into a specific version and size. As a result, if you are drawing at any other size, it has to scale the cached image every time. With a custom image rep, it’s cached at the exact size so the cache can be used as is.

What are the lessons here?

  1. When defining your image, you should use an NSImageRep subclass and override -draw. If you don’t want to create a whole subclass just to create an image, use my NoodleCustomImageRep (included in NoodleKit) which allows you to pass in a drawing block. Using an image rep gives your drawing code better contextual information than you would get just drawing in a -lockFocus.
  2. If you follow point #1, then you can let NSImage make the decision of which representation to use. Use  one of the drawing methods, -CGImageForProposeRect:... or -bestRepresentationForRect:... and you’ll get the best sized representation for the job. Do not assume, though, that this representation will be the actual size you want. When drawing, it also helps to specify the rect to draw into.
  3. Avoid using -lockFocus. It doesn’t produce the correct image in different contexts and can be destructive in terms of kicking out the other reps in the NSImage. While still ok in specific circumstances, you have to know what you are doing.
  4. If using the same NSImage in different contexts, copy it. In 10.6 onwards, this is an inexpensive operation as bitmap data is copy-on-write. Copying NSImages is is also a good idea in case some decides to use -lockFocus on the image (see #3).

If you have access to it, I highly recommend watching the video for Ken Ferry’s session at WWDC 2009: Session 111: NSImage in Snow Leopard (you may need a developer account to view/download it). Much of this is derived from that presentation and it has even more interesting bits about NSImage than what I’ve presented here.

And in case you missed it above, you can find NoodleKit (which contains NoodleCustomImageRep as well as the example program) here.

Addendum (Apr. 18, 2011):

I received an email from Ken Ferry himself pointing out a couple things. Mainly, that as of 10.6, lock focusing doesn’t draw into the cache anymore. It creates a representation whose context is suited for the main display. It is still good for images which are meant for that context. Also as the NSImage context will maintain any size-to-pixel ratio that is on the main display, it can still be used in this situation should resolution independence come into play. It’s not much different than before in this respect but it’s not as volatile as a cache, which I believe is the key point here.

That said, all the caveats above about using -lockFocus still hold true. It’s not so great for when the image needs to be scaled or if you have representations you want to keep (it does remove all other reps when you lock focus). Also, because the representation is tied to the machine, it’s not very suitable for persisting.

Comment » | Cocoa, Downloads, Icons, OS X, Programming, Quartz, User Interface

Fun with Glue

July 5th, 2010 — 11:41am

In my last post, I introduced a little adapter class called NoodleGlue. It’s a simple class that wraps around a block allowing you to use it as the target and action for APIs that require it. It also optionally can take an extra block that is executed when the glue object is deallocated. This allows you to do any cleanup. In my NSTimer category from last time, I used this cleanup ability to allow the object to unregister from any notifications. Being a category, I couldn’t hook into the -invalidate or -dealloc methods of NSTimer so I created a glue object with a clean up block which I then set as an associated object on the timer. As a result, when the timer was deallocated, so would my glue object and my cleanup block would be triggered.

In more general terms, what does this mean? With my glue object plus the associative references feature, you can inject your code into any object’s deallocation sequence. It’s like being able to be notified if any object is freed. I’ve wrapped this up in an NSObject category with a new method, -addCleanupBlock:, which allows you to add a block to be invoked when the object is deallocated. It’s probably more of a proof of concept than something you’d want to use in production code but you can find it in the latest version of NoodleKit (it’s tacked onto the end of the NoodleGlue class).

Now, this sounds pretty cool on paper but this may be one of those cases of a solution looking for a problem. Not to be deterred, I’ve come up with some contrived cases that might actually be useful.

Add extra cleanup for extra stuff you attached to an object

It’s how I used it in my NSTimer category. You can unregister notifications and invalidate objects which need more than just a -release call.

Zero out references

One of the cool things about weak references in a GC environment is that they nil themselves out when the object is collected. You can simulate this behavior when using retain/release by setting up a cleanup block to nil out your reference to the object in question.

Tracing/debugging deallocations

Sure, you can do this in the debugger or Instruments but in some cases you want to trigger more involved code that you can’t do in gdb. Also, in a multithreaded program where the timing affects whether the bug shows up or not, this allows you to track deallocations without setting a breakpoint in the debugger which might otherwise change the timing.

Test if objects are actually deallocated (unit tests)

Credit goes to Guy English for this one. If you need to do a unit test where you want to test your memory management, you can “do the glue” to test that objects actually get deallocated. The problem here is if it’s not working (i.e. object doesn’t get deallocated), your block won’t get called so any assertions you put in there will just silently not happen. So, have your block set a variable and at the end of the test, do an STAssertTrue() on the variable.

• • •

Keep in mind that there are a couple of important caveats:

  1. You cannot retain the object you are “observing” in the block itself as this will create a retain loop and the observed object (as well as the glue) will never be dealloc’ed. Note that just referencing an object retains it unless you assign the object to a variable declared with __block. This has already been done for you in my code as the cleanup block is sent a non-retaining reference to the object being freed so use that reference instead.
  2. 
    	[self addCleanupBlock:
    		^(id object)
    		{
    			// Do not reference "self"
    			[object doSomething];
    		}];
    
  3. It’s not defined when in the deallocation process the associated object is freed. As a result, you can’t rely on any state of the observed object as it may be the case that all its ivars have already been released. But, you can have the block “capture” variables and the block will retain and subsequently release them. For instance, if you want to log the “name” of an object as it’s deallocated, you can reference the name in the block and it will be retained. But be careful though, if you reference an ivar in the object directly, you end up retaining that object. So, if the observed object is creating the block and you access its ivar in that block, you end up implicitly retaining the observed object, which is in violation of point 1 above. In those cases, assign the ivar to a local variable and use that (in this case, do not use __block in the variable declaration as you want the block to retain the object.
  4. 
    	NSString	*localName = _name;	// _name is an ivar
    	[self addCleanupBlock:
    		^(id object)
    		{
    			// Do not reference _name here as that will implicitly retain self
    			NSLog(@"Object %@ has been deallocated.", localName);
    		}];
    

I suggest reading up on the memory management issues with blocks as described here. Joachim Bengtsson has a great article on blocks (that link goes to the section relevant to this article but I recommend reading the whole thing) and there is always Mike Ash’s reliable series of articles on the subject, which I have consulted many a time myself.

Comment » | Cocoa, Debugging, Downloads, OS X, Programming

Back to top