Noodlings

Recently, I took some time to do some clean up on my Xcode project as it has collected its share of odd scripts and settings over the years. Little did I know that I’d make a couple of discoveries surprising enough that I’d actually bother to blog about it.

Speeding up builds

The first thing I wanted to address was build speed. Over the years, it’s felt like build times have gotten longer. Of course, before we start mucking with things, we needed a way of measuring which leads me to the first tip: you can have Xcode display build times in the main activity display in the titlebar. Just use this default:

After doing that and performing a build, it should look something like this:

I tried tightening things up: getting rid of unnecessary steps, removing redundancies, making sure all script phases had inputs and outputs specified. They had minimal impact. Looking at my build logs in Xcode (which helpfully show the time for each step), I noticed that code signing was taking up a significant amount of time. Around 3-8 seconds each occurrence, even for standalone binaries. I have quite a few of these as well as frameworks so that time adds up. Note that for debug builds, I had the “—timestamp=none” option set so that was not the culprit in this case.

Poking around, I came across this thread. While I didn’t have the main problem described there of duplicate certificates, buried in that thread was the following advice: trim ~/Library/Preferences/com.apple.security.plist 

Opening that file up revealed that I had several entries, all except one pointing to non-existent files with the one valid entry pointing to my login keychain. After removing the invalid entries, code signing only took up to 1 second, max. This shaved 40-60 seconds off of my full release builds and 10 seconds off of incremental ones. Huge savings.

Cleaning out frameworks

Another thing I noticed in the cleanup was that some of my frameworks were being copied without their headers. I had a script of my own to remove the headers after copying for deployment builds, but the frameworks remained header-free even when I disabled/removed this script. To make things even stranger, this was was only happening for one target as another target would always have the headers in included frameworks. I checked all sorts of settings but it was only after asking in a Slack channel did I find the answer. Thanks to Nicholas Riley, who didn’t miss a beat and pointed me to this thread.

Apparently, there’s a hidden setting in your project.pbxproj file for copying frameworks where you can specify whether headers get copied over. This is not exposed anywhere in Xcode’s UI, as far as I can tell. It’s also a mystery why it gets set on some targets and not others. The only way to enable/disable this is to edit the project.pbxproj by hand. 

The setting looks something like this:

The flag in question is RemoveHeadersOnCopy. It appears the setting sticks even if Xcode writes to the file after your edit. Of course, if you add new frameworks to copy, you may need to edit the file again.

Hopefully the above helps you out with your builds. If you have any eye-opening tips to make your build process quicker/better, comment below as I’d love to hear them.

Apple announced numerous changes in Catalina in WWDC but there’s one that they failed to mention. System Preferences has had a major change: preference panes now load in a separate process. Apple ones get their own while all the third party ones get stuck in a process called “legacyLoader (System Preferences)”.

It’s becoming more common nowadays to split up programs into separate processes to isolate them from each other, usually done in the name of stability and security. The upshot of this to me is that all display and events happen via IPC between my pane and System Preferences. If done right, this should all be seamless and I shouldn’t notice a thing:

Of course, it’s never that simple.

First off, dark mode is broken. You can test this out by creating a blank preference pane in Xcode and loading that. For you Apple folks, that’s FB7076716. Interesting side note: problem doesn’t manifest itself if SIP is disabled.

The other problem is that magenta. What is that? Apparently, it’s a security feature. Any windows besides the main preference window will have their transparent areas colored magenta. I’ve been trying to pin down the rules as to when this occurs but it’s been slow going and also the screenshot above contradicts what I’ve been told. Note that my software is notarized so it’s not like this is only applying to unknown software.

To top this all off: none of this has been announced or documented. I’ve only found out about this through backchannels and then later, via a “conversation” via Feedback Assistant (FB6758586). It’s a big change and requires some major overhauls on my part. Aside from leaving things as they are, my options are to either rework my whole UI to avoid using any extra windows or convert my preference pane to an app. The latter is a long term goal but neither of those are realistic for doing over the summer between WWDC and Catalina’s launch. And mind you, that’s assuming they actually announced and documented this stuff up front (which they didn’t). How Hazel is going to look on Catalina is very much up in the air right now.

Maybe this will help other people writing pref panes as info on this is nowhere to be found (if I’m wrong on this, please send a link my way as I dying for any clarification on this). I don’t know how much of this Apple will fix as a good part of it is meant as a feature but one can hope that it will somehow sort itself out in the end.

In my frameworks, I have extra programs that my framework code needs to use. Before, I would have these auxiliary programs outside the framework bundle and would set a variable in the framework at runtime to point to the location. Now, having everything self-contained in the framework bundle makes things a bit nicer.

This approach is not without it’s little gotchas, which I’ll go through here.

The basics are straightforward enough. In Xcode, for your framework target, add the binary to the target dependencies and add a Copy File build phase which copies it to the Executables location.

With a normal app bundle, these binaries are put in the MacOS subdirectory. In your code, you can use -[NSBundle pathForAuxiliaryExecutable:] which knows to look in that directory.

With frameworks, it’s not as simple. First off, there is a Versions directory with (potentially) various versions underneath. There is a symlink to the current version as well as symlinks at the top level of the framework directory. These symlinks point to the individual directories and files in the framework, including a symlink to the main framework image. Binary images are not put in their own subdirectory as they are with app bundles.

Any other executables aside from the framework binary are not automatically symlinked. On top of that, NSBundle can’t find them unless they are at the top level. So, in addition to the above steps, you need to add a Run Script build phase which symlinks your extra binaries to the top level. Make sure to use the -h flag (in addition to -s) to ln so as to not resolve the Current link.

Now, suppose your executable links against another framework in your app? For example, let’s take the following situation:

In this case, let’s say someprogram links against the Bar framework. We’ll need to specify an @rpath but what isn’t clear here is what @loader_path is (for an overview of @rpath and @loader_path, check out this post).

It would make sense that @loader_path is the top level of the framework. After all, NSBundle will return the top level symlink to your binary which is what you actually run in your code. In such a case, the rpath would be @loader_path/../ (we want to go one level up from someprogram so that we can see Bar.framework).

Unfortunately, that isn’t the case. Even though the resulting crash report you would get from doing the above will show the path to the symlinked binary, the loader seems to resolve the link, and then set @loader_path to that. So, in this case, it is referring to the original binary in Versions/Current/. Taking this into account, the correct rpath would be @loader_path/../../../

Not a huge deal but hopefully this will save someone from some confusion. As usual, it’s possible I’m missing out on something here, in which case, please comment below on how wrong I am.

Now that the dust has mostly settled, I thought I’d follow up on my previous post concerning Mojave and its new privacy protections.

When Mojave was first released, there were complaints about Hazel not working. In these cases, Hazel was in the privacy list, but not checked. Users claimed to have not seen any sort of prompt. It’s hard to say if the prompt was just missed somehow or if there was a bug where the prompt was not showing up.

Months later, the issues stopped (or at least people stopped reporting them). I can’t say whether some sort of bug was fixed or if people are just now more familiar with how all the new privacy protections work. I’m leaning towards a bug of some sort as the reports have stopped altogether; I’d expect the occasional report of confusion if it was just a knowledge thing.

Regarding unit tests, it seems that changing my unit tests to run within an app, and adding the appropriate usage strings to the test app, was enough to get them to run. There is an issue of including Appkit in unit tests that are testing Appkit-independent functionality, possibly in resulting in subtle false positives in some tests. I suspect those cases will be very rare and, frankly, I’m just glad I can actually run my tests on Mojave now.

One issue that remains is filesystem level protections. With Mojave, certain directories (which I believe still aren’t documented) are protected, not by permissions, but a separate system tied in with SIP. The problem with this is lack of transparency both to us devs as well as users. I’ve had users waste a bunch of time trying to fix the issue by tweaking permissions and ACLs, not realizing that there is another system in place that is invisible to them. Very few users were able to suss out that they needed to add Hazel to the “Full Disk Access” list (instructions on how to do that can be found here).

Overall, it’s confusing since there’s little to no indication in the OS. Finder can view these folders fine whereas other Apple apps (like Terminal) can’t. The red badge on the folder icon looks the same as one for a folder protected by UNIX permissions.

At least, there seems to be one way as a dev to be able to make sense of things. Accessing protected directories (for instance, by using access() ) will fail with an EPERM error. This differs from accessing a directory protected by UNIX permissions. In those cases, you will get an EACCESS error. While that’s great for differentiating between the two cases what’s unclear to me is if there are other situations, outside of Mojave privacy protections, that would give me an EPERM error. Nonetheless, in the latest release, I’ve tried to detect this in Hazel and put up an alert. It’s a fairly uncommon case so I suspect most users will not run into this.

By now, most of the kinks have been worked out but it would be nice if Apple were to document this stuff. The path getting to this point was far more rocky than it should have been.

This morning I released a beta of Hazel with a good deal of trepidation. If you’ve been following the dev blogosphere, you should know that the AEpocalpyse is coming. For those not in the know, Mojave introduces privacy protections for access to AppleEvents, prompting the user whenever an app tries to send AppleEvents to another app.

Here’s some reading to get you up to speed:

Felix Schwarz: macOS Mojave gets new APIs around AppleEvent sandboxing – but AEpocalypse still looms
Daniel Jalkut: Apple Events Usage Description
Dave Nanian: macOS Mojave: Opening New Vistas in Security for Mac Users
Paulo Andrade: Apple Events And The State Of macOS Automation

The result of all this is that we have a not-very-well-thought-out and buggy feature which was added late in the beta with no documentation. The last part makes it particularly hard as we have no way of knowing if the current behavior is a bug or by design, nor whether there will be any more changes or fixes before Mojave ships.

As it stands right now, I am faced with the following problems:

• I cannot run a number of my automated tests.
  My tests test functionality that uses AppleEvents or the tests use AppleScripts themselves to verify test results. Adding the usage description keys to the Info.plists of the xctest bundles does not work. Also, it’s unclear which app the keys should be added to as it’s not documented whether a helper app, or its enclosing app needs to include them. On top of that, I don’t own the running app in this case so changing those keys will break the code signature. If anyone has a workaround for this, I’m all ears.
• If the user is away when the prompt comes up, the request can time out and it will silently default to denying authorization thereafter.
  
  As pointed out by Dave Nanian in his post above, users may never see the prompt to provide access and later find that key functions are not working. To fix it requires using the commandline (provided the user knows what’s going on). Update: This is fixed in beta 10.
• In some cases, users may be prompted twice for the same feature within Hazel.
  For example, when using the “Import into Photos” action, users will be prompted in the UI as Hazel needs to fetch the list of Photos albums via the MediaLibrary API. Then to do the import itself, Hazel has to use AppleEvents, which brings up another prompt. Guarding AppleEvents is a bit orthogonal to the other access restrictions as it is focusing on an access mechanism and not the information being accessed, resulting in redundancies like this.
• Running user scripts is problematic
  Since Hazel allows users to run their own scripts, I may end up having to specify usage strings for everything, in the off-chance a user’s script accesses that area of the OS.
• AEDeterminePermissionToAutomateTarget doesn’t work very well
  Felix’s post outlines this function. It has a critical bug making it near useless for me. On top of that, it requires the app you want to access to be running to determine permissions which leads to a less than ideal user experience.

In short, unless Apple fixes this soon, or even better, rolls it back to be released later, things are going to be a bit bumpy in Mojave. As a user, you may be surprised how often AppleEvents come up in the apps that you use.

The Hazel beta is out (forum login required) so if you are on Mojave (or even if you’re not; I need to make sure I didn’t break things on previous OS versions), give it a try. Reports from the field will help greatly. My options may be limited given the scant documentation and tools provided by Apple but hopefully something can be done.

Update (Sept 13): Beta 11 is out. It fixes a couple of issues I’ve found:

• It doesn’t seem to prompt anymore when using the MediaLibrary API so the “double-prompting” issue is no longer a concern. I was wrong on this. I thought I had reset permissions when doing the test but trying it again, it seems I didn’t. So, the double-prompt is still a thing.
• If the user denies access, the app now appears in the SysPrefs Privacy settings, unchecked. This is huge as the user can now enable access without having to resort to tccutil

tccutil itself is still broken in that it does not work when specifying a bundle identifier. Makes it such that if you want to reset permissions, you have to do it for all apps, not just for the one you care about. This results in you getting re-prompted all over again. The fix to the Privacy settings above does alleviate much of the need for this but this still indicates a lack of thoroughness when implementing this new system.

Let’s talk about APIs. Like UIs, APIs provide an interface (hence the “I”) to a class of users. In this case, the users are programmers. Because of this, many times you have APIs that are cluttered and overly difficult to use since programmers can probably figure that stuff out. It’s unfortunate as many just see APIs as a way to expose functionality without any regard for design.

I feel a hallmark of a good UI is one that avoids presenting the user with unnecessary decisions. The more you can either filter or figure out on the user’s behalf, the better. I’ve touched upon this before: 1 2

I feel the same holds true for APIs. APIs should do more than just provide functionality. They should strive to do more for the user without burdening them with unnecessary details. Provide power with little effort on the user’s part. Let the programmer worry about the “what” and not about the “how”.

This leads me to my favorite API: NSArray

Yes, NSArray.

What is an NSArray? It’s an ordered collection of objects. Doesn’t sound terribly exciting but ask yourself this: what data structure does it implement? If you ask around, you’ll get a bunch of different answers but the real answer here is “It doesn’t matter.”

Now, you may be thinking that NSArray is insufficient to cover all the different cases where you’d need an ordered list. Surely, you have to use a skip list in this one case or a circular queue in this other one. Before going further, I highly recommend reading this oldie but goodie (and one of my favorite blog posts): Array.

As it turns out, NSArray is adaptive. It’s not implementing a single data structure but instead changes depending on usage. Every time I, or a colleague, has implemented their own special data structure, it has ended up performing no better, or sometimes even worse, than NSArray. Now, I’m sure you can come up with some special set of circumstances where you’d need to roll your own, but from my experiences with Cocoa over the years, I’d say that for 99% of the cases, NSArray is fine.

Another example of adaptive behavior is Introsort, which will use either quicksort or heapsort depending on the circumstances. Quicksort has great average performance but has a horrible worst case (O(n²)). Introsort is able to detect when things are headed towards that worst case and switches to heap sort instead (worst case: O(n log n)). No need for the user to evaluate the data to figure out which sort algorithm to use.

Now, sometimes things are inherently complex (threading comes to mind). It does take a bit of care and experience to make sure that instead of making something simple, you end up making it simplistic. Nonetheless, I feel that NSArray hits that sweet spot of providing what you need while at the same time cutting out what you really don’t.

Hopefully this shows that APIs can be so much more than a simple wrapper around some code. The use of proper abstraction as well as going the extra mile on the user’s behalf can go a long way towards making a merely functional API something wonderful.

Things have settled down with my 4.1 release so I thought I’d tend to my neglected blog.

Today, I’m going to talk about getting info about your iTunes library, playlists and albums.

Now, the obvious way to do this would be through AppleScript/AppleEvents. The problem with this approach is that it requires iTunes to be running for it to work. To actually change stuff in your iTunes library, you will have to use AppleScript but for the purposes of this article, we are just looking at ways to get information about it.

The old way to do this was to parse the iTunes Library.xml file (located in ~/Music/iTunes). This file contains a dump of everything you need. Unfortunately, recent versions of iTunes do not generate this automatically; you have to enable it in its preferences (it’s in the “Advanced” tab).

Now, the notion of having to parse an XML file does sound painful but luckily it’s actually a property list so you can load it that way and access everything as NSArrays, NSDictionaries, NSStrings, etc.

With OS X 10.9 (yes, I know it’s macOS now but back then it was still OS X), Apple introduced the MediaLibrary API (ML). This provided a way of not just accessing the iTunes library, but the libraries of all of Apple’s other media apps (Photos, GarageBand, even Final Cut). No parsing involved but it is a fully asynchronous API so expect to deal with a lot of callbacks.

The last one (I don’t know exactly when it was released but it was the last one I discovered) is iTunesLibrary (ITLib) and it’s a bit easy to miss because, from what I can tell, it is not a part of the OS, but iTunes itself. You can find the framework under /Library/Frameworks (not /System/Library/Frameworks). It provides similar functionality to the MediaLibrary framework, but specific to iTunes.

With so many options, which do you use? For the most part, they return the same information though it depends on the version of iTunes. With some versions, one API may return some special playlist that the others don’t. Also, ML is asynchronous while the others are synchronous. I don’t see that as a big deal either way as you can convert one into the other with a tiny bit of work.

The big differentiating factor is performance. Here are the results for getting a list of playlists for my library:

XML: 1.664939s
ML: 18.792855s
ITLib: 0.844415

I don’t think my library is the biggest in the world but it is significant (about 2000 albums). As you can see, ML is pretty damn slow with ITLib being much, much, much faster. What is surprising to me is how much faster parsing the XML file is than ML. It is possible, though, that I’m holding it wrong.

So, ITLib is my recommended API for this. It’s what Hazel uses starting with 4.1 (previously, it was parsing the XML when it was available and falling back to ML). That said, there is one wrinkle.

The persistent IDs of entities in iTunes are hex strings. In the XML file, they are zero padded. In ML, though it returns strings, they are not zero padded (rdar://26624642 for you Apple folks watching from home). To add even more confusion to the mix, they are NSNumbers in ITLib.

If you are using any of these APIs in isolation then it’s probably not a big deal, but if you want to later manipulate that entity using AppleScript (like importing files into a playlist), you will need to provide the zero-padded hex string. Something to keep in mind.

I’ve kept my hat out of the discussions of Swift vs. Objective-C for several reasons, but the main one is that frankly, I’ve been busy. I ship a product, one that I would consider successful, and recently shipped a major release. What does that mean? It means that debating these types of issues and playing with the latest thing is a luxury. Because I have a shipping product with a lot of users, I can’t just change things unless I find that it adds value to my users. My job is to solve my users’ problems. Now that I have a little bit of time to do so, this is one lens I’m hoping to add to this discussion

In addition, years ago, when Java was introduced, I jumped on that bandwagon, leaving behind Objective-C. It started out writing replacements for Foundation and AppKit in Java and ended with writing large scale servers. I was on that wagon for a good 10 years before returning, so I wasn’t a tourist there. This is yet another lens I also want to add to the discussion as history seems to be repeating itself and many younger people seem to be missing out on this part.

But let’s start with the current issue about dynamism since that’s the hot topic.

One thing many people seem to overlook about the dynamism of Objective-C is that it enabled NeXT (and Apple) to provide better GUI tools. Using dynamism, they were able to make GUI building declarative in nature. Connect this to that. Call this method. All stored in a file that was (and still is) data, not code. Competitors at the time (and today) resorted to code generation which is fragile and, ironically, unsafe. Yes, you could have a more declarative file format, but implementing that in using a static language required a lot of hard-coding and switch statements. Not the elegance that many people claim to be moving towards.

I’m not saying that a language has to be purely dynamic but it shouldn’t be purely static either. It think it’s spurious not to credit a level of dynamism for the quality of apps on Apple platforms over the years, and to be pedantic, the NeXT ones as well – many of which were considered the best on any platform at the time. To deny that, I feel, shows a lack of understanding of what has made the platform great all these years.

Moving from a dynamic language to a more static one seems refreshing at first. I went through the exact same thing when switching to Java. Yes, I can hear some of you grumbling “Java” out there but the irony is that there are more similarities in the evolution between Java and Swift than you may want to admit.

Everything seemed so much “tighter” in Java. Interfaces (Java’s version of protocols) everywhere! No more pointers! Checked exceptions! It’s all so new and exciting. You might just have to take my word on this, but any new language that you really like will always seem to make you “more productive”. A colleague and I (plus maybe one, or one and a half, devs who were on the periphery) implemented our own version of Foundation and AppKit in what I would consider a pretty short amount of time. Other people at Lighthouse can correct me on this but my recollection was that it was on the scale of months, not years. You can read up more on it here.

Was it the language? At the time we might say yes but in hindsight, I’d say it was just that we were just super excited and into the project. I was working with an exceptional team and being a new platform, we were excited about making our mark on it. Yeah, there were bits about Java that were better than Objective-C. But there were bits that were worse. The point here, though, is that until the honeymoon is over and you are maintaining a project over several years, it’s hard to make real proclamations about productivity. Truth be told, I’d say the biggest impact on software quality and productivity is the quality of engineers regardless of any other factors.

Looking back on my years with Java, I find the biggest problems with the platform was this idea that everything can be done by the compiler. It sounds great on paper but when implemented, it resulted in all sorts of contortions to work around not letting any dynamism in. It resulted in a lot of glue being written that I wouldn’t have to write in a more dynamic platform. It resulted in cascading changes across code to make trivial modifications just to appease the compiler. Maybe in a future post I’ll go into specific language features that I found problematic and show parallel trends/features in Swift but let’s just say that less is more. It’s an aesthetic that I always felt that put Apple above the rest and something which I feel like in some ways is being abandoned.

Now, I’m sure many of you will just paint me as an old, stubborn Obj-C luddite. And it’s true on some level but that doesn’t mean you should discount my experience. Many of us old fogeys are working on products that have shipped for years. That means that (1) we are solving user problems (2) have immense experience with doing app development (3) really understanding the frameworks (4) have been maintaining a code base.

Let’s go through these points:

1. We are solving user problems.

We have real problems to solve for people outside of ourselves. We need to be practical to survive. Users don’t care about the tech underneath. We have to ship them solutions. This oftentimes requires rapid response. This also oftentimes involves working around Apple bugs, bugs that never get fixed and will only grow in number as Apple maintains their yearly release schedule (something to think about for proponents of the “final” keyword). Agility and flexibility are paramount here. Reality is not pretty. The user could care less how beautiful your code is if you aren’t solving their problems.

2. We have immense experience with doing app development

I want to emphasize the “app” part of that. A common argument for very strict type safety are for cases which aren’t apps. The predominant software in the Apple ecosystem is end user applications. Let’s try and not lose focus here and argue the case for categories of software that aren’t Apple’s forte. If you are writing some life-critical software on an Apple platform, god help you. Even with Swift, you are stuck with Xcode and Apple’s insistence on yearly new OS releases.

3. We really understand the frameworks

I would take many of the arguments for Swift more seriously if there weren’t strawman examples set up on the Obj-C side. You’ll have to dig up Brent Simmons’ posts on this, but when I see the kooky examples of horrible old Obj-C way, what it tells me is more that you don’t understand the frameworks. It resembles informercials where people are fumbling opening a jar and then are presented with some useless gadget to solve a non-existent problem. I’m sure things can be done better than they are now but providing bad examples indicating ignorance of what it is you are trying to replace doesn’t help your argument. And I have seen plenty of examples of new people replacing what they don’t understand. Please don’t be one of those people.

4. We have been maintaining a code base for a while

I have been writing Hazel for ten years. Many of us old-time app developers are in similar positions. Speaking with many other devs, most of whom write iOS apps, I’d be lucky to find those that have maintained a code base for a third of that time. Sure, there are parts of the code that are ugly, but that happens on any platform. Same thing happened on Java. No matter what you do, time is always against you. When I was working with Java, I found the technical debt grew in a different fashion. It was hard to change anything. This was because changes in stricter languages tended to cascade. Because everything has to declare everything about itself, changing code turns into a Facebook feed, except where you and everyone else are obligated to respond to every stupid pic and forward of Snopes-bait. With a more dynamic code base, I can say I have less code to maintain and that I can be more agile. Not that it’s without it’s own problems. But let’s not act like type-safety is a panacea.

And to give more of my personal perspective on the language debate, I’d love to have something better than Objective-C. But I’d also like to have something better than Swift. My personal pet peeve is that we are in the 21st century and we still have to give a shit about the storage type of numbers. I’d be happy to get rid of scalar primitives entirely.

Ultimately, though, I need something that solves my problems. My problems aren’t performance. They aren’t type-safety (maybe it’s just me, but I rarely have issues with it and when I do, the time it takes to fix it is far less than the time specifying everything to a tee everywhere else). They aren’t being able to write clever VDLs. For me, it’s writing apps that solve my users’ problems and getting them out in a timely fashion. As it stands now, Swift (at least pure-Swift, or even current Swift as a non-ABI-stable moving target) does not do that for me.

Now maybe these same problems can be solved in a static way but what I’m not seeing from the static-camp are (decent) solutions. What I’m seeing are either hand-waving or the same crufty code-generation, write tons of repetitive boilerplate type of solutions that I had hoped we had left behind in the 90s.

What I do see makes me worry that it’s not the experienced app-writers that are being heard. Unfortunately, many of us are too busy to sit in the various mailing lists and forums. Yes, ok, we lose. But we all lose if those creating the platform don’t draw from the experience of those who have built upon it successfully. As someone who has done both (write frameworks and apps) and seen what happens when the former ignore the latter (the rise and fall of Java), I’m hoping history doesn’t repeat itself.

That’s my somewhat meandering brain dump for the moment as I do have to get back to working on my next patch. Sure, these are just my opinions but my hope is that we can at least look for more balance in our approaches to solving actual problems.

In case you haven’t been keeping up with the Twitter feed or the forums, Apple introduced a glaring bug in Gatekeeper in 10.11.4. Any non-app bundles (that includes preference panes, screensavers, plugins, etc.) are rejected by Gatekeeper regardless of whether they are signed or not. For those of us that ship such bundles, this is what I’d consider a big deal.

The workaround itself isn’t so bad: either right-click and select “Open” or drag the pref pane into System Preferences. For users curious enough to email support, it’s not such a bad thing as I can suggest one of the workarounds. The problem it gives new users installing it for the first time a bad impression of the software, not realizing that it’s actually Apple’s fault. I can’t really measure this type of thing but I think it’s safe to say a good number of them just nope out of there without contacting me. The result is that my software has an association with being untrusted.

Daniel Jalkut gives a more detailed account of the mechanics of it here so I won’t go retread that ground.

After filing a DTS incident, Apple confirmed it as a bug (I’ve filed rdar://25466753). There was no information about when a fix would be made available and given that Apple is not known for being nimble about these things and that I was losing customers, I followed the advice of some colleagues and took matters into my own hands.

The result is that starting with Hazel 3.3.8, it will ship with an installer app. The installer app still goes through System Preferences as I still think that its installation process works well. Note that an installer package was also an option but I couldn’t figure out if/how to make it use a previous install location if the software was already installed (If anyone knows, I’d be happy to hear about it just in case I have to resort to it in the future). Hopefully now I can direct my efforts back to getting Hazel 4 shipped.

One can debate how much Apple cares about non-app-bundled software but when the workaround is to suggest people bypass Gatekeeper, they should be very concerned. False positives only erode the confidence people have in your security systems and you don’t want them to get in the habit of casually bypassing them.

I’ve had it on my list to convert all my NSCoding classes to use NSSecureCoding and this recent post reminded me so I figured now is as good a time as ever.

How do you go about using secure coding? Here are the Apple docs. Unfortunately, those docs don’t give the whole picture. There are a few details you need to be aware of.

When you subclass an object that supports NSSecureCoding, if you override -initWithCoder, you must also implement +supportsSecureCoding

Even if you don’t call -decodeObject:… in your -initWithCoder: you still have to implement +supportsSecureCoding  and return YES in your class, even if a superclass already did it.

When decoding collections, you must specify the classes of the objects it contains

This is actually documented elsewhere in Apple’s XPC docs. Suppose you have an NSArray. You can’t just say -decodeObjectOfClass:[NSArray class]…. That’s because when NSArray unarchives itself, it can’t specify the class of the objects within it, since they are typed as id  Therefore, you need to specify the classes of those objects for it. In this case, you would use -decodeObjectOfClasses:…, specifying the classes contained by the array as well as NSArray itself.

Some objects will be disabled after secure decoding

Objects like NSPredicate and NSSortDescriptor can take in key paths or selectors making them potentially unsafe. As a result, they are disabled after being securely decoded. To re-enable them, you have to call -allowEvaluation (presumably after doing some sort of check).

You have to turn on secure coding on the coder instances

Sounds a bit obvious except that most of the time you are using the class, not instance, methods to archive/unarchive. Those do not use secure coding nor are there similar class methods for secure coding. Instead, you’ll have to create an instance, set it to secure coding and encode/decode the root object yourself. Of course, you can bundle this up into convenience methods and stick it into a category like so:

@implementation NSKeyedArchiver (NoodleExtensions)

+ (NSData *)noodleSecurelyArchivedDataWithRootObject:(id)rootObject;

{

    NSMutableData   *data;

    NSKeyedArchiver *archiver;

    data = [NSMutableData data];

    archiver = [[NSKeyedArchiver alloc] initForWritingWithMutableData:data];

    [archiver setRequiresSecureCoding:YES];

    [archiver encodeObject:rootObject forKey:NSKeyedArchiveRootObjectKey];

    [archiver finishEncoding];

    return data;

}

@end

@implementation NSKeyedUnarchiver (NoodleExtensions)

+ (id)noodleSecurelyUnarchiveObjectOfClass:(Class)classObject withData:(NSData *)data

{

    NSKeyedUnarchiver       *unarchiver;

    unarchiver = [[NSKeyedUnarchiver alloc] initForReadingWithData:data];

    [unarchiver setRequiresSecureCoding:YES];

    return [unarchiver decodeObjectOfClass:classObject forKey:NSKeyedArchiveRootObjectKey];

}

+ (id)noodleSecurelyUnarchiveObjectOfClasses:(NSSet *)classes withData:(NSData *)data

{

    NSKeyedUnarchiver       *unarchiver;

    unarchiver = [[NSKeyedUnarchiver alloc] initForReadingWithData:data];

    [unarchiver setRequiresSecureCoding:YES];

    return [unarchiver decodeObjectOfClasses:classes forKey:NSKeyedArchiveRootObjectKey];

}

@end

Hopefully this will make your conversion to NSSecureCoding go a little smoother.