The Bundle Blame Game

October 20th, 2022 — 12:28pm

macOS Catalina introduced extra privacy protections where the user can control what apps can or cannot do. Key to this is providing the user with a correct identification of the app in question.

For many apps, this is straightforward. If the “Foo” executable, which is located in …Foo.app/Contents/MacOS, does something that requires permission, the system knows to present Foo.app as the responsible bundle. For many apps, though, it’s slightly less straightforward. There may be multiple executables in the app bundle. Nonetheless, it shouldn’t be much of a problem figuring out the responsible app bundle but for some reason, a consistent solution has eluded Apple.

One of the main permissions that Hazel may require is Full Disk Access. If you go to that section in System Preferences/Settings, you’ll see a list of apps. You think adding and enabling your app there is all that is needed; that it should handle permissions for that app and all its subsidiary executables in its bundle. Here’s how it normally looks for Hazel.

But sometimes things go awry. All of sudden, the system requires separate permissions for the other executables in your bundle. Users will find the app not working properly and they won’t think to look at the permissions again as they thought they already given the proper permissions. Another trip to the privacy settings may show it listing the helper executables there.

Now, what the hell is that godforsaken app name? Well, it’s the team id followed by the bundle id. Unfortunately, Apple requires the crazy naming scheme for apps that are login items bundled in the main app. Is this something the user really needs to be exposed to? Minimally, Apple should observe the display name specified by the bundle but in the ultimately, this should not appear there at all.

The user should not have to understand the different running components internal to an app. This is a level of “transparency” that muddles the issue and adds nothing of value. Does the user really need to know that “Foo Helper” exists and want to really control that separately from the main app? I know for a fact that the vast majority of Hazel users have no idea how the different executables in Hazel work and interact with each other. I would not expet them to be able to judge whether it’s a good idea to give permissions to one, but not the other.

At the other end of the spectrum, other parts of the system sometimes fail to list an app at all. For instance, Hazel sends notifications via a commandline program in one of the standard locations within the bundle. With Monterey, while Hazel was properly being recognized as the responsible app in the Notification settings, it would be absent in the list of app you choose for the Focus feature. This was remedied in a Monterey point release with Hazel properly listed in both places there.

Unfortunately, Ventura broke it again and now it’s absent from both locations .

This all seems to indicate that each team working on these different bits are re-inventing the same wheel over and over and doing it incorrectly in different ways. You’d think it would be in their best interest of have someone write the canonical algorithm for this and use it across all these subsystems.

Unfortunately, for me, this has resulted in various DTS incidents and bug reports in recent years. As of this writing, it’s still a problem in Ventura (last checked on the release candidate). Apple, please get this sorted. FB11462910 (which also mentions several other related FBs) for those working at Fruit Company.

There is an unrelated major issue which I am hoping has been worked around in the current Hazel beta. If you are interested in testing, please see the forum article here (forum login required).

For the moment, if Hazel is an important part of your work, I suggest holding off on upgrading to Ventura. I’ll keep you posted on the status of these bugs via Twitter.

Comment » | Uncategorized

Hazel’s 15th 16th Anniversary

September 6th, 2022 — 9:05am

It’s been 16 years ago today since Hazel’s first release. Why celebrate the 16th anniversary? Because I thought it was actually 15 years but got the dates wrong. So, to mark the 16th anniversary (or a belated 15th, if you prefer), Hazel is 30% off. That applies to all license types (single, family, and upgrade). Just go to the store. Sale ends at 3am EDT.

Chances are if you are reading this, then you already own a copy so share the news with friends and family. Just don’t tell Zach in Accounting. He knows what he did.

While odd bits have changed since the 10th anniversary, things remain mostly the same. After all these years, still not in the App Store. No plans to shift to a subscription model. I’ve resisted many of the “innovations” in software sales and distribution in recent years. I could probably write another post why but in short, I haven’t found any need to change something that works.

Can’t say whether I’ll still be here doing this for Hazel’s 20th anniversary but I hope you’ll forgive me if I forget again and end up being a year or two late.

1 comment » | Business, Hazel, Noodlesoft

Codesigning & Notarization Woes

January 25th, 2021 — 1:34pm

As mentioned in my last post, I had various issues with codesigning and notarization during Hazel 5’s launch. Now before I get into it, remember during all this that Hazel is codesigned and notarized with no warnings or errors. In my release build process, I do a codesign —verify as well as spctl to make sure everything is hunky-dory before submitting to Apple for notarization.

The biggest problem at launch was some users getting an “Unidentified developer” alert when opening the dmg. I had various users send in logs, but it was only when someone found a log message pertaining to the rpath for one of the binaries in the bundle that I was able to identify the problem. Strangely enough, that person didn’t receive the “Unidentified developer” error alert.

Turns out, I had mistakenly added a binary to the top level bundle (it was originally in a framework). The binary had an rpath which escaped the framework bundle as it depended on a sibling framework. This worked fine when the binary was in the framework bundle but when it was at the top level of the main app bundle, it escaped the app bundle itself. Not sure how the binary got added to the Copy Files phase there but it was an easy fix to remove.

Another issue was that the installer wasn’t triggering for some people. For the installer to trigger, Hazel needs to detect that it is running from a disk image. Further investigation turned up that Hazel was being translocated. If you don’t know what translocation is, I suggest reading this article: https://lapcatsoftware.com/articles/app-translocation.html

When translocated, the binary is no longer on the disk image, instead it is copied to a temp location on disk. Hazel is unaware of this and as a result, doesn’t run the installer. Why was Hazel being translocated? I’m still not sure. It’s my understanding that if an app and its containing dmg is signed and notarized, it shouldn’t be translocated. As a workaround, I ended up using the private SecTranslocate.h APIs to detect the translocation and compensate appropriately.

Possibly related to the above, there was also an issue with the embedded helper app not running. Logs from users showed that the quarantine flag was still set on the helper and that was preventing it from being run. When the user copies an app, like say from a disk image to /Applications, the quarantine flag should be cleared for the app and everything inside but for some reason it was not clearing it for the embedded binaries. Note that unlike when a user launches an app from Finder where they will be asked to run the app, a login item helper will fail to launch without any prompt. As a short term workaround, on first run, I had Hazel recursively remove its quarantine flag before it attempted to run the helper.

Even with the workarounds, something didn’t sit right with me. I suspected there was something wrong with my bundle that was causing these issues. I ended up filing a DTS incident to get some feedback from Apple. The DTS engineer was able to reproduce the issue on a VM and dig up a log message indicating that it could not clear the quarantine flag on the Autoupdate helper app in the Sparkle framework. For those that don’t know, Sparkle is a very commonly used framework for apps to update themselves. The Autoupdate helper runs when your app is terminated so that it can install and run the new version.

This all lead to an investigation into where should Autoupdate, or executable binaries in general, be put in a framework. Now, I had looked into this once before . Sparkle originally put Autoupdate in Resources which is incorrect and can cause subtle issues. The solution I came up with back then was to put any executables alongside the framework binary, which would be in Versions/A with a symbolic link at the top level of the framework bundle. This seemed to work at the time. It got through codesigning and notarization checks without issue and ran fine after that.

But it seems that the problem is more subtle than that as whatever code that clears quarantine flags on embedded bundles didn’t like that. I was pointed to this article which suggests I put the program in Contents/Helpers. Despite the insistence that this was the way to do it, it flat out did not work. An important thing to note is that framework bundles do not have a Contents directory. Putting a Contents directory, either in Versions/A or at the top level will make codesign barf, complaining about an invalid bundle format. My guess is that when codesign sees a Contents directory, it considers the bundle to be another (non-framework) type of bundle. After wasting a good bit of time on this, I went with what I thought made more sense, putting it in a Helpers directory (sans Contents). If you have an Xcode “Copy Files” phase copy to Resources for a framework, for instance, it doesn’t put it in Contents/Resources, but in Resources so it made some sense to do it this way.

Sure enough, that worked. Of course, after fixing it, someone pointed out how some of the system frameworks use the Helper directory. I guess I should’ve checked there first. At least I have some confirmation that I was on the right track. Note that you should put the Helper directory under Versions/A. You can symlink to the top level but that’s optional. I did it mainly to give myself a more consistent path to access it in the very unlikely event that I end up shipping different versions of the framework.

Overall, this experience has been very frustrating. There is little to no documentation on all the various things that can go wrong in your bundle that will cause things to fail, and when it does fail, the logging is very inconsistent. Lastly, none of the tools or processes in place (codesign, spctl, notarization) catch these cases. These are all issues related to the static structure of the app bundle so it seems like they should be detectable. Having a tool that developers can run on their bundles before submitting them would be very helpful as the current ones are far from complete in that regard.

After all this, I’m still not sure if the translocation problem is still there or not. I’m not about to remove a workaround just to have it fail for users just so I can get some feedback so that question will remain unanswered for the time being. The lack of definitive and correct information during this whole ordeal is a bit disturbing and makes me think that there’s no one that actually knows for sure how things really work.

For those at Fruit Company, I’ve filed FBAs FB8981011 and FB8981016.

Comment » | Hazel, Software, Tools, Xcode

Hazel 5 Launch Postmortem

December 20th, 2020 — 4:57pm

As promised, I thought I’d write about my launch. While not disastrous, it had its share of bumps. I had hoped that I had learned something from the Hazel 4 launch four years ago. One of the issues was server capacity. This year, I deployed an extra server. It was an asymmetrical setup, with my main setup handling the website and doling out free upgrades to recent purchasers, while the second server handled the store. Unfortunately, it wasn’t enough. The store still got swamped. Given how busy I was handling requests and trying to troubleshoot other issues, I didn’t have time to test and deploy yet another server, plus they would all be hitting the same database so it was unclear if it would help much. Given that this type of load tends to subside within the day, I rode it out.

One thing I could have done to help alleviate this was to spread out sending messages to the mailing list. Before sending anything to the list, traffic was quite manageable. Once the list got blasted, so did my site. I’m not sure if my email campaign provider supports it, but sending messages in chunks or just slowing down the sending rate would have probably minimized the problems.

Then there were packaging issues. Hazel is codesigned and notarized yet on some people’s systems, it would reject Hazel, either wholesale or in parts. This is worth a whole post on its own so expect one later. Suffice it to say, I did fix some of the issues and came up with workarounds for the others.

And finally, there were actual issues with the software.

First was a bug with trial mode expiring soon after install. I had Hazel reset the trial period for people coming from a previous version but it contained a bug which I did not catch. I did have a beta period but the version used then accepted Hazel 4 licenses, which meant that trial mode was not tested.

The other major bug was black backgrounds appearing in some views on 10.13. I take total responsibility for this as I did not test for 10.13. I did have a 10.13 partition on a drive I keep with various macOS versions, but it got nuked by an early Big Sur beta install which went awry. I tried reinstalling but my installer was corrupt. Add to that that Apple doesn’t allow you to download old installers and you can see how this fell by the wayside as other issues came up. It is ultimately my fault and my apologies to those running 10.13. As for the issue, it seems that 10.13 has problems with certain named/system colors when the app is linked against 11.0. Solution was to do special-case code for 10.13 using non-named colors.

There were plenty of other bugs but those were the most apparent and the ones I had to address quickly.

And with all of the above issues, I had to deal with thousands of people reporting them. Especially in the first few days, it was a frantic balancing act of being responsive to users while trying to carve out time to investigate the issues they were reporting. Logic would dictate stopping the bleeding first (i.e. investigate and address the problems) but it’s hard to ignore the huge number of messages piling up. Some would say that having all that attention would be a good problem to have, but when it was happening, it sure didn’t feel like it.

Lessons to be learned:

  • Try and slow down or space out announcements. Having everyone find out at once is asking for trouble. One idea I toyed with before launch but didn’t implement was to have a preview for those on the mailing list. Have a separate store that was available early where they could purchase an upgrade before the release to the public at large. That might have helped with the initial crush.
  • When running a beta test, be mindful of the holes in your testing, including differences between the beta and final product and missing demographics in your pool of beta testers.
  • Keep your priorities straight. Not everything needs to be handled immediately. It’s ok to ignore stuff.
  • Accept that no matter how much you prepare, you are never fully ready for what comes next.

Oddly, I found that the press was noticeably absent. It seems that even though the Mac market keeps growing, there are fewer and fewer outlets reporting and reviewing Mac products. Hazel has enough of a following that it didn’t matter as much but it feels as if things have regressed on that front, which is a bit sad.

Next time, I’ll be talking about my journey into the nightmare world of code signing and notarization. Fun times to be had by all. Until then…

2 comments » | Hazel, Noodlesoft, Software, System Administration

Hazel 5

December 9th, 2020 — 11:40am

I’m almost a month late with this announcement but Hazel 5 is out! I’ve been quite busy since the launch but after releasing 5.0.3 (yes, there have already been three patches) I feel like I have a little breathing room to post even if I’m bit exhausted.

I won’t give a full rundown of what’s new but you can get more details here.

The most obvious thing you’ll probably notice is that Hazel is no longer a preference pane, instead shipping in an app form factor. Why? The main reason is Apple. If you missed my post on this topic, in short, Apple screwed over preference panes in Catalina. The result is a buggy mess and I’ve seen almost no improvement on that front since then. It seemed clear to me that preference panes were a developmental dead end.

I had already been planning on releasing a version 5 so I was prepared to make some major changes. Problem was, I had to make sure I released 5.0 around when Apple’s next OS (now known as Big Sur) would arrive. Why? Because any Big Sur compatibility work put into Hazel 4 would most likely not be usable in version 5. Partly because of the major changes between the two but mostly because many of the issues that affect preference panes since Catalina aren’t a problem for apps. I could either do double the work or cut my losses and do a big push for version 5 in the time available. The downside is that the conversion to an app displaced other features I had planned. Some of these will make it into point releases but the bigger ones may have to wait until 6.0. 

It is nice to be able to have things in an app. There are fewer limitations but at the same time, it changes expectations. One of the major reasons why I had Hazel as a preference pane was that people expected preference panes to be for configuring services that run in the background. Since releasing as an app, I’ve had numerous people ask me to provide a way to hide the dock icon. People have the notion that they need to keep the app running for Hazel to run its rules, not realizing there’s a helper doing all that separately and that they can quit the app at any time. I did add in a little tip that pops up if you activate Hazel after a few minutes of inactivity (which would imply, to me, that they were keeping it running longer than needed) but I may need to be a bit more forceful in educating users.

Besides all the interface improvements, I’ve also added support for tables and lists. This expands the possibilities with text matching, hopefully allowing users to consolidate their rules. This took a good deal of back and forth in the design until I finally settled on what’s there now. There’s a chance these features will change in the future, especially with how list items work, but I’ll try and make sure it is done in a way to preserve compatibility with old rules. That’s always been a concern for me: making sure any changes I make to the rule schema/interface are done in a way such that old rules are expressible in the new paradigm. Except in the rare case where I drop a function entirely, I strive to make sure everything carries over.

One focus this time around was making the install process a bit nicer. Hazel 5 ships as an app on a disk image. The preferred way to install is to run it right off the disk image. Hazel will detect this case and offer to install into Applications (or a folder of your choosing) and relaunch. Or if you prefer, you can drag Hazel to a folder and run it there, in which case, the installer doesn’t kick in. The latter should reassure users that the app is self-contained and doesn’t install stuff in unexpected places. I’ve always felt that when you are first presented with an app, a double-click should run it, ready to do work. This includes any steps needed to get to that point, like installation. That was something that worked very well when Hazel was a preference pane and I wanted that same experience as an app. Having the user drag the app to Applications, navigate to it and then launch it seems like too many steps and I felt I could do better. If you want to do this with your own app, there are frameworks you can use.

Another thing I’ve added is the “Full Disk Access Guide”. For some operations, such as managing your Trash, Hazel requires full disk access. Unfortunately, Apple dropped the ball on this, providing little to no guidance or APIs to detect these cases and having no UI to help the user do it, thus leaving it to individual developers to cook up their own schemes. Previously, I pointed the user to a webpage which outlined the steps of navigating to the correct pane in System Preferences, unlocking it and then dragging the app into the list. For Hazel 5, I now bring up the correct pane in System Preferences and show a window/overlay which attaches itself to System Preferences, showing each step with arrows for guidance. Screenshot included below. And if it looks familiar to you, yes, I stole it from SuperDuper, with permission from its developer Dave Nanian (thanks Dave!).

Pricing is always a tricky issue. It’s always hard coming up with the right balance of something users will pay while also making sure your business is sustainable. But let’s address the elephant in room, or maybe the lack of said elephant. Subscriptions. I never really considered them for two reasons (a) I’m not particularly fond of them and neither are a good number of users and (b) I don’t need them. I’ve been fortunate in being able to keep doing this while charging a price that is fair to me and my users. Could I make more money? Sure, but if you’ve followed this blog for any period of time, you should realize that’s not my main goal.

That all said, I did end up raising prices as a lot of work went into this release. While users have been pretty supportive of the price changes, I can’t please everyone. In my defense, I’ll point out that I’ve been pretty good with providing significant amounts of functionality for free in between major releases (don’t believe me? Check out my release notes). Additionally, my major releases are pretty far apart (about every 4 years). Not that I’m doing that on purpose and I can’t guarantee the next one will happen at the same interval. It just happens to be how I do development. Major release sees major shift in architecture, paradigm or functionality. Minor releases flesh that out until I feel like that has run its course. Rinse, repeat.

And on that note, I hope to get one more release out before the end of the year. With luck, this will include a bonus feature which should make tables even more useful. Also, I’ll try and write up a post-mortem for the launch as it didn’t go quite so smoothly and I’m sure there are lessons to be learned there. And before I forget, thanks to you for your support. It’s been a long journey and it feels great reaching this milestone.

Comment » | Hazel, Noodlesoft, Software

A couple of random Xcode tips to improve your builds

August 24th, 2020 — 11:20am

Recently, I took some time to do some clean up on my Xcode project as it has collected its share of odd scripts and settings over the years. Little did I know that I’d make a couple of discoveries surprising enough that I’d actually bother to blog about it.

Speeding up builds

The first thing I wanted to address was build speed. Over the years, it’s felt like build times have gotten longer. Of course, before we start mucking with things, we needed a way of measuring which leads me to the first tip: you can have Xcode display build times in the main activity display in the titlebar. Just use this default:

defaults write com.apple.dt.Xcode ShowBuildOperationDuration YES

After doing that and performing a build, it should look something like this:


I tried tightening things up: getting rid of unnecessary steps, removing redundancies, making sure all script phases had inputs and outputs specified. They had minimal impact. Looking at my build logs in Xcode (which helpfully show the time for each step), I noticed that code signing was taking up a significant amount of time. Around 3-8 seconds each occurrence, even for standalone binaries. I have quite a few of these as well as frameworks so that time adds up. Note that for debug builds, I had the “—timestamp=none” option set so that was not the culprit in this case.

Poking around, I came across this thread. While I didn’t have the main problem described there of duplicate certificates, buried in that thread was the following advice: trim ~/Library/Preferences/com.apple.security.plist 

Opening that file up revealed that I had several entries, all except one pointing to non-existent files with the one valid entry pointing to my login keychain. After removing the invalid entries, code signing only took up to 1 second, max. This shaved 40-60 seconds off of my full release builds and 10 seconds off of incremental ones. Huge savings.

Cleaning out frameworks

Another thing I noticed in the cleanup was that some of my frameworks were being copied without their headers. I had a script of my own to remove the headers after copying for deployment builds, but the frameworks remained header-free even when I disabled/removed this script. To make things even stranger, this was was only happening for one target as another target would always have the headers in included frameworks. I checked all sorts of settings but it was only after asking in a Slack channel did I find the answer. Thanks to Nicholas Riley, who didn’t miss a beat and pointed me to this thread.

Apparently, there’s a hidden setting in your project.pbxproj file for copying frameworks where you can specify whether headers get copied over. This is not exposed anywhere in Xcode’s UI, as far as I can tell. It’s also a mystery why it gets set on some targets and not others. The only way to enable/disable this is to edit the project.pbxproj by hand. 

The setting looks something like this:

…={isa =PBXBuildFile; fileRef = F19543FC17EC99FB62CA62C8 /* HockeySDK.framework */; settings ={ATTRIBUTES =(CodeSignOnCopy,RemoveHeadersOnCopy,);};};

The flag in question is RemoveHeadersOnCopy. It appears the setting sticks even if Xcode writes to the file after your edit. Of course, if you add new frameworks to copy, you may need to edit the file again.

Hopefully the above helps you out with your builds. If you have any eye-opening tips to make your build process quicker/better, comment below as I’d love to hear them.


Comment » | OS X, Programming, Tools, Xcode

Preference Panes and Catalina

August 28th, 2019 — 1:09pm

Apple announced numerous changes in Catalina in WWDC but there’s one that they failed to mention. System Preferences has had a major change: preference panes now load in a separate process. Apple ones get their own while all the third party ones get stuck in a process called “legacyLoader (System Preferences)”.

It’s becoming more common nowadays to split up programs into separate processes to isolate them from each other, usually done in the name of stability and security. The upshot of this to me is that all display and events happen via IPC between my pane and System Preferences. If done right, this should all be seamless and I shouldn’t notice a thing:

Window2

Of course, it’s never that simple.

First off, dark mode is broken. You can test this out by creating a blank preference pane in Xcode and loading that. For you Apple folks, that’s FB7076716. Interesting side note: problem doesn’t manifest itself if SIP is disabled.

The other problem is that magenta. What is that? Apparently, it’s a security feature. Any windows besides the main preference window will have their transparent areas colored magenta. I’ve been trying to pin down the rules as to when this occurs but it’s been slow going and also the screenshot above contradicts what I’ve been told. Note that my software is notarized so it’s not like this is only applying to unknown software.

To top this all off: none of this has been announced or documented. I’ve only found out about this through backchannels and then later, via a “conversation” via Feedback Assistant (FB6758586). It’s a big change and requires some major overhauls on my part. Aside from leaving things as they are, my options are to either rework my whole UI to avoid using any extra windows or convert my preference pane to an app. The latter is a long term goal but neither of those are realistic for doing over the summer between WWDC and Catalina’s launch. And mind you, that’s assuming they actually announced and documented this stuff up front (which they didn’t). How Hazel is going to look on Catalina is very much up in the air right now.

Maybe this will help other people writing pref panes as info on this is nowhere to be found (if I’m wrong on this, please send a link my way as I dying for any clarification on this). I don’t know how much of this Apple will fix as a good part of it is meant as a feature but one can hope that it will somehow sort itself out in the end.

Comment » | Hazel, OS X, Software, User Interface

Frameworks and Auxiliary Binaries

April 4th, 2019 — 7:09pm

In my frameworks, I have extra programs that my framework code needs to use. Before, I would have these auxiliary programs outside the framework bundle and would set a variable in the framework at runtime to point to the location. Now, having everything self-contained in the framework bundle makes things a bit nicer.

This approach is not without it’s little gotchas, which I’ll go through here.

The basics are straightforward enough. In Xcode, for your framework target, add the binary to the target dependencies and add a Copy File build phase which copies it to the Executables location.

With a normal app bundle, these binaries are put in the MacOS subdirectory. In your code, you can use -[NSBundle pathForAuxiliaryExecutable:] which knows to look in that directory.

With frameworks, it’s not as simple. First off, there is a Versions directory with (potentially) various versions underneath. There is a symlink to the current version as well as symlinks at the top level of the framework directory. These symlinks point to the individual directories and files in the framework, including a symlink to the main framework image. Binary images are not put in their own subdirectory as they are with app bundles.

Any other executables aside from the framework binary are not automatically symlinked. On top of that, NSBundle can’t find them unless they are at the top level. So, in addition to the above steps, you need to add a Run Script build phase which symlinks your extra binaries to the top level. Make sure to use the -h flag (in addition to -s) to ln so as to not resolve the Current link.

Now, suppose your executable links against another framework in your app? For example, let’s take the following situation:

In this case, let’s say someprogram links against the Bar framework. We’ll need to specify an @rpath but what isn’t clear here is what @loader_path is (for an overview of @rpath and @loader_path, check out this post).

It would make sense that @loader_path is the top level of the framework. After all, NSBundle will return the top level symlink to your binary which is what you actually run in your code. In such a case, the rpath would be @loader_path/../ (we want to go one level up from someprogram so that we can see Bar.framework).

Unfortunately, that isn’t the case. Even though the resulting crash report you would get from doing the above will show the path to the symlinked binary, the loader seems to resolve the link, and then set @loader_path to that. So, in this case, it is referring to the original binary in Versions/Current/. Taking this into account, the correct rpath would be @loader_path/../../../

Not a huge deal but hopefully this will save someone from some confusion. As usual, it’s possible I’m missing out on something here, in which case, please comment below on how wrong I am.

Comment » | OS X, Programming, Xcode

AEftermath

January 29th, 2019 — 1:30pm

Now that the dust has mostly settled, I thought I’d follow up on my previous post concerning Mojave and its new privacy protections.

When Mojave was first released, there were complaints about Hazel not working. In these cases, Hazel was in the privacy list, but not checked. Users claimed to have not seen any sort of prompt. It’s hard to say if the prompt was just missed somehow or if there was a bug where the prompt was not showing up.

Months later, the issues stopped (or at least people stopped reporting them). I can’t say whether some sort of bug was fixed or if people are just now more familiar with how all the new privacy protections work. I’m leaning towards a bug of some sort as the reports have stopped altogether; I’d expect the occasional report of confusion if it was just a knowledge thing.

Regarding unit tests, it seems that changing my unit tests to run within an app, and adding the appropriate usage strings to the test app, was enough to get them to run. There is an issue of including Appkit in unit tests that are testing Appkit-independent functionality, possibly in resulting in subtle false positives in some tests. I suspect those cases will be very rare and, frankly, I’m just glad I can actually run my tests on Mojave now.

One issue that remains is filesystem level protections. With Mojave, certain directories (which I believe still aren’t documented) are protected, not by permissions, but a separate system tied in with SIP. The problem with this is lack of transparency both to us devs as well as users. I’ve had users waste a bunch of time trying to fix the issue by tweaking permissions and ACLs, not realizing that there is another system in place that is invisible to them. Very few users were able to suss out that they needed to add Hazel to the “Full Disk Access” list (instructions on how to do that can be found here).

Overall, it’s confusing since there’s little to no indication in the OS. Finder can view these folders fine whereas other Apple apps (like Terminal) can’t. The red badge on the folder icon looks the same as one for a folder protected by UNIX permissions.

At least, there seems to be one way as a dev to be able to make sense of things. Accessing protected directories (for instance, by using access() ) will fail with an EPERM error. This differs from accessing a directory protected by UNIX permissions. In those cases, you will get an EACCESS error. While that’s great for differentiating between the two cases what’s unclear to me is if there are other situations, outside of Mojave privacy protections, that would give me an EPERM error. Nonetheless, in the latest release, I’ve tried to detect this in Hazel and put up an alert. It’s a fairly uncommon case so I suspect most users will not run into this.

By now, most of the kinks have been worked out but it would be nice if Apple were to document this stuff. The path getting to this point was far more rocky than it should have been.

Comment » | Hazel, OS X, Programming

Hazel and the AEpocalypse

September 4th, 2018 — 1:13pm

This morning I released a beta of Hazel with a good deal of trepidation. If you’ve been following the dev blogosphere, you should know that the AEpocalpyse is coming. For those not in the know, Mojave introduces privacy protections for access to AppleEvents, prompting the user whenever an app tries to send AppleEvents to another app.

Here’s some reading to get you up to speed:

Felix Schwarz: macOS Mojave gets new APIs around AppleEvent sandboxing – but AEpocalypse still looms
Daniel Jalkut: Apple Events Usage Description
Dave Nanian: macOS Mojave: Opening New Vistas in Security for Mac Users
Paulo Andrade: Apple Events And The State Of macOS Automation

The result of all this is that we have a not-very-well-thought-out and buggy feature which was added late in the beta with no documentation. The last part makes it particularly hard as we have no way of knowing if the current behavior is a bug or by design, nor whether there will be any more changes or fixes before Mojave ships.

As it stands right now, I am faced with the following problems:

  • I cannot run a number of my automated tests.
    My tests test functionality that uses AppleEvents or the tests use AppleScripts themselves to verify test results. Adding the usage description keys to the Info.plists of the xctest bundles does not work. Also, it’s unclear which app the keys should be added to as it’s not documented whether a helper app, or its enclosing app needs to include them. On top of that, I don’t own the running app in this case so changing those keys will break the code signature. If anyone has a workaround for this, I’m all ears.
  • If the user is away when the prompt comes up, the request can time out and it will silently default to denying authorization thereafter.

    As pointed out by Dave Nanian in his post above, users may never see the prompt to provide access and later find that key functions are not working. To fix it requires using the commandline (provided the user knows what’s going on). Update: This is fixed in beta 10.
  • In some cases, users may be prompted twice for the same feature within Hazel.
    For example, when using the “Import into Photos” action, users will be prompted in the UI as Hazel needs to fetch the list of Photos albums via the MediaLibrary API. Then to do the import itself, Hazel has to use AppleEvents, which brings up another prompt. Guarding AppleEvents is a bit orthogonal to the other access restrictions as it is focusing on an access mechanism and not the information being accessed, resulting in redundancies like this.
  • Running user scripts is problematic
    Since Hazel allows users to run their own scripts, I may end up having to specify usage strings for everything, in the off-chance a user’s script accesses that area of the OS.
  • AEDeterminePermissionToAutomateTarget doesn’t work very well
    Felix’s post outlines this function. It has a critical bug making it near useless for me. On top of that, it requires the app you want to access to be running to determine permissions which leads to a less than ideal user experience.

In short, unless Apple fixes this soon, or even better, rolls it back to be released later, things are going to be a bit bumpy in Mojave. As a user, you may be surprised how often AppleEvents come up in the apps that you use.

The Hazel beta is out (forum login required) so if you are on Mojave (or even if you’re not; I need to make sure I didn’t break things on previous OS versions), give it a try. Reports from the field will help greatly. My options may be limited given the scant documentation and tools provided by Apple but hopefully something can be done.

Update (Sept 13): Beta 11 is out. It fixes a couple of issues I’ve found:

  • It doesn’t seem to prompt anymore when using the MediaLibrary API so the “double-prompting” issue is no longer a concern. I was wrong on this. I thought I had reset permissions when doing the test but trying it again, it seems I didn’t. So, the double-prompt is still a thing.
  • If the user denies access, the app now appears in the SysPrefs Privacy settings, unchecked. This is huge as the user can now enable access without having to resort to tccutil

tccutil itself is still broken in that it does not work when specifying a bundle identifier. Makes it such that if you want to reset permissions, you have to do it for all apps, not just for the one you care about. This results in you getting re-prompted all over again. The fix to the Privacy settings above does alleviate much of the need for this but this still indicates a lack of thoroughness when implementing this new system.

Comment » | Hazel, OS X, Programming

Back to top