Apple's Code-signing Quagmire

My quixotic quest to defeat the scourge of code-signing once and for all

This week, I want to cut straight to the heart of the raison d'être for jDeploy: Apple’s code-signing rules.

In a previous article, I waxed nostalgic about the golden age of Java on the Mac, when you could package your Java app inside a native .app bundle and distribute it to your users with no strings attached. No code-signing required. No notarization. No bulky JVM dependency (because OS X included it standard). You could just build the app and send it to your users, and they could run it, none the wiser that the app was written in Java. Ahhh, those were the days…. before the dark times, before the app store!

First they came for my “bundle size”. In 2011, they deprecated Java on the Mac, forcing me to embed the Java runtime environment into my app bundle, ballooning my app-size from 3 megs, to 70 megs. No big deal, I suppose, as this curse coincided with the blessing of being allowed to sell my apps in the app store.

Next, in 2012, they came for my developer “sovereignty”, requiring me to sign my apps using an Apple developer certificate. As of OS X Lion (10.7.3), users of “unsigned” apps would be met with a scary warning that the app was from an “unidentified developer” the first time they tried to open the app. Initially they could still choose to open it despite the warning, but in later iterations, they restricted this further. First they removed the option to “Run anyways” in the warning dialog. Users had to “Right click” the app icon and choose “Open” in order to see the “Run Anyways” option.

In more recent version or Mac OS, they have added the requirement to “notarize” your apps in addition to code-signing. This process involves sending your app to Apple so they can “inspect it”. If they don’t detect any “malware”, they’ll provide you with a manifest that you can “staple” to your app bundle.

It is my experience that every new release of Mac OS has made it more difficult to build apps that can be distributed to my users. As of this writing (2022), developers must sign and notarize their apps or users won’t be able to run them at all. They’ll get a message saying something to the effect of “This app is damaged. You should move it to the trash”, when they try to run your app for the first time.

I have written several shell scripts, ANT scripts, and more recently, GitHub action workflows to automate the signing and notarization process for my apps, but the goal posts keep moving. With nearly every new Mac release, I find that my automation scripts no longer work. Apple also loves to dictate that you use their latest OS and Xcode tools for performing builds, so I’m always having to adapt my scripts to accommodate them.

So, presently, if I want to create a little utility app in Java that I distribute to my users I need to:

1. Build the app (usually as an executable jar file).

2. Bundle the app as a native package (e.g., .app, .dmg, .pkg). Requires a Mac running a recent version of Mac OS, and a tool like javapackager.

3. Code-sign the native bundle using my apple developer certificate. Requires an Apple developer account (99 USD/year), and almost always requires me to futz with my certificates, revoking old ones, regenerating new ones, re-downloading Apple’s root CA certificates.

4. Notarize the signed app bundle. Also requires an Apple Developer account, and almost requires me to futz with the command-line options of their tools that make this happen. To be confident of success, you need to run this on the latest Mac OS with the latest Xcode.

Rinse and repeat each time I update the app.

There are so many things to hate about this process, and it is sufficiently cumbersome that I only do this for “very important” apps that have commercial viability.

The idea for jDeploy was born when I decided that there had to be a better way. On the back of my mental napkin, I jotted down a few characteristics that I wanted my “desktop app” deployment process to have. They included:

1. Cross-platform. You shouldn’t need a Mac to build for Mac, or a Windows to build for Windows. You should be able to build your app bundle for all platforms (Mac, Windows and Linux) on any platform. E.g., you should be able to build your Mac app on Linux, if that’s what you use for development.

2. Auto-updates. Deployed apps should be able to automatically download updates so that users don’t need to keep downloading your latest version. (99% of support responses begin with the question: “what version are you using?”)

3. No bundled JVM. Apps shouldn’t need to embed the JVM in the app bundle. The launcher should be smart enough to download an appropriate JVM as needed. This way if you deploy multiple apps with compatible JVM requirements, they can all share the same JVM.

4. No code-signing required. The code-signing process is the most painful part of app distribution, and I don’t think it would be possible to achieve the cross-platform requirement unless I found a way to work around code-signing.

5. Apps should feel like first-class desktop apps. The app should integrate properly with the native system. There shouldn’t be any “clues” to the user that it was written in Java or packaged in a special tool.

The Launcher

The key to bundling a Java app as a native app is to create a native launcher. This is a tiny native program that sets up the JVM and runs the Java app. On Windows and Linux, this launcher is the executable that users “double-click”. On Mac, apps are actually directories with the “.app” extension, and include a particular file structure. The launcher is typically placed inside the Contents/MacOS directory of the bundle. When a user double-clicks the app, the system looks up the launcher name in the bundle’s Info.plist file, and runs it.

The launcher was the key to solving most of the problems I had with Java app distribution. Most existing launchers just load the JVM and run the app. The jDeploy launcher would first download app updates, and install an appropriate JVM (if it couldn’t find one already installed), before loading the JVM and running the app. This would solve the auto-update and no bundled JVM requirements.

In order to ensure a cross-platform build process, the launcher would be pre-compiled and included as part of jDeploy. This implies that every app built with jDeploy would use the same launcher binary. The launcher would load some XML meta-data to discover which app it needed to load. On Mac, this meta-data would be included as a file inside the app bundle. On Windows and Linux, since the launcher was a single binary file, the meta-data would be appended to end of the binary - and the launcher would know how to extract it at runtime.

This strategy (using the same launcher binary in each app) allowed me to write cross-platform build workflows for native apps that didn’t require the use of any third-party tools. On Mac, for example, the workflow constructs the .app bundle by creating the correct structure of text files and directories, then just copies the pre-compiled binary into the Contents/MacOS directory of the app bundle.

Slaying the Code-sign Dragon

Suffice to say that I was pretty satisfied with myself the first time I built a native Mac bundle with jDeploy that actually launched. I felt immense gratification when, after double-clicking my app, it popped up a progress window indicating that it was downloading app updates, then downloading the java runtime environment. Actually, I’m taking some artistic license in describing that scene, as it never really happened that way. The road to “success” was iterative, and full of set-backs. There were many gratifying moments where I witnessed that “Update” dialog working properly, but I can’t honestly think of any particular moment as the moment. That scene, therefore, is a fictional aggregation of many such moments.

While that successful launch was a triumph, for sure, there was still an elephant lurking in the room, and it couldn’t be ignored forever. In order for my app bundle to be useful, I needed to be able to distribute it, but Apple’s code-signing rules made that difficult. If I zipped up the app and copied it to my web server, then downloaded it on a different Mac, it would complain that the app wasn’t signed. And on anything newer than Catalina (I think Catalina was the beginning of strict compliance, though it might have been Mojave), it wouldn’t let me run the app at all. This is because, when you download an app from the internet using your web browser, it will set a “Quarantine” flag on the app bundle, which tells GateKeeper (Mac’s security agent) to run some security checks on the app the first time it is run, and to prevent it from running at all if it isn’t signed.

Adding Code-signing to the Build Process

In order to move forward, I temporarily compromised on my values, and added support for “code-signing” as a part of the jDeploy build process. This, of course, only worked when I was building the app on a Mac with Xcode’s build tools installed. Unfortunately, this wasn’t enough to satisfy even my own use-cases, despite the fact that I use a Mac. My main workhorse is a 2012 Mac Pro, and at the time I added the code-signing feature, I was limited to running 10.13 (High Sierra), and, for some reason, the apps signed on High Sierra still wouldn’t run on the latest versions. I needed to do the code-sign step on a newer Mac. Ugh!!

So I went further down the rabbit hole of compromised values, and built a “Code-sign daemon” mode for jDeploy that allowed me to delegate the “code-signing” to a different machine, using a “client-server” model. I made it so that I could run jDeploy in “server mode”. When running in server mode, it could receive requests from jDeploy clients to code-sign an app bundle. This allowed me to run jDeploy on my older version of Mac OS, and have the code-signing performed by my app running a newer version of Mac OS. Technically this also solved the “cross-platform” requirement, sort of, since client mode would work on Windows and Linux as well.

Wiping out the stain of Code-signing

A smarter man might have taken this “win” and called it a day, but I still wasn’t satisfied with the process. While you could technically now build a Mac app on Windows or Linux, you still needed to have a Mac on your network to do the actual code-signing. So you still had to jump through all of Apple’s code-signing hoops, and in some ways having to set up a separate machine made the process more complicated, not less.

The mere act of code-signing was a stain on the entire process. I would not be satisfied until I had found a way to either skip the process entirely, or to “automate” it away. As far as I know, there is no way to completely “automate” it away. If you’re going to participate in the code-sign circus, you will need to at least occasionally log into your Apple developer account, sign some agreements, and generate new certificates. There was no way around that. Perhaps this quixotic quest was doomed to fail.

Inspired by Chrome

When I’m stuck, I like to look to prior art for inspiration. Who else has faced a similar challenge before, and how did they solve it? While engaging in such a thought exercise, I recalled that Chrome allowed you to save a PWA (Portable Web App) as a sort of native app, that you could double-click, and it would open the PWA in Chrome. I doubted that Chrome signed these apps, but wasn’t sure. To I peeked inside one of them to see what was going on. As I suspected, it wasn’t signed. But if these app bundles weren’t signed, I wondered, how did they by-pass GateKeeper?

That’s when the light went on for me. The “Quarantine” flag is applied to an app when it is downloaded. If the app is never downloaded it doesn’t get quarantined. In the spirit of completeness, I tested various archive solutions (tar, zip, etc…) to see if they were somehow immune to the quarantine flag, but, as you would expect, they aren’t. If you download a Zip file, it will flag the zip, and when you extract it, the flag gets inherited by the extracted files. So the solution might be to avoid downloading the app, somehow. But how? In Chrome’s case, it avoided signing its PWA bundles because it constructed those bundles directly - it didn’t download them. Perhaps I could use the same strategy here.

Following this insight, I created an “installer” app that would know how to construct the native bundle on the fly. Because the app, then, wouldn’t be downloaded, it wouldn’t have the quarantine flag applied to it, and thus, it could run without being signed. It turns out that using an installer had some additional benefits, such as the ability to automatically add aliases to the Dock and Desktop. It also provided the user with an opportunity to customize their auto-update preferences. However, it didn’t quite solve the code-sign problem. It just kicked the can down the road a little bit.

So, I now had a workable solution that would allow me to not code-sign the app bundle itself, but I had a new problem: the installer needed to be code-signed. At first, I feared that I was effectively no further ahead. After all if I’m going to have to code-sign the installer, I might as well have just code-signed the app and be done with it.

Sign Once Run Anywhere

Then I had another thought. I had already managed to solve some of the cross-platform issues by pre-compiling the launcher and including the same launcher in every app. What if I did the same with the installer? I.e., what if every app used the same installer? Then I could “pre-sign-and-notarize” the installer bundle and include it with jDeploy. Because the installer was pre-signed, I wouldn’t be able to change it in any way on a per-app basis. That might be a show-stopper, unless I could find a way for the installer to load some meta-data that could be used to customize the app.

In the worst case, I could just prompt the user to enter some sort of code or URL into the installer, to load the correct app configuration. But I’d reserve that for “plan Z”. I preferred to find something seamless that didn’t involve prompting the user.

My first idea was to package the installer inside an archive (e.g., .tar or .zip), but include a hidden directory with some meta-data. The installer could then look for this hidden directory when it runs, and load the meta-data seamlessly behind the scenes. Initial tests seemed to work, but when I tried this on Catalina, I found that it wasn’t able to find this hidden directory for some reason. I discovered that, when you run a “quarantined” app, it will actually make a copy of the app bundle into a temporary directory, and run that copy instead. This made it impossible to find the meta-data that I had provided. Clever move, Apple.

The only thing I was able to change on the installer without breaking the integrity of the code signature was the name of the app. That isn’t much, but it was all that I needed. jDeploy apps were already deployed in the cloud (on npm), so it was already making HTTP requests to the npm registry to get app information. I just needed to embed some sort of code in the app’s name so that the installer knew which app it was installing. Including some big long npm project name would be ugly, so I decided to embed a short code into the version string, which is part of the installer app’s name, that could be used to look up the app details.

This required a small change to the build process for it to work. I set up a central registry to keep track of these app codes. When jDeploy publishes an app to npm, it will register the app with the jDeploy registry. When the user later runs the installer, it will contact the registry and load the app details. This is all seamless, automated, and cross-platform.

Mission Accomplished…

So here we are. We now have a simple process for building and distributing Java apps native desktop bundles on Mac, Windows, and Linux. The initial setup is relatively painless, and involves only running the jDeploy GUI to set up things like your app name, version, and icon. Thereafter it can be completely automated. You don’t need to futz with Apple developer certificates ever again, if you don’t want to. For that matter, you don’t need to have a Mac at all. As far as my original goals go, I’m at “Mission Accomplished” stage.

Is jDeploy right for every project? No. If you want to distribute your app in the Mac app store, you can’t do it the jDeploy way. You need to bundle everything, including the JVM, inside your app bundle, and you need to sign it using an Apple Developer certificate. In some cases, even if you’re not deploying to the app store, you may prefer the more conventional approach of code-signing and notarizing your app. It is a hassle to jump through those hoops, and you’ll never get those hours of fighting with Apple’s certificates back, but if the app is a major commercial app, it might still be worth it. But for the rest of the apps - for the simple little utilities you develop to make peoples’ lives better, or for your own internal development tools, or just about anything else you want to distribute to your desktop users, jDeploy is a good choice.

Share your own stories

The goal of this newsletter is to build a place for Java desktop developers to come together, and talk about ways to make Java more compelling for desktop development. I write a mix of historical articles on Java-related topics, and technical tutorials on things desktop-related. I try to publish a new article weekly. If you want to be notified of new posts as they happen, remember to subscribe.

Also, please share your own code-signing war stories in the comments. I’m always happy to hear about how code-signing has ruined the lives of other devs. Makes me feel less “alone” in the world :)