Apple notarization failure

[wetneb]

Hello,

Thanks for maintaining this library! OpenRefine depends on it, and I am looking into signing our MacOS release properly, including "notarizing" it with Apple, which is required on recent versions of MacOS for the security warnings to disappear.
As part of that process, we get the following error messages:

[INFO]      [exec]   "issues": [
[INFO]      [exec]     {
[INFO]      [exec]       "severity": "error",
[INFO]      [exec]       "code": null,
[INFO]      [exec]       "path": "openrefine-mac-3.6-SNAPSHOT.dmg/OpenRefine.app/Contents/Resources/webapp/extensions/jython/module/MOD-INF/lib/jython-standalone-2.7.2.jar/jni/Darwin/libjffi-1.2.jnilib",
[INFO]      [exec]       "message": "The binary uses an SDK older than the 10.9 SDK.",
[INFO]      [exec]       "docUrl": null,
[INFO]      [exec]       "architecture": "i386"
[INFO]      [exec]     },
[INFO]      [exec]     {
[INFO]      [exec]       "severity": "error",
[INFO]      [exec]       "code": null,
[INFO]      [exec]       "path": "openrefine-mac-3.6-SNAPSHOT.dmg/OpenRefine.app/Contents/Resources/webapp/extensions/jython/module/MOD-INF/lib/jython-standalone-2.7.2.jar/jni/Darwin/libjffi-1.2.jnilib",
[INFO]      [exec]       "message": "The binary uses an SDK older than the 10.9 SDK.",
[INFO]      [exec]       "docUrl": null,
[INFO]      [exec]       "architecture": "x86_64"
[INFO]      [exec]     }
[INFO]      [exec]   ]

It looks like the fact that libjffi-1.2.jnilib was built with a version of Xcode that is too old is preventing us from notarizing the package.
Looking at your README, it looks like there is no chance to upgrade this version of Xcode since you need to support 32bit architectures.

It looks like it is therefore impossible for us to support both 32-bit architectures and have a successful notarization such that users with recent versions of MacOS do not get a security warning. Therefore I wonder if it would make sense to generate a build of libjffi with only 64 bit support, allowing it to be generated by a recent version of Xcode. I wonder if you would consider offering such a build directly (e.g. by dropping support for 32-bit) or if we should do it ourselves.

Issue on OpenRefine's side: OpenRefine/OpenRefine#4568

Note: The OpenRefine project has funding available to work on this. Do you know someone who would be able to get this fixed? Send them our way!
https://openrefine.org/blog/2022/09/30/windows-macos-packaging.html

[headius]

A 64-bit only build is certainly possible but I have to admit I am not very good at working with the complicated Apple build pipeline! Are you able to help me get the right changes made so we can publish a binary that works for you?

[wetneb]

Ok! As I do not have access to a recent MacOS machine, my first step towards that would be to write a workflow file for GitHub Actions to run the compilation of the library on this platform. Perhaps this is something that could be useful to integrate to your existing CI.
I am not anticipating to work on this very soon but I am keeping it on my radar.

[AndyWi]

Hi, just thought I should mention since, hopefully, someone's looking into this - I see a different notarisation error for libjffi-1.2, to do with no valid developer id certificate. This is a Java app built with openjdk-16.0.1.

"issues": [
{
"severity": "error",
"code": null,
"path": "myapp.app.zip/myapp.app/Contents/Resources/Java/libs/jffi-1.3.9-native.jar/jni/Darwin/libjffi-1.2.jnilib",
"message": "The binary is not signed with a valid Developer ID certificate.",
"docUrl": null,
"architecture": "x86_64"
},
{
"severity": "error",
"code": null,
"path": "myapp.app.zip/myapp.app/Contents/Resources/Java/libs/jffi-1.3.9-native.jar/jni/Darwin/libjffi-1.2.jnilib",
"message": "The signature does not include a secure timestamp.",
"docUrl": null,
"architecture": "x86_64"
},
{
"severity": "error",
"code": null,
"path": "myapp.app.zip/myapp.app/Contents/Resources/Java/libs/jffi-1.3.9-native.jar/jni/Darwin/libjffi-1.2.jnilib",
"message": "The binary is not signed with a valid Developer ID certificate.",
"docUrl": null,
"architecture": "arm64"
},
{
"severity": "error",
"code": null,
"path": "myapp.app.zip/myapp.app/Contents/Resources/Java/libs/jffi-1.3.9-native.jar/jni/Darwin/libjffi-1.2.jnilib",
"message": "The signature does not include a secure timestamp.",
"docUrl": null,
"architecture": "arm64"
}
]

[wetneb]

Ah yes, in my case this error did not appear because I am recursively signing all native libraries in jars of our dependencies.

[headius]

I definitely want to solve this, and have an Apple developer account I should be able to use to sign it, but I am not great with the build process on MacOS.

[AndyWi]

Ah yes, in my case this error did not appear because I am recursively signing all native libraries in jars of our dependencies.

Hi, thanks for the suggestion but may I ask how you're doing this recursive signing please? It seems that codesign --force --deep should do the trick but sadly not in this case.

[wetneb]

We do this: https://github.com/OpenRefine/OpenRefine/blob/master/packaging/apple_certs/codesign.sh

[AndyWi]

Thanks Antonin, much appreciated - that solved my problem

[wetneb]

The OpenRefine project has funding available to work on this. Do you know someone who would be able to get this fixed? Send them our way!
https://openrefine.org/blog/2022/09/30/windows-macos-packaging.html

[headius]

@wetneb Hello there! Normally that would be me, but currently we have an contributor @splatch looking into overhauling how we build binaries, which should include this fix too. Perhaps they can be enticed to spend more time on it? 😃

Otherwise, I can help get this working and signed properly if someone can give me some clear instructions on what to fix (I am not an Apple developer but I have access to machines).

[wetneb]

Thanks @headius! Hi @splatch, I think it would be great if we could support your effort in improving your build pipeline. I'll reach out by email to coordinate on the details.

[wetneb]

I ended up just building the file on a MacOS machine (just by building this repository with ant on Mac OS X 12), and dynamically replacing the file bundled in the .jar by it, at packaging time. This lets us then sign it and notarize the entire app.

If that is of interest, our version of libjffi-1.2.jnilib can be downloaded here: https://github.com/OpenRefine/OpenRefine/blob/master/packaging/apple_certs/libjffi-1.2.jnilib?raw=true

Of course patching the .jar on our side is not a long term solution, but it confirms that building the next release on a recent Apple machine (for instance on a GitHub Action) should fix the problem.