Flutter (Dart) apps fail Apple's submission validation, likely due to libsherpa dylib being present in root of Frameworks folder #1172

BrutalCoding · 2024-07-26T16:55:46Z

Heya,

First off, many thanks for working on bringing this package to Dart. I have been working with this package since several months ago in Flutter in my own way and today I tried to check out your official Dart version on pub.dev - it's great!

TTS, STT, VAD (the new v5) are all working perfect on my physical iPhone 12 Pro, so the features are not an issue here.

There is an issue though when deploying an app for App Store because sherpa-onnx bundles the dynamic library libsherpa-onnx-c-api.dylib in the root of the Frameworks directory and that gets automatically rejected by Apple when validating an app app for submission. It must be packaged as a framework. Note: This is an issue on at least iOS, as I haven't tried sherpa-onnx (Dart plugin) on macOS yet, but it's likely an issue there too.

Misleading error message: ITMS-90426: Invalid Swift Support - The SwiftSupport folder is missing. Rebuild your app using the current public (GM) version of Xcode and resubmit it. (Doesn't help - already on latest version. See below for suspected real cause of error in this case).

Here's what I've tried:

Failed submission of Runner.ipa with Transporter.app

Build app for release with flutter build ios
Drag the newly built .ipa file into Apple's Transporter.app app
During the validation stage, Transporter will show this error:

Failed submission of Runner.xcarchive via XCode

Find the Runner.xcarchive in Finder
Double click to open it in Xcode (if I recall correctly). Alternatively, open the ios project in Xcode and click on "View" (top menu-bar) -> Organizer -> Drag and Drop the Runner.xcarchive file in there.
Click on the archive in the Organizer pane and either click on "Validate App" or "Distribute App" (I tried both of course).
Click through and when asked, tell Xcode you're uploading the app for App Store Connect (do not choose for Ad hoc, Enterprise or Debugging)
All validity checks passed with success
Quickly visit App Store Connect, visit your app there and check the TestFlight page. You'll get the impression that your app is almost done processing and ready to test because the new version is appearing there for a minute or so (it silently disappears).
Check your e-mail inbox for Apple's e-mail, it's the same error:

Identified issues

The error message is misleading, since I am running the latest Xcode and I have submitted it in all possible ways. This led me to investigate the contents of the Runner.xcarchive to see how sherpa-onnx is shipped with the app, and I found an issue here:

We can not ship an app with a dylib thrown in the root of the Frameworks folder, instead, it must be packaged in it's own contained "framework" (just a folder structure with a couple files for metadata e.g. info.plist).

I'm sorry to not have the specific details at hand, I just wanted to highlight this issue before I'm fixing this. A quick work-around that I will try first is not fixing any source code, but by modifying the contents of the archive and re-attempting to upload the app for submission.

Better way would be to fix this in the source code of course. Some changes are likely required where you build with CMake against the C API, after producing the the shared library (the .dylib) we could add a few more lines to produce a xcframework folder and move the .dylib in there.

If there's anything unclear, ask away.

edit:

Fixed typo from '.xarchive' to '.xcarchive'
Replaced '<app_name>' with 'Runner' for clarity

The text was updated successfully, but these errors were encountered:

BrutalCoding · 2024-07-27T01:07:58Z

I will work on this issue and post an update tomorrow. I will also see if I can find and fix the issue in sherpa-onnx itself, if so, I'll open a PR of course.

csukuangfj · 2024-07-27T01:30:30Z

Thank you!

BrutalCoding · 2024-07-28T13:09:39Z

Quick update, I've written a script that fixes this issue. Script expects 1 argument that points to the file path of the affected .xcarchive and fixes it.

I haven't had time to look into the sherpa-onnx source itself, but I'm pretty confident that I can solve this here too.

If I can't find and fix the issue in sherpa-onnx itself in the upcoming days, I'll share my script. That way, we can fix it together in sherpa-onnx.

BrutalCoding · 2024-07-28T13:20:25Z

Here's the new .xcarchive content, showing that sherpa-onnx is now a good .framework instead of being absent (none listed). This is a screenshot of mine taken from one of the confirmation dialogs in Xcode > Organizer (Archives) > Distribute App (blue button):

BrutalCoding · 2024-07-31T17:59:56Z

Update: I discovered an issue regarding missing symbols during runtime when I tested my test app from TestFlight. I'll post here once I've figured it out.

Just adding this note for reference, it's related to the issue I am facing (the error message) and the observation I made (the .dylib in the Frameworks folder):
https://developer.apple.com/library/archive/technotes/tn2435/_index.html#//apple_ref/doc/uid/DTS40017543-CH1-TROUBLESHOOTING_BUNDLE_ERRORS-EMBEDDED__DYLIB_FILES

Here are some more links with valuable info:

This issue is related to the same type of issue we're facing. Link: Native Assets should use Framework instead of dylib on iOS. flutter/flutter#140544 (comment)
I personally faced this issue earlier too during the development of my Flutter plugin, and solved it too. Yet, I can't fully grasp the issue right now. Link: https://github.com/BrutalCoding/aub.ai/blob/b16f7e5f9d317d83b38a5814afb1556c8de8a3d3/brutalcoding_scripts/autoupdate_ios.sh#L66C3-L66C7. I looked and changed things around files such as your sherpa_onnx_ios.podspec, sherpa_onnx.dart (e.g. trying DynamicLibrary.process() instead), and I also looked at your cmake stuff at build-ios-shared.sh/build-ios.sh etc.

I'll need some more time to figure this out.

Okay, 1 hour after I placed this comment and it's fixed. What a pain to work with Apple's confusing dev requirements but here's what I did in short:

Run the build-ios-shared.sh script
Find and copy the libsherpa-onnx-c-api.dylib file.
Navigate to sherpa-onnx/flutter/sherpa_onnx/sherpa_onnx_ios/ios/. Place the copied .dylib here.
Still within this ios folder, create a SherpaOnnx.framework folder and add a proper Info.plist inside this new .framework folder.
Now, use lipo and install_name_tool to convert libsherpa-onnx-c-api.dylib into SherpaOnnx (no file extension). Move this SherpaOnnx binary into the .framework folder.
Update the podspec file by adding a vendored_frameworks that is referring to this new .framework. Also, remove the existing vendored_libraries line.
Update the dart file to replace the dynamic library lookup for iOS from DynamicLibrary.open("libsherpa-onnx-c-api.dylib") to DynamicLibrary.process(), this method will be able to find all global symbols that are running along with the app.

I will review my own changes over the next days and will try to keep it to a mimimum.

helang1991 · 2024-08-04T06:11:58Z

Update: I discovered an issue regarding missing symbols during runtime when I tested my test app from TestFlight. I'll post here once I've figured it out.

Just adding this note for reference, it's related to the issue I am facing (the error message) and the observation I made (the .dylib in the Frameworks folder): https://developer.apple.com/library/archive/technotes/tn2435/_index.html#//apple_ref/doc/uid/DTS40017543-CH1-TROUBLESHOOTING_BUNDLE_ERRORS-EMBEDDED__DYLIB_FILES
Here are some more links with valuable info:

This issue is related to the same type of issue we're facing. Link: Native Assets should use Framework instead of dylib on iOS. flutter/flutter#140544 (comment)

I personally faced this issue earlier too during the development of my Flutter plugin, and solved it too. Yet, I can't fully grasp the issue right now. Link: https://github.com/BrutalCoding/aub.ai/blob/b16f7e5f9d317d83b38a5814afb1556c8de8a3d3/brutalcoding_scripts/autoupdate_ios.sh#L66C3-L66C7. I looked and changed things around files such as your sherpa_onnx_ios.podspec, sherpa_onnx.dart (e.g. trying DynamicLibrary.process() instead), and I also looked at your cmake stuff at build-ios-shared.sh/build-ios.sh etc.

I'll need some more time to figure this out.

Okay, 1 hour after I placed this comment and it's fixed. What a pain to work with Apple's confusing dev requirements but here's what I did in short:

Run the build-ios-shared.sh script

Find and copy the libsherpa-onnx-c-api.dylib file.

Navigate to sherpa-onnx/flutter/sherpa_onnx/sherpa_onnx_ios/ios/. Place the copied .dylib here.

Still within this ios folder, create a SherpaOnnx.framework folder and add a proper Info.plist inside this new .framework folder.

Now, use lipo and install_name_tool to convert libsherpa-onnx-c-api.dylib into SherpaOnnx (no file extension). Move this SherpaOnnx binary into the .framework folder.

Update the podspec file by adding a vendored_frameworks that is referring to this new .framework. Also, remove the existing vendored_libraries line.

Update the dart file to replace the dynamic library lookup for iOS from DynamicLibrary.open("libsherpa-onnx-c-api.dylib") to DynamicLibrary.process(), this method will be able to find all global symbols that are running along with the app.

I will review my own changes over the next days and will try to keep it to a mimimum.

Hello, I followed your method, but encountered a problem. Can you share detail steps, like screenshots

helang1991 · 2024-08-06T09:13:38Z

@BrutalCoding Hello, I found a problem that uploading to testFlight

eschmidbauer · 2024-08-15T18:41:16Z

any update on this? i'm trying to build for iOS and encountering same error.
here are some steps i outlined based on the resources in this thread:

bash build-ios-shared.sh

cd build-ios-shared/ios-arm64-simulator/

lipo -create libsherpa-onnx-c-api.dylib -output SherpaOnnx

hantrungkien · 2024-10-25T18:04:30Z

@BrutalCoding I did like #1283 (comment) but I got a error #1283 (comment)

Do you have any solution for that? Many thanks!

hantrungkien · 2024-10-26T17:21:46Z

This is the complete solution for everyone having the same problem as me:

Step 1.

git clone https://github.com/k2-fsa/sherpa-onnx.git && \
cd sherpa-onnx && \
sh build-ios-shared.sh && \
cd build-ios-shared

Step 2.

mkdir ios-arm64_x86_64-simulator && \
lipo -create \
ios-arm64-simulator/libsherpa-onnx-c-api.dylib \
ios-x86_64-simulator/libsherpa-onnx-c-api.dylib \
-output ios-arm64_x86_64-simulator/libsherpa-onnx-c-api.dylib

Step 3.

cd ios-arm64 && \
mkdir libsherpa-onnx-c-api.framework && \
lipo -create libsherpa-onnx-c-api.dylib -output libsherpa-onnx-c-api && \
mv libsherpa-onnx-c-api libsherpa-onnx-c-api.framework/ && \
rm -rf libsherpa-onnx-c-api.dylib && \
cd libsherpa-onnx-c-api.framework && \
install_name_tool -change @rpath/libsherpa-onnx-c-api.dylib @rpath/libsherpa-onnx-c-api.framework/libsherpa-onnx-c-api libsherpa-onnx-c-api && \
install_name_tool -id "@rpath/libsherpa-onnx-c-api.framework/libsherpa-onnx-c-api" libsherpa-onnx-c-api && \
chmod +x libsherpa-onnx-c-api && \
cd .. && \
cd ..

Step 4.

cd ios-arm64_x86_64-simulator && \
mkdir libsherpa-onnx-c-api.framework && \
lipo -create libsherpa-onnx-c-api.dylib -output libsherpa-onnx-c-api && \
mv libsherpa-onnx-c-api libsherpa-onnx-c-api.framework/ && \
rm -rf libsherpa-onnx-c-api.dylib && \
cd libsherpa-onnx-c-api.framework && \
install_name_tool -change @rpath/libsherpa-onnx-c-api.dylib @rpath/libsherpa-onnx-c-api.framework/libsherpa-onnx-c-api libsherpa-onnx-c-api && \
install_name_tool -id "@rpath/libsherpa-onnx-c-api.framework/libsherpa-onnx-c-api" libsherpa-onnx-c-api && \
chmod +x libsherpa-onnx-c-api && \
cd .. && \
cd ..

Step 5. Add Info.plist

ios-arm64/libsherpa-onnx-c-api.framework/Info.plist
ios-arm64_x86_64-simulator/libsherpa-onnx-c-api.framework/Info.plist

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>BuildMachineOSBuild</key>
    <string>{YOUR_MACHINE_BUILD}</string>
    <key>CFBundleDevelopmentRegion</key>
    <string>en</string>
    <key>CFBundleExecutable</key>
    <string>libsherpa-onnx-c-api</string>
    <key>CFBundleIdentifier</key>
    <string>{YOUR_BUNDLE_ID}</string>
    <key>CFBundleInfoDictionaryVersion</key>
    <string>6.0</string>
    <key>CFBundleName</key>
    <string>libsherpa-onnx-c-api</string>
    <key>CFBundlePackageType</key>
    <string>FMWK</string>
    <key>CFBundleShortVersionString</key>
    <string>1.0.0</string>
    <key>CFBundleSupportedPlatforms</key>
    <array>
        <string>iPhoneOS</string>
    </array>
    <key>CFBundleVersion</key>
    <string>1</string>
    <key>MinimumOSVersion</key>
    <string>{YOUR_MINIMUM_VERSION}</string>
</dict>
</plist>

Step 7.

xcodebuild -create-xcframework \
-framework ios-arm64/libsherpa-onnx-c-api.framework \
-framework ios-arm64_x86_64-simulator/libsherpa-onnx-c-api.framework \
-output sherpa_onnx.xcframework

Step 8. Copy sherpa_onnx.xcframework to ios/Frameworks and add it to Runner

Step 9. Only get 2 packages from sherpa-onnx repository

sherpa_onnx
sherpa_onnx_android

Step 10. Modify

`sherpa_onnx`

pubspec.yaml

Remove other dependencies except sherpa_onnx_android

dependencies:
  ffi: ^2.1.0
  flutter:
    sdk: flutter

  sherpa_onnx_android:
    path: sherpa_onnx_android

dev_dependencies:
  flutter_lints: ^3.0.0

flutter:
  plugin:
    platforms:
      android:
        default_package: sherpa_onnx_android

lib/sherpa_onnx.dart

void initBindings() {
  SherpaOnnxBindings.init(Platform.isAndroid
      ? DynamicLibrary.open('libsherpa-onnx-c-api.so')
      : DynamicLibrary.open(
          'libsherpa-onnx-c-api.framework/libsherpa-onnx-c-api'));
}

`sherpa_onnx_android`

Move sherpa_onnx_android to inside sherpa_onnx

w-rui · 2024-11-08T03:02:30Z

@hantrungkien @csukuangfj 有计划什么时候推送线上更新吗？手动改这个挺麻烦的，更新也不方便

hantrungkien · 2024-11-08T04:30:13Z

@w-rui You can fork this repo and make your script follow as my guide to automatically build and publish your library.

csukuangfj · 2024-11-08T04:47:00Z

@hantrungkien @csukuangfj 有计划什么时候推送线上更新吗？手动改这个挺麻烦的，更新也不方便

没计划。

因为，pub.dev限制每个包解压后，不超过100MB。我们没法把simulator相关的库也上传到pub.dev

w-rui · 2024-11-08T06:39:13Z

明白了，想要更新线上，只能将 framework 相关的大文件上传 cocopods，然后才能上传 pub。
不过你们可以考虑将上文的手动 script 流程写到 flutter/build_ios_framework.sh 之类的。
多谢

csukuangfj · 2024-11-16T08:07:25Z

@hantrungkien @csukuangfj 有计划什么时候推送线上更新吗？手动改这个挺麻烦的，更新也不方便

不需要手动改了。目前最新版 v1.10.31 支持了. 不需要做任何代码改动。@w-rui

csukuangfj · 2024-11-16T08:08:24Z

This is the complete solution for everyone having the same problem as me:

Step 1.

git clone https://github.com/k2-fsa/sherpa-onnx.git && \
cd sherpa-onnx && \
sh build-ios-shared.sh && \
cd build-ios-shared

Step 2.

mkdir ios-arm64_x86_64-simulator && \
lipo -create \
ios-arm64-simulator/libsherpa-onnx-c-api.dylib \
ios-x86_64-simulator/libsherpa-onnx-c-api.dylib \
-output ios-arm64_x86_64-simulator/libsherpa-onnx-c-api.dylib

Step 3.

cd ios-arm64 && \
mkdir libsherpa-onnx-c-api.framework && \
lipo -create libsherpa-onnx-c-api.dylib -output libsherpa-onnx-c-api && \
mv libsherpa-onnx-c-api libsherpa-onnx-c-api.framework/ && \
rm -rf libsherpa-onnx-c-api.dylib && \
cd libsherpa-onnx-c-api.framework && \
install_name_tool -change @rpath/libsherpa-onnx-c-api.dylib @rpath/libsherpa-onnx-c-api.framework/libsherpa-onnx-c-api libsherpa-onnx-c-api && \
install_name_tool -id "@rpath/libsherpa-onnx-c-api.framework/libsherpa-onnx-c-api" libsherpa-onnx-c-api && \
chmod +x libsherpa-onnx-c-api && \
cd .. && \
cd ..

Step 4.

cd ios-arm64_x86_64-simulator && \
mkdir libsherpa-onnx-c-api.framework && \
lipo -create libsherpa-onnx-c-api.dylib -output libsherpa-onnx-c-api && \
mv libsherpa-onnx-c-api libsherpa-onnx-c-api.framework/ && \
rm -rf libsherpa-onnx-c-api.dylib && \
cd libsherpa-onnx-c-api.framework && \
install_name_tool -change @rpath/libsherpa-onnx-c-api.dylib @rpath/libsherpa-onnx-c-api.framework/libsherpa-onnx-c-api libsherpa-onnx-c-api && \
install_name_tool -id "@rpath/libsherpa-onnx-c-api.framework/libsherpa-onnx-c-api" libsherpa-onnx-c-api && \
chmod +x libsherpa-onnx-c-api && \
cd .. && \
cd ..

Step 5. Add Info.plist

ios-arm64/libsherpa-onnx-c-api.framework/Info.plist
ios-arm64_x86_64-simulator/libsherpa-onnx-c-api.framework/Info.plist

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>BuildMachineOSBuild</key>
    <string>{YOUR_MACHINE_BUILD}</string>
    <key>CFBundleDevelopmentRegion</key>
    <string>en</string>
    <key>CFBundleExecutable</key>
    <string>libsherpa-onnx-c-api</string>
    <key>CFBundleIdentifier</key>
    <string>{YOUR_BUNDLE_ID}</string>
    <key>CFBundleInfoDictionaryVersion</key>
    <string>6.0</string>
    <key>CFBundleName</key>
    <string>libsherpa-onnx-c-api</string>
    <key>CFBundlePackageType</key>
    <string>FMWK</string>
    <key>CFBundleShortVersionString</key>
    <string>1.0.0</string>
    <key>CFBundleSupportedPlatforms</key>
    <array>
        <string>iPhoneOS</string>
    </array>
    <key>CFBundleVersion</key>
    <string>1</string>
    <key>MinimumOSVersion</key>
    <string>{YOUR_MINIMUM_VERSION}</string>
</dict>
</plist>

Step 7.

xcodebuild -create-xcframework \
-framework ios-arm64/libsherpa-onnx-c-api.framework \
-framework ios-arm64_x86_64-simulator/libsherpa-onnx-c-api.framework \
-output sherpa_onnx.xcframework

Step 8. Copy sherpa_onnx.xcframework to ios/Frameworks and add it to Runner

Step 9. Only get 2 packages from sherpa-onnx repository

sherpa_onnx
sherpa_onnx_android

Step 10. Modify

`sherpa_onnx`

pubspec.yaml

Remove other dependencies except sherpa_onnx_android

dependencies:
  ffi: ^2.1.0
  flutter:
    sdk: flutter

  sherpa_onnx_android:
    path: sherpa_onnx_android

dev_dependencies:
  flutter_lints: ^3.0.0

flutter:
  plugin:
    platforms:
      android:
        default_package: sherpa_onnx_android

lib/sherpa_onnx.dart

void initBindings() {
  SherpaOnnxBindings.init(Platform.isAndroid
      ? DynamicLibrary.open('libsherpa-onnx-c-api.so')
      : DynamicLibrary.open(
          'libsherpa-onnx-c-api.framework/libsherpa-onnx-c-api'));
}

`sherpa_onnx_android`

Move sherpa_onnx_android to inside sherpa_onnx

@hantrungkien Thank you for sharing.

We have integrated it into our latest release v1.10.31

You don't need to change any code now.

hantrungkien · 2024-11-16T08:16:48Z

@csukuangfj Many thanks and what is the commit that's related with v1.10.31? I ask because I don't find it.

csukuangfj · 2024-11-16T09:02:52Z

@csukuangfj Many thanks and what is the commit that's related with v1.10.31? I ask because I don't find it.

please see #1547

csukuangfj · 2024-11-16T13:24:32Z

Does it work now?

hantrungkien · 2024-11-16T18:17:24Z

@csukuangfj sorry, I don't have time to upgrade to v1.10.31 yet. Maybe I will do that in next week...

csukuangfj mentioned this issue Aug 5, 2024

Questions related to MeloTTS #1193

Open

hantrungkien mentioned this issue Oct 25, 2024

[Flutter] Can not work on iOS Simulator #1283

Closed

csukuangfj mentioned this issue Nov 16, 2024

Use xcframework for Flutter iOS plugin. #1547

Merged

csukuangfj closed this as completed in #1547 Nov 16, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Flutter (Dart) apps fail Apple's submission validation, likely due to libsherpa dylib being present in root of Frameworks folder #1172

Flutter (Dart) apps fail Apple's submission validation, likely due to libsherpa dylib being present in root of Frameworks folder #1172

BrutalCoding commented Jul 26, 2024 •

edited

Loading

BrutalCoding commented Jul 27, 2024

csukuangfj commented Jul 27, 2024

BrutalCoding commented Jul 28, 2024

BrutalCoding commented Jul 28, 2024

BrutalCoding commented Jul 31, 2024 •

edited

Loading

helang1991 commented Aug 4, 2024

helang1991 commented Aug 6, 2024 •

edited

Loading

eschmidbauer commented Aug 15, 2024 •

edited

Loading

hantrungkien commented Oct 25, 2024

hantrungkien commented Oct 26, 2024

w-rui commented Nov 8, 2024

hantrungkien commented Nov 8, 2024

csukuangfj commented Nov 8, 2024

w-rui commented Nov 8, 2024

csukuangfj commented Nov 16, 2024

csukuangfj commented Nov 16, 2024

`sherpa_onnx`

`sherpa_onnx_android`

hantrungkien commented Nov 16, 2024

csukuangfj commented Nov 16, 2024

csukuangfj commented Nov 16, 2024

hantrungkien commented Nov 16, 2024

Flutter (Dart) apps fail Apple's submission validation, likely due to libsherpa dylib being present in root of Frameworks folder #1172

Flutter (Dart) apps fail Apple's submission validation, likely due to libsherpa dylib being present in root of Frameworks folder #1172

Comments

BrutalCoding commented Jul 26, 2024 • edited Loading

Failed submission of Runner.ipa with Transporter.app

Failed submission of Runner.xcarchive via XCode

Identified issues

BrutalCoding commented Jul 27, 2024

csukuangfj commented Jul 27, 2024

BrutalCoding commented Jul 28, 2024

BrutalCoding commented Jul 28, 2024

BrutalCoding commented Jul 31, 2024 • edited Loading

helang1991 commented Aug 4, 2024

helang1991 commented Aug 6, 2024 • edited Loading

eschmidbauer commented Aug 15, 2024 • edited Loading

hantrungkien commented Oct 25, 2024

hantrungkien commented Oct 26, 2024

sherpa_onnx

sherpa_onnx_android

w-rui commented Nov 8, 2024

hantrungkien commented Nov 8, 2024

csukuangfj commented Nov 8, 2024

w-rui commented Nov 8, 2024

csukuangfj commented Nov 16, 2024

csukuangfj commented Nov 16, 2024

sherpa_onnx

sherpa_onnx_android

hantrungkien commented Nov 16, 2024

csukuangfj commented Nov 16, 2024

csukuangfj commented Nov 16, 2024

hantrungkien commented Nov 16, 2024

BrutalCoding commented Jul 26, 2024 •

edited

Loading

BrutalCoding commented Jul 31, 2024 •

edited

Loading

helang1991 commented Aug 6, 2024 •

edited

Loading

eschmidbauer commented Aug 15, 2024 •

edited

Loading

`sherpa_onnx`

`sherpa_onnx_android`

`sherpa_onnx`

`sherpa_onnx_android`