Skip to content

Commit

Permalink
orlando - for updates of setting of tensorflow_tf
Browse files Browse the repository at this point in the history
  • Loading branch information
llv22 committed Jul 30, 2023
1 parent f8da7cc commit 0ee32d7
Show file tree
Hide file tree
Showing 21 changed files with 137 additions and 20 deletions.
57 changes: 55 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,9 +87,9 @@ bazel build --verbose_failures --experimental_repo_remote_exec --cxxopt='-std=c+
# comment out /private/var/tmp/_bazel_llv23/a82ad01ec0c5d2a91897f1531acdf67b/external/com_github_azure_azure_sdk_for_cpp/sdk/core/azure-core/inc/azure/core/http/policies/policy.hpp and /private/var/tmp/_bazel_llv23/a82ad01ec0c5d2a91897f1531acdf67b/external/com_github_azure_azure_sdk_for_cpp/sdk/storage/azure-storage-common/inc/azure/storage/common/internal/storage_per_retry_policy.hpp for std::make_unique
bazel build --verbose_failures --experimental_repo_remote_exec --cxxopt='-std=c++14' --macos_sdk_version=10.14 //tensorflow_io/...
# refer to https://stackoverflow.com/questions/73141963/cannot-build-tensorflow-io-linking-tensorflow-io-python-ops-libtensorflow-io-g
bazel build -s --verbose_failures $BAZEL_OPTIMIZATION --experimental_repo_remote_exec //tensorflow_io/... //tensorflow_io_gcs_filesystem/...
bazel build -s --verbose_failures $BAZEL_OPTIMIZATION --experimental_repo_remote_exec --cxxopt='-std=c++14' --macos_sdk_version=10.14 //tensorflow_io/... //tensorflow_io_gcs_filesystem/...
python setup.py bdist_wheel
python setup.py bdist_wheel --project tensorflow-io-gcs-filesystem
python setup.py bdist_wheel --project tensorflow-io-gcs-filesystem``
```

1, change /private/var/tmp/_bazel_llv23/a82ad01ec0c5d2a91897f1531acdf67b/external/build_bazel_rules_swift/swift/internal/xcode_swift_toolchain.bzl
Expand Down Expand Up @@ -264,6 +264,59 @@ then install for

cp -rf /Users/llv23/Documents/05_machine_learning/dl_gpu_mac/dl-built-libraries/tensorflow-built/2.9.1-cuda10.1-py3.10/ios/tensorflow_io_gcs_filesystem/core/python/ops/*.so /Users/llv23/opt/miniconda3/lib/python3.10/site-packages/tensorflow_io/python/ops/

#### Fix for video and audio module

1. Issue about "<unknown>:0: error: unknown argument: '-debug-prefix-map'"

/private/var/tmp/_bazel_llv23/a82ad01ec0c5d2a91897f1531acdf67b/external/build_bazel_rules_swift/tools/worker/swift_runner.cc
Line 279-280 comment out

```bash
// consumer("-debug-prefix-map");
// consumer(GetCurrentDirectory() + "=.");
```

2, Issue about "<unknown>:0: error: unknown argument: '-no-clang-module-breadcrumbs'"
/private/var/tmp/_bazel_llv23/a82ad01ec0c5d2a91897f1531acdf67b/execroot/org_tensorflow_io/external/build_bazel_rules_swift/swift/internal/compiling.bzl

Line 219 comment out

```bash
# Don't embed Clang module breadcrumbs in debug info.
swift_toolchain_config.action_config(
actions = [swift_action_names.COMPILE],
configurators = [
swift_toolchain_config.add_arg(
"-Xfrontend",
# "-no-clang-module-breadcrumbs",
),
],
),
```

/private/var/tmp/_bazel_llv23/a82ad01ec0c5d2a91897f1531acdf67b/execroot/org_tensorflow_io/bazel-out/darwin-fastbuild/bin/tools/build/swift/audio_video.swiftmodule-0.params

Line 11 comment out

```bash
#-no-clang-module-breadcrumbs
```

3, Issue ""

```bash
cd /private/var/tmp/_bazel_llv23/a82ad01ec0c5d2a91897f1531acdf67b/execroot/org_tensorflow_io
swiftc -framework AVFoundation -target x86_64-apple-macosx10.14 -sdk $(xcrun --sdk macosx --show-sdk-path) -emit-module-path bazel-out/darwin-fastbuild/bin/tools/build/swift/audio_video.swiftmodule -F__BAZEL_XCODE_DEVELOPER_DIR__/Platforms/MacOSX.platform/Developer/Library/Frameworks -I__BAZEL_XCODE_DEVELOPER_DIR__/Platforms/MacOSX.platform/Developer/usr/lib -emit-object -output-file-map bazel-out/darwin-fastbuild/bin/tools/build/swift/audio_video_swift.output_file_map.json -Xfrontend -DDEBUG -Onone -Xfrontend -serialize-debugging-options -enable-testing -gline-tables-only -Xcc -iquote. -Xcc -iquotebazel-out/darwin-fastbuild/bin -Xfrontend -color-diagnostics -enable-batch-mode -module-name audio_video -parse-as-library -target x86_64-apple-macosx10.14 -Xcc -O0 -Xcc -DDEBUG=1 tensorflow_io/core/swift/audio.swift tensorflow_io/core/swift/video.swift
cp bazel-out/darwin-fastbuild/bin/tools/build/swift/audio_video_swift_objs/swift/* bazel-out/darwin-fastbuild/bin/_swift_incremental/tools/build/swift/audio_video_swift_objs/swift/
cd bazel-out/darwin-fastbuild/bin/tools/build/swift/audio_video_swift_objs/swift
ar rcs libaudio_video_swift.a audio.swift.o video.swift.o
swiftc -framework AVFoundation -target x86_64-apple-macosx10.14 -sdk $(xcrun --sdk macosx --show-sdk-path) -emit-library audio.swift.o video.swift.o -o libaudio_video_swift.dylib
cd /private/var/tmp/_bazel_llv23/a82ad01ec0c5d2a91897f1531acdf67b/execroot/org_tensorflow_io
cp bazel-out/darwin-fastbuild/bin/tools/build/swift/audio_video_swift_objs/swift/libaudio_video_swift.* bazel-out/darwin-fastbuild/bin/tools/build/swift/
cp bazel-out/darwin-fastbuild/bin/tools/build/swift/libaudio_video_swift.a bazel-out/darwin-fastbuild/bin/tools/build/swift/libaudio_video.a
cp bazel-out/darwin-fastbuild/bin/tools/build/swift/libaudio_video_swift.dylib bazel-out/darwin-fastbuild/bin/tools/build/swift/libaudio_video.dylib
```

### Python Package

The `tensorflow-io` Python package can be installed with pip directly using:
Expand Down
2 changes: 1 addition & 1 deletion tensorflow_io/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ cc_binary(
"//tensorflow_io/core:bigquery_ops",
"//tensorflow_io/core:bigtable_ops",
# see: disable audio_video_ops so you can use it
# "//tensorflow_io/core:audio_video_ops",
"//tensorflow_io/core:audio_video_ops",
"//tensorflow_io/core:avro_ops",
"//tensorflow_io/core:orc_ops",
"//tensorflow_io/core:cpuinfo",
Expand Down
89 changes: 72 additions & 17 deletions tools/build/swift/BUILD
Original file line number Diff line number Diff line change
@@ -1,23 +1,78 @@
load("@build_bazel_rules_swift//swift:swift.bzl", "swift_library")
# load("@build_bazel_rules_swift//swift:swift.bzl", "swift_library")

package(default_visibility = ["//visibility:public"])

swift_library(
# swift_library(
# name = "audio_video_swift",
# srcs = [
# "//tensorflow_io/core:swift/audio.swift",
# "//tensorflow_io/core:swift/video.swift",
# ],
# copts = [
# "-target",
# "x86_64-apple-macosx10.14",
# ],
# linkopts = [
# "-L/Applications/Xcode.app/Contents/Developer//Toolchains/XcodeDefault.xctoolchain/usr/lib/swift",
# "-Wl,-rpath,/Applications/Xcode.app/Contents/Developer//Toolchains/XcodeDefault.xctoolchain/usr/lib/swift",
# "-target",
# "x86_64-apple-macosx10.14",
# ],
# module_name = "audio_video",
# alwayslink = True,
# )

cc_library(
name = "audio_video_swift",
srcs = [
"//tensorflow_io/core:swift/audio.swift",
"//tensorflow_io/core:swift/video.swift",
],
copts = [
"-target",
"x86_64-apple-macosx10.14",
],
linkopts = [
"-L/usr/lib/swift",
"-Wl,-rpath,/usr/lib/swift",
"-target",
"x86_64-apple-macosx10.14",
],
module_name = "audio_video",
srcs = [":sub/audio_video_swift"],
# srcs = select({
# "@bazel_tools//src/conditions:windows": [
# "//tools/build/swift:libaudio_video_swift.dylib",
# ],
# "//conditions:default": [
# "//tools/build/swift:libaudio_video_swift.dylib",
# ],
# }),
#data = ["lib/libtensorflow_framework.so"],
visibility = ["//visibility:public"],
alwayslink = True,
)

cc_binary(
name = "sub/audio_video_swift",
srcs = [],
linkopts = select({
"@bazel_tools//src/conditions:windows": [],
"@bazel_tools//src/conditions:darwin": [
"-Wl,-install_name,@rpath/libaudio_video_swift.dylib",
],
"//conditions:default": [
"-Wl,--disable-new-dtags",
"-Wl,-rpath,'$$ORIGIN/'",
"-Wl,-soname,libaudio_video_swift.dylib",
],
}),
linkshared = 1,
deps = [],
)

# cc_library(
# name = "libaudio_video_swift.dylib",
# srcs = ["//tools/build/swift:libaudio_video_swift.dylib"],
# # copts = [
# # "-target",
# # "x86_64-apple-macosx10.14",
# # ],
# # linkopts = [
# # "-L/Applications/Xcode.app/Contents/Developer//Toolchains/XcodeDefault.xctoolchain/usr/lib/swift",
# # "-Wl,-rpath,/Applications/Xcode.app/Contents/Developer//Toolchains/XcodeDefault.xctoolchain/usr/lib/swift",
# # "-target",
# # "x86_64-apple-macosx10.14",
# # ],
# # srcs = ["//tools/build/swift:libaudio_video_swift.a", "//tools/build/swift:libaudio_video_swift.dylib"],
# # hdrs = glob(["path/to/headers/*.h"]), # Include headers if needed
# # module_name = "audio_video",
# visibility = ["//visibility:public"],
# # static_build_only = True,
# alwayslink = True,
# )
5 changes: 5 additions & 0 deletions tools/build/swift/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# README

```bash
swiftc -framework AVFoundation -target x86_64-apple-macosx10.14 -sdk $(xcrun --sdk macosx --show-sdk-path) -emit-module-path audio_video.swiftmodule -F__BAZEL_XCODE_DEVELOPER_DIR__/Platforms/MacOSX.platform/Developer/Library/Frameworks -I__BAZEL_XCODE_DEVELOPER_DIR__/Platforms/MacOSX.platform/Developer/usr/lib -emit-object -output-file-map audio_video_swift.output_file_map.json -Xfrontend -DDEBUG -Onone -Xfrontend -serialize-debugging-options -enable-testing -gline-tables-only -Xcc -iquote. -Xcc -iquotebazel-out/darwin-fastbuild/bin -Xfrontend -color-diagnostics -enable-batch-mode -module-name audio_video -parse-as-library -target x86_64-apple-macosx10.14 -Xcc -O0 -Xcc -DDEBUG=1 ../../../tensorflow_io/core/swift/audio.swift ../../../tensorflow_io/core/swift/video.swift
```
Binary file added tools/build/swift/audio_video.swiftdoc
Binary file not shown.
Binary file added tools/build/swift/audio_video.swiftmodule
Binary file not shown.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"":{"swift-dependencies":"bazel-out/darwin-fastbuild/bin/_swift_incremental/tools/build/swift/audio_video_swift.swiftdeps"},"tensorflow_io/core/swift/audio.swift":{"ast-dump":"bazel-out/darwin-fastbuild/bin/tools/build/swift/audio_video_swift_objs/swift/audio.swift.ast","object":"bazel-out/darwin-fastbuild/bin/_swift_incremental/tools/build/swift/audio_video_swift_objs/swift/audio.swift.o","swift-dependencies":"bazel-out/darwin-fastbuild/bin/_swift_incremental/tools/build/swift/audio_video_swift_objs/swift/audio.swift.swiftdeps","swiftmodule":"bazel-out/darwin-fastbuild/bin/_swift_incremental/tools/build/swift/audio_video_swift_objs/swift/audio.swift.partial_swiftmodule"},"tensorflow_io/core/swift/video.swift":{"ast-dump":"bazel-out/darwin-fastbuild/bin/tools/build/swift/audio_video_swift_objs/swift/video.swift.ast","object":"bazel-out/darwin-fastbuild/bin/_swift_incremental/tools/build/swift/audio_video_swift_objs/swift/video.swift.o","swift-dependencies":"bazel-out/darwin-fastbuild/bin/_swift_incremental/tools/build/swift/audio_video_swift_objs/swift/video.swift.swiftdeps","swiftmodule":"bazel-out/darwin-fastbuild/bin/_swift_incremental/tools/build/swift/audio_video_swift_objs/swift/video.swift.partial_swiftmodule"}}
1 change: 1 addition & 0 deletions tools/build/swift/audio_video_swift.output_file_map.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"tensorflow_io/core/swift/audio.swift":{"ast-dump":"bazel-out/darwin-fastbuild/bin/tools/build/swift/audio_video_swift_objs/swift/audio.swift.ast","object":"bazel-out/darwin-fastbuild/bin/tools/build/swift/audio_video_swift_objs/swift/audio.swift.o","swiftmodule":"bazel-out/darwin-fastbuild/bin/tools/build/swift/audio_video_swift_objs/swift/audio.swift.partial_swiftmodule"},"tensorflow_io/core/swift/video.swift":{"ast-dump":"bazel-out/darwin-fastbuild/bin/tools/build/swift/audio_video_swift_objs/swift/video.swift.ast","object":"bazel-out/darwin-fastbuild/bin/tools/build/swift/audio_video_swift_objs/swift/video.swift.o","swiftmodule":"bazel-out/darwin-fastbuild/bin/tools/build/swift/audio_video_swift_objs/swift/video.swift.partial_swiftmodule"}}
1 change: 1 addition & 0 deletions tools/build/swift/audio_video_swift.runfiles/MANIFEST
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
org_tensorflow_io/tools/build/swift/audio_video_swift /private/var/tmp/_bazel_llv23/a82ad01ec0c5d2a91897f1531acdf67b/execroot/org_tensorflow_io/bazel-out/darwin-fastbuild/bin/tools/build/swift/audio_video_swift
Binary file not shown.
1 change: 1 addition & 0 deletions tools/build/swift/audio_video_swift.runfiles_manifest
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
org_tensorflow_io/tools/build/swift/audio_video_swift /private/var/tmp/_bazel_llv23/a82ad01ec0c5d2a91897f1531acdf67b/execroot/org_tensorflow_io/bazel-out/darwin-fastbuild/bin/tools/build/swift/audio_video_swift
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file added tools/build/swift/libaudio_video_swift.a
Binary file not shown.
Binary file added tools/build/swift/libaudio_video_swift.dylib
Binary file not shown.

0 comments on commit 0ee32d7

Please sign in to comment.