Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unable to install sagemaker-training on Windows #110

Open
martinlyra opened this issue Oct 21, 2021 · 4 comments
Open

Unable to install sagemaker-training on Windows #110

martinlyra opened this issue Oct 21, 2021 · 4 comments

Comments

@martinlyra
Copy link

Describe the bug
Whenever I try to run pip install sagemaker-training, the wheel step fails because there is a linker error when it is time to compile the .c files.

To reproduce
Run pip install sagemaker-training on a Windows 10 machine that has Visual Studo 2019 or a MSVC toolchain installed.

Expected behavior
Compile just fine. Or alert the user of what dependecies the user might be missing.

Screenshots or logs
The log is dense, so the probably most interest part of the log is the linker error unable to find strndup as seen near to end of log.

Building wheels for collected packages: sagemaker-training
  Building wheel for sagemaker-training (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: 'c:\program files\python39\python.exe' -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\matly\\AppData\\Local\\Temp\\pip-install-0ar0b3rh\\sagemaker-training_8676a0ae965040bd99685308a5519a1d\\setup.py'"'"'; __file__='"'"'C:\\Users\\matly\\AppData\\Local\\Temp\\pip-install-0ar0b3rh\\sagemaker-training_8676a0ae965040bd99685308a5519a1d\\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d 'C:\Users\matly\AppData\Local\Temp\pip-wheel-q4lstx5x'
       cwd: C:\Users\matly\AppData\Local\Temp\pip-install-0ar0b3rh\sagemaker-training_8676a0ae965040bd99685308a5519a1d\
  Complete output (119 lines):
  c:\program files\python39\lib\site-packages\setuptools\dist.py:642: UserWarning: Usage of dash-separated 'description-file' will not be supported in future versions. Please use the underscore name 'description_file' instead
    warnings.warn(
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build\lib.win-amd64-3.9
  creating build\lib.win-amd64-3.9\sagemaker_training
  copying src\sagemaker_training\content_types.py -> build\lib.win-amd64-3.9\sagemaker_training
  copying src\sagemaker_training\encoders.py -> build\lib.win-amd64-3.9\sagemaker_training
  copying src\sagemaker_training\entry_point.py -> build\lib.win-amd64-3.9\sagemaker_training
  copying src\sagemaker_training\environment.py -> build\lib.win-amd64-3.9\sagemaker_training
  copying src\sagemaker_training\errors.py -> build\lib.win-amd64-3.9\sagemaker_training
  copying src\sagemaker_training\files.py -> build\lib.win-amd64-3.9\sagemaker_training
  copying src\sagemaker_training\functions.py -> build\lib.win-amd64-3.9\sagemaker_training
  copying src\sagemaker_training\intermediate_output.py -> build\lib.win-amd64-3.9\sagemaker_training
  copying src\sagemaker_training\logging_config.py -> build\lib.win-amd64-3.9\sagemaker_training
  copying src\sagemaker_training\mapping.py -> build\lib.win-amd64-3.9\sagemaker_training
  copying src\sagemaker_training\modules.py -> build\lib.win-amd64-3.9\sagemaker_training
  copying src\sagemaker_training\mpi.py -> build\lib.win-amd64-3.9\sagemaker_training
  copying src\sagemaker_training\params.py -> build\lib.win-amd64-3.9\sagemaker_training
  copying src\sagemaker_training\process.py -> build\lib.win-amd64-3.9\sagemaker_training
  copying src\sagemaker_training\recordio.py -> build\lib.win-amd64-3.9\sagemaker_training
  copying src\sagemaker_training\record_pb2.py -> build\lib.win-amd64-3.9\sagemaker_training
  copying src\sagemaker_training\runner.py -> build\lib.win-amd64-3.9\sagemaker_training
  copying src\sagemaker_training\smdataparallel.py -> build\lib.win-amd64-3.9\sagemaker_training
  copying src\sagemaker_training\timeout.py -> build\lib.win-amd64-3.9\sagemaker_training
  copying src\sagemaker_training\trainer.py -> build\lib.win-amd64-3.9\sagemaker_training
  copying src\sagemaker_training\_entry_point_type.py -> build\lib.win-amd64-3.9\sagemaker_training
  copying src\sagemaker_training\__init__.py -> build\lib.win-amd64-3.9\sagemaker_training
  creating build\lib.win-amd64-3.9\sagemaker_training\cli
  copying src\sagemaker_training\cli\train.py -> build\lib.win-amd64-3.9\sagemaker_training\cli
  copying src\sagemaker_training\cli\__init__.py -> build\lib.win-amd64-3.9\sagemaker_training\cli
  running build_ext
  building 'gethostname' extension
  creating build\temp.win-amd64-3.9
  creating build\temp.win-amd64-3.9\Release
  creating build\temp.win-amd64-3.9\Release\src
  creating build\temp.win-amd64-3.9\Release\src\sagemaker_training
  creating build\temp.win-amd64-3.9\Release\src\sagemaker_training\c
  C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30037\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -Isrc/sagemaker_training/c -Ic:\program files\python39\include -Ic:\program files\python39\include -IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30037\include -IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.7.2\include\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\shared -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\winrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\cppwinrt /Tcsrc/sagemaker_training/c/gethostname.c /Fobuild\temp.win-amd64-3.9\Release\src/sagemaker_training/c/gethostname.obj -Wall -shared -export-dynamic -ldl
  cl : Command line warning D9002 : ignoring unknown option '-shared'
  cl : Command line warning D9002 : ignoring unknown option '-export-dynamic'
  cl : Command line warning D9002 : ignoring unknown option '-ldl'
  gethostname.c
  C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt\corecrt.h(212): warning C4668: '__cplusplus' is not defined as a preprocessor macro, replacing with '0' for '#if/#elif'
  C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt\corecrt_io.h(49): warning C4820: '_finddata32i64_t': '4' bytes padding added after data member 'name'
  C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt\corecrt_io.h(54): warning C4820: '_finddata64i32_t': '4' bytes padding added after data member 'attrib'
  C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt\corecrt_io.h(64): warning C4820: '__finddata64_t': '4' bytes padding added after data member 'attrib'
  C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt\corecrt_io.h(69): warning C4820: '__finddata64_t': '4' bytes padding added after data member 'name'
  C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt\time.h(36): warning C4820: '_timespec64': '4' bytes padding added after data member 'tv_nsec'
  C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt\time.h(43): warning C4820: 'timespec': '4' bytes padding added after data member 'tv_nsec'
  c:\program files\python39\include\object.h(198): warning C4820: '<unnamed-tag>': '4' bytes padding added after data member 'slot'
  c:\program files\python39\include\object.h(206): warning C4820: '<unnamed-tag>': '4' bytes padding added after data member 'flags'
  c:\program files\python39\include\cpython/object.h(228): warning C4820: '_typeobject': '4' bytes padding added after data member 'tp_flags'
  c:\program files\python39\include\cpython/object.h(272): warning C4820: '_typeobject': '4' bytes padding added after data member 'tp_version_tag'
  c:\program files\python39\include\pytime.h(123): warning C4115: 'timeval': named type definition in parentheses
  c:\program files\python39\include\cpython/bytesobject.h(15): warning C4820: '<unnamed-tag>': '7' bytes padding added after data member 'ob_sval'
  c:\program files\python39\include\cpython/bytesobject.h(65): warning C4820: '<unnamed-tag>': '4' bytes padding added after data member 'small_buffer'
  c:\program files\python39\include\cpython/unicodeobject.h(218): warning C4820: '<unnamed-tag>': '4' bytes padding added after data member 'state'
  c:\program files\python39\include\cpython/unicodeobject.h(626): warning C4820: '<unnamed-tag>': '2' bytes padding added after data member 'readonly'
  c:\program files\python39\include\longintrepr.h(88): warning C4820: '_longobject': '4' bytes padding added after data member 'ob_digit'
  c:\program files\python39\include\memoryobject.h(45): warning C4820: '<unnamed-tag>': '4' bytes padding added after data member 'flags'
  c:\program files\python39\include\memoryobject.h(62): warning C4820: '<unnamed-tag>': '4' bytes padding added after data member 'flags'
  c:\program files\python39\include\methodobject.h(40): warning C4820: 'PyMethodDef': '4' bytes padding added after data member 'ml_flags'
  c:\program files\python39\include\moduleobject.h(63): warning C4820: 'PyModuleDef_Slot': '4' bytes padding added after data member 'slot'
  c:\program files\python39\include\cpython/code.h(27): warning C4820: 'PyCodeObject': '4' bytes padding added after data member 'co_firstlineno'
  c:\program files\python39\include\cpython/code.h(64): warning C4820: 'PyCodeObject': '3' bytes padding added after data member 'co_opcache_size'
  c:\program files\python39\include\cpython/initconfig.h(16): warning C4820: '<unnamed-tag>': '4' bytes padding added after data member '_type'
  c:\program files\python39\include\cpython/initconfig.h(19): warning C4820: '<unnamed-tag>': '4' bytes padding added after data member 'exitcode'
  c:\program files\python39\include\cpython/initconfig.h(209): warning C4820: '<unnamed-tag>': '4' bytes padding added after data member 'parse_argv'
  c:\program files\python39\include\cpython/initconfig.h(364): warning C4820: '<unnamed-tag>': '4' bytes padding added after data member 'legacy_windows_stdio'
  c:\program files\python39\include\cpython/initconfig.h(374): warning C4820: '<unnamed-tag>': '4' bytes padding added after data member 'pathconfig_warnings'
  c:\program files\python39\include\cpython/initconfig.h(381): warning C4820: '<unnamed-tag>': '4' bytes padding added after data member 'module_search_paths_set'
  c:\program files\python39\include\cpython/initconfig.h(401): warning C4820: '<unnamed-tag>': '4' bytes padding added after data member 'skip_source_first_line'
  c:\program files\python39\include\cpython/initconfig.h(424): warning C4820: '<unnamed-tag>': '4' bytes padding added after data member '_isolated_interpreter'
  c:\program files\python39\include\cpython/pystate.h(65): warning C4820: '_ts': '2' bytes padding added after data member 'recursion_critical'
  c:\program files\python39\include\cpython/pystate.h(73): warning C4820: '_ts': '4' bytes padding added after data member 'use_tracing'
  c:\program files\python39\include\cpython/pystate.h(96): warning C4820: '_ts': '4' bytes padding added after data member 'gilstate_counter'
  c:\program files\python39\include\cpython/pystate.h(130): warning C4820: '_ts': '4' bytes padding added after data member 'coroutine_origin_tracking_depth'
  c:\program files\python39\include\genobject.h(33): warning C4820: '<unnamed-tag>': '7' bytes padding added after data member 'gi_running'
  c:\program files\python39\include\genobject.h(52): warning C4820: '<unnamed-tag>': '7' bytes padding added after data member 'cr_running'
  c:\program files\python39\include\genobject.h(67): warning C4820: '<unnamed-tag>': '7' bytes padding added after data member 'ag_running'
  c:\program files\python39\include\genobject.h(80): warning C4820: '<unnamed-tag>': '4' bytes padding added after data member 'ag_running_async'
  c:\program files\python39\include\descrobject.h(29): warning C4820: 'wrapperbase': '4' bytes padding added after data member 'offset'
  c:\program files\python39\include\descrobject.h(33): warning C4820: 'wrapperbase': '4' bytes padding added after data member 'flags'
  c:\program files\python39\include\structseq.h(20): warning C4820: 'PyStructSequence_Desc': '4' bytes padding added after data member 'n_in_sequence'
  c:\program files\python39\include\cpython/pyerrors.h(19): warning C4820: '<unnamed-tag>': '7' bytes padding added after data member 'suppress_context'
  c:\program files\python39\include\cpython/pyerrors.h(23): warning C4820: '<unnamed-tag>': '7' bytes padding added after data member 'suppress_context'
  c:\program files\python39\include\cpython/pyerrors.h(33): warning C4820: '<unnamed-tag>': '7' bytes padding added after data member 'suppress_context'
  c:\program files\python39\include\cpython/pyerrors.h(40): warning C4820: '<unnamed-tag>': '7' bytes padding added after data member 'suppress_context'
  c:\program files\python39\include\cpython/pyerrors.h(49): warning C4820: '<unnamed-tag>': '7' bytes padding added after data member 'suppress_context'
  c:\program files\python39\include\cpython/pyerrors.h(54): warning C4820: '<unnamed-tag>': '7' bytes padding added after data member 'suppress_context'
  c:\program files\python39\include\cpython/pyerrors.h(66): warning C4820: '<unnamed-tag>': '7' bytes padding added after data member 'suppress_context'
  c:\program files\python39\include\modsupport.h(100): warning C4820: '_PyArg_Parser': '4' bytes padding added after data member 'max'
  c:\program files\python39\include\cpython/import.h(41): warning C4820: '_frozen': '4' bytes padding added after data member 'size'
  c:\program files\python39\include\cpython/fileutils.h(54): warning C4820: '_Py_stat_struct': '4' bytes padding added after data member 'st_dev'
  c:\program files\python39\include\cpython/fileutils.h(56): warning C4820: '_Py_stat_struct': '2' bytes padding added after data member 'st_mode'
  c:\program files\python39\include\cpython/fileutils.h(60): warning C4820: '_Py_stat_struct': '4' bytes padding added after data member 'st_rdev'
  c:\program files\python39\include\cpython/fileutils.h(63): warning C4820: '_Py_stat_struct': '4' bytes padding added after data member 'st_atime_nsec'
  c:\program files\python39\include\cpython/fileutils.h(65): warning C4820: '_Py_stat_struct': '4' bytes padding added after data member 'st_mtime_nsec'
  c:\program files\python39\include\cpython/fileutils.h(69): warning C4820: '_Py_stat_struct': '4' bytes padding added after data member 'st_reparse_tag'
  src/sagemaker_training/c/gethostname.c(70): warning C4013: 'strndup' undefined; assuming extern returning int
  src/sagemaker_training/c/gethostname.c(70): warning C4047: 'initializing': 'char *' differs in levels of indirection from 'int'
  src/sagemaker_training/c/gethostname.c(90): warning C4100: 'self': unreferenced formal parameter
  src/sagemaker_training/c/gethostname.c(124): warning C4255: 'PyInit_gethostname': no function prototype given: converting '()' to '(void)'
  C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt\stdio.h(948): warning C4710: 'int printf(const char *const ,...)': function not inlined
  C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt\stdio.h(948): note: see declaration of 'printf'
  C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30037\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -Isrc/sagemaker_training/c -Ic:\program files\python39\include -Ic:\program files\python39\include -IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30037\include -IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.7.2\include\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\shared -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\winrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\cppwinrt /Tcsrc/sagemaker_training/c/jsmn.c /Fobuild\temp.win-amd64-3.9\Release\src/sagemaker_training/c/jsmn.obj -Wall -shared -export-dynamic -ldl
  cl : Command line warning D9002 : ignoring unknown option '-shared'
  cl : Command line warning D9002 : ignoring unknown option '-export-dynamic'
  cl : Command line warning D9002 : ignoring unknown option '-ldl'
  jsmn.c
  C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt\corecrt.h(212): warning C4668: '__cplusplus' is not defined as a preprocessor macro, replacing with '0' for '#if/#elif'
  C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30037\bin\HostX86\x64\link.exe /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:c:\program files\python39\libs /LIBPATH:c:\program files\python39\PCbuild\amd64 /LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30037\lib\x64 /LIBPATH:C:\Program Files (x86)\Windows Kits\NETFXSDK\4.7.2\lib\um\x64 /LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.17763.0\ucrt\x64 /LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.17763.0\um\x64 /EXPORT:PyInit_gethostname build\temp.win-amd64-3.9\Release\src/sagemaker_training/c/gethostname.obj build\temp.win-amd64-3.9\Release\src/sagemaker_training/c/jsmn.obj /OUT:build\lib.win-amd64-3.9\gethostname.cp39-win_amd64.pyd /IMPLIB:build\temp.win-amd64-3.9\Release\src/sagemaker_training/c\gethostname.cp39-win_amd64.lib
     Creating library build\temp.win-amd64-3.9\Release\src/sagemaker_training/c\gethostname.cp39-win_amd64.lib and object build\temp.win-amd64-3.9\Release\src/sagemaker_training/c\gethostname.cp39-win_amd64.exp
  gethostname.obj : error LNK2001: unresolved external symbol strndup
  build\lib.win-amd64-3.9\gethostname.cp39-win_amd64.pyd : fatal error LNK1120: 1 unresolved externals
  error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Tools\\MSVC\\14.29.30037\\bin\\HostX86\\x64\\link.exe' failed with exit code 1120
  ----------------------------------------
  ERROR: Failed building wheel for sagemaker-training
  Running setup.py clean for sagemaker-training
Failed to build sagemaker-training

System information
A description of your system.

  • SageMaker Training version: 4.0.0.
  • Windows 10 Pro N
  • Python 3.9.5
  • MSVC:
    • Compiler: Microsoft (R) C/C++ Optimizing Compiler Version 19.29.30037 for x64
    • Linker: Microsoft (R) Incremental Linker Version 14.29.30037.0

Additional context
sagemaker-training is required by a framework to be installed locally on computer for development purposes. While it is recommended to use Linux or MacOSX due to amount of libraries only compatibile with Linux, there are a goal to expand the framework to be compatible with Windows. Any help or advice is appreciated.
@Chuukwudi
Copy link

Exact same issue here.

@kiyer-godaddy kiyer-godaddy mentioned this issue Mar 17, 2023
Open
@nicholasguimaraes
Copy link

same here

@lorenzwalthert
Copy link

On macOS, it also seems not to work.

@sam-hey
Copy link

sam-hey commented Nov 12, 2024

I'm experiencing the same issue on a Mac with an M1 chip. Does anyone have a reliable workaround?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@lorenzwalthert @martinlyra @Chuukwudi @sam-hey @nicholasguimaraes