You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
MacOS Monterey, 12.6
MacBook Pro M1
Conda (miniconda) environment 4.12.0
python 3.9.16 [email protected] installed via brew at /opt/homebrew/var (tried [email protected] as well)
pymongo 4.3.3 (I've tested 3.11.0 and 3.12.0 as well)
pandas 1.1.5
numpy 1.24.2
Description of problem and/or code sample that reproduces the issue
The code works just fine when I use up to 3 of the M1's 8 cores. However, once I start using 4 or more, I get the following error:
File "/Users/username/Desktop/Projects/path/to/file.py", line 248, in function1
var1, var2, var3 = function2(
File "/Users/username/Desktop/Projects/path/to/file2.py", line 369, in function2
var11, var12 = function3(input1, input2, input3, input4,
File "/Users/username/Desktop/Projects/path/to/file3.py", line 79, in function3
close_df = get_df_1m(start, end, ticker)
File "/Users/username/Desktop/Projects/path/to/file4.py", line 57, in get_df_1m
arc.initialize_library(
File "/Users/username/miniconda3/envs/env1/lib/python3.9/site-packages/arctic/decorators.py", line 49, in f_retry
return f(*args, **kwargs)
File "/Users/username/miniconda3/envs/env1/lib/python3.9/site-packages/arctic/arctic.py", line 310, in initialize_library
self._cache.append('list_libraries', self._sanitize_lib_name(library))
AttributeError: 'NoneType' object has no attribute 'append'
And that in turn eventually leads to the following error due to (I think) the database shutting down:
File "/Users/username/Desktop/Projects/path/to/file.py", line 214, in function1
var21, var22, var23 = function4(
File "/Users/username/Desktop/Projects/path/to/file2.py", line 689, in function4
close_df = get_df_1m(start, end, ticker)
File "/Users/username/Desktop/Projects/path/to/file3.py", line 57, in get_df_1m
arc.initialize_library(
File "/Users/username/miniconda3/envs/env1/lib/python3.9/site-packages/arctic/decorators.py", line 49, in f_retry
return f(*args, **kwargs)
File "/Users/username/miniconda3/envs/env1/lib/python3.9/site-packages/arctic/arctic.py", line 297, in initialize_library
lib = ArcticLibraryBinding(self, library)
File "/Users/username/miniconda3/envs/env1/lib/python3.9/site-packages/arctic/arctic.py", line 501, in __init__
self._curr_conn = self.arctic._conn
File "/Users/username/miniconda3/envs/env1/lib/python3.9/site-packages/arctic/decorators.py", line 49, in f_retry
return f(*args, **kwargs)
File "/Users/username/miniconda3/envs/env1/lib/python3.9/site-packages/arctic/arctic.py", line 152, in _conn
self._cache = Cache(self.__conn)
File "/Users/username/miniconda3/envs/env1/lib/python3.9/site-packages/arctic/_cache.py", line 26, in __init__
if cache_col not inself._cachedb.list_collection_names():
File "/Users/username/miniconda3/envs/env1/lib/python3.9/site-packages/pymongo/database.py", line 959, in list_collection_names
return [result["name"] forresultin self.list_collections(session=session, **kwargs)]
File "/Users/username/miniconda3/envs/env1/lib/python3.9/site-packages/pymongo/database.py", line 911, in list_collections
return self.__client._retryable_read(_cmd, read_pref, session)
File "/Users/username/miniconda3/envs/env1/lib/python3.9/site-packages/pymongo/_csot.py", line 105, in csot_wrapper
return func(self, *args, **kwargs)
File "/Users/username/miniconda3/envs/env1/lib/python3.9/site-packages/pymongo/mongo_client.py", line 1441, in _retryable_read
server = self._select_server(read_pref, session, address=address)
File "/Users/username/miniconda3/envs/env1/lib/python3.9/site-packages/pymongo/mongo_client.py", line 1257, in _select_server
server = topology.select_server(server_selector)
File "/Users/username/miniconda3/envs/env1/lib/python3.9/site-packages/pymongo/topology.py", line 272, in select_server
server = self._select_server(selector, server_selection_timeout, address)
File "/Users/username/miniconda3/envs/env1/lib/python3.9/site-packages/pymongo/topology.py", line 261, in _select_server
servers = self.select_servers(selector, server_selection_timeout, address)
File "/Users/username/miniconda3/envs/env1/lib/python3.9/site-packages/pymongo/topology.py", line 223, in select_servers
server_descriptions = self._select_servers_loop(selector, server_timeout, address)
File "/Users/username/miniconda3/envs/env1/lib/python3.9/site-packages/pymongo/topology.py", line 238, in _select_servers_loop
raise ServerSelectionTimeoutError(
pymongo.errors.ServerSelectionTimeoutError: localhost:27017: [Errno 61] Connection refused, Timeout: 30.0s, Topology Description: <TopologyDescription id: 6414395fce485adfeec51165, topology_type: Unknown, servers: [<ServerDescription ('localhost', 27017) server_type: Unknown, rtt: None, error=AutoReconnect('localhost:27017: [Errno 61] Connection refused')>]>
Exception ServerSelectionTimeoutError Occured
Any help on how I can get this working with all 8 threads is much appreciated.
The text was updated successfully, but these errors were encountered:
JakobLS
changed the title
"AttributeError: 'NoneType' object has no attribute 'append'" using 4 or more threads. 1, 2 & 3 threads works fine.
"AttributeError: 'NoneType' object has no attribute 'append'" using 4 or more threads. Fewer threads works fine.
Mar 17, 2023
Arctic Version
However, I've tested other versions as well.
Arctic Store
Platform and version
MacOS Monterey, 12.6
MacBook Pro M1
Conda (miniconda) environment 4.12.0
python 3.9.16
[email protected] installed via brew at
/opt/homebrew/var
(tried [email protected] as well)pymongo 4.3.3 (I've tested 3.11.0 and 3.12.0 as well)
pandas 1.1.5
numpy 1.24.2
Description of problem and/or code sample that reproduces the issue
The code works just fine when I use up to 3 of the M1's 8 cores. However, once I start using 4 or more, I get the following error:
And that in turn eventually leads to the following error due to (I think) the database shutting down:
Function
get_df_1m
looks like this:And the way I initiate multiprocessing is as follows:
Any help on how I can get this working with all 8 threads is much appreciated.
The text was updated successfully, but these errors were encountered: