Support Document Level Security for non-Windows hosted SMB Network Shares

[josephschultz-expedient]

Problem Description

There are a number of storage platforms that are able to present Windows (SMB) shares. Most can connect with Active Directory for user/group permissions. In this case, a Windows Share is presented by a Ctera network filer.

The issue is that while the connector is able to properly access and retrieve the hosted documents, document level security is not possible due to the requirement for remote Powershell execution on the server hosting the share. This is not possible with non-Windows platforms.

Proposed Solution

The ability to remotely retrieve permission information without the remote execution requirements

Additional Context

Snippet from the Connector logs

[FMWK][20:30:20][CRITICAL] [Connector id: ###, index name: .search-acl-filter-test-smb-connector, Sync job id: G-ASu5IBdIWPRlBBeNaq] Document extractor failed
Traceback (most recent call last):
  File "/app/lib/python3.10/site-packages/urllib3/connection.py", line 196, in _new_conn
    sock = connection.create_connection(
  File "/app/lib/python3.10/site-packages/urllib3/util/connection.py", line 85, in create_connection
    raise err
  File "/app/lib/python3.10/site-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/app/lib/python3.10/site-packages/urllib3/connectionpool.py", line 789, in urlopen
    response = self._make_request(
  File "/app/lib/python3.10/site-packages/urllib3/connectionpool.py", line 495, in _make_request
    conn.request(
  File "/app/lib/python3.10/site-packages/urllib3/connection.py", line 398, in request
    self.endheaders()
  File "/usr/lib/python3.10/http/client.py", line 1278, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.10/http/client.py", line 1038, in _send_output
    self.send(msg)
  File "/usr/lib/python3.10/http/client.py", line 976, in send
    self.connect()
  File "/app/lib/python3.10/site-packages/urllib3/connection.py", line 236, in connect
    self.sock = self._new_conn()
  File "/app/lib/python3.10/site-packages/urllib3/connection.py", line 211, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f86fbadd4b0>: Failed to establish a new connection: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/app/lib/python3.10/site-packages/requests/adapters.py", line 667, in send
    resp = conn.urlopen(
  File "/app/lib/python3.10/site-packages/urllib3/connectionpool.py", line 843, in urlopen
    retries = retries.increment(
  File "/app/lib/python3.10/site-packages/urllib3/util/retry.py", line 519, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='10.20.1.229', port=5985): Max retries exceeded with url: /wsman (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f86fbadd4b0>: Failed to establish a new connection: [Errno 111] Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/connectors/sources/network_drive.py", line 801, in get_access_control
    users_info = await asyncio.to_thread(self.security_info.fetch_users)
  File "/usr/lib/python3.10/asyncio/threads.py", line 25, in to_thread
    return await loop.run_in_executor(None, func_call)
  File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/app/connectors/sources/network_drive.py", line 247, in fetch_users
    users = self.session.run_ps(GET_USERS_COMMAND)
  File "/app/lib/python3.10/site-packages/winrm/__init__.py", line 53, in run_ps
    rs = self.run_cmd('powershell -encodedcommand {0}'.format(encoded_ps))
  File "/app/lib/python3.10/site-packages/winrm/__init__.py", line 40, in run_cmd
    shell_id = self.protocol.open_shell()
  File "/app/lib/python3.10/site-packages/winrm/protocol.py", line 166, in open_shell
    res = self.send_message(xmltodict.unparse(req))
  File "/app/lib/python3.10/site-packages/winrm/protocol.py", line 243, in send_message
    resp = self.transport.send_message(message)
  File "/app/lib/python3.10/site-packages/winrm/transport.py", line 309, in send_message
    self.build_session()
  File "/app/lib/python3.10/site-packages/winrm/transport.py", line 292, in build_session
    self.setup_encryption()
  File "/app/lib/python3.10/site-packages/winrm/transport.py", line 298, in setup_encryption
    self._send_message_request(prepared_request, '')
  File "/app/lib/python3.10/site-packages/winrm/transport.py", line 327, in _send_message_request
    response = self.session.send(prepared_request, timeout=self.read_timeout_sec)
  File "/app/lib/python3.10/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "/app/lib/python3.10/site-packages/requests/adapters.py", line 700, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='10.20.1.229', port=5985): Max retries exceeded with url: /wsman (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f86fbadd4b0>: Failed to establish a new connection: [Errno 111] Connection refused'))

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/app/connectors/es/sink.py", line 496, in run
    await self.get_access_control_docs(generator)
  File "/app/connectors/es/sink.py", line 733, in get_access_control_docs
    async for doc in generator:
  File "/app/connectors/logger.py", line 247, in __anext__
    return await self.gen.__anext__()
  File "/app/connectors/es/sink.py", line 523, in _decorate_with_metrics_span
    async for doc in generator:
  File "/app/connectors/sync_job_runner.py", line 510, in generator
    async for doc in self.data_provider.get_access_control():
  File "/app/connectors/sources/network_drive.py", line 810, in get_access_control
    raise ConnectionError(msg) from exception
requests.exceptions.ConnectionError: Something went wrong

[seanstory]

Thanks for filing, @josephschultz-expedient !
We'll work with our Product Management team to prioritize this enhancement request.

If you already have a support relationship with Elastic, don't hesitate to reach out through your contact to advocate for this feature.