Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Messages not delivered #18095

Open
head47 opened this issue Jan 18, 2025 · 0 comments
Open

Messages not delivered #18095

head47 opened this issue Jan 18, 2025 · 0 comments

Comments

@head47
Copy link

head47 commented Jan 18, 2025

Description

Version: synapse Docker image, latest (v1.122.0)

None of my messages are delivered since the latest update.

Steps to reproduce

  • attempt to send any message
  • see the client report it as not sent
  • check with the recipient that they haven't received the message

Homeserver

my personal homeserver

Synapse Version

1.122.0

Installation Method

Docker (matrixdotorg/synapse)

Database

PostgreSQL 17.2, single server, no portdb, never restored a backup

Workers

Single process

Platform

Arch Linux

Configuration

Presence is on

Relevant log output

2025-01-18 11:22:53,701 - synapse.http.server - 146 - ERROR - PUT-9046 - Failed handle request via 'RoomSendEventRestServlet': <XForwardedForRequest at 0x726483bc63c0 method='PUT' uri='/_matrix/client/v3/rooms/!REDACTED/send/m.room.encrypted/REDACTED' clientproto='HTTP/1.1' site='8008'>
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/twisted/internet/defer.py", line 2010, in _inlineCallbacks
    result = context.run(
             ^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/twisted/python/failure.py", line 549, in throwExceptionIntoGenerator
    return g.throw(self.value.with_traceback(self.tb))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/synapse/rest/client/room.py", line 415, in _do
    ) = await self.event_creation_handler.create_and_send_nonmember_event(
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/synapse/handlers/message.py", line 1025, in create_and_send_nonmember_event
    async with self._worker_lock_handler.acquire_read_write_lock(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/synapse/handlers/worker_lock.py", line 224, in __aenter__
    lock = await self.store.try_acquire_read_write_lock(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/synapse/storage/databases/main/lock.py", line 224, in try_acquire_read_write_lock
    lock = await self.db_pool.runInteraction(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/synapse/storage/database.py", line 952, in runInteraction
    return await delay_cancellation(_runInteraction())
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/twisted/internet/defer.py", line 2010, in _inlineCallbacks
    result = context.run(
             ^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/twisted/python/failure.py", line 549, in throwExceptionIntoGenerator
    return g.throw(self.value.with_traceback(self.tb))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/synapse/storage/database.py", line 918, in _runInteraction
    result: R = await self.runWithConnection(
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/synapse/storage/database.py", line 1047, in runWithConnection
    return await make_deferred_yieldable(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/twisted/python/threadpool.py", line 269, in inContext
    result = inContext.theWork()  # type: ignore[attr-defined]
             ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/twisted/python/threadpool.py", line 285, in <lambda>
    inContext.theWork = lambda: context.call(  # type: ignore[attr-defined]
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/twisted/python/context.py", line 117, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/twisted/python/context.py", line 82, in callWithContext
    return func(*args, **kw)
           ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/twisted/enterprise/adbapi.py", line 282, in _runWithConnection
    result = func(conn, *args, **kw)
             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/synapse/storage/database.py", line 1040, in inner_func
    return func(db_conn, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/synapse/storage/database.py", line 780, in new_transaction
    r = func(cursor, *args, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/synapse/storage/databases/main/lock.py", line 252, in _try_acquire_read_write_lock_txn
    self.db_pool.simple_insert_txn(
  File "/usr/local/lib/python3.12/site-packages/synapse/storage/database.py", line 1098, in simple_insert_txn
    txn.execute(sql, vals)
  File "/usr/local/lib/python3.12/site-packages/synapse/storage/database.py", line 426, in execute
    self._do_execute(self.txn.execute, sql, parameters)
  File "/usr/local/lib/python3.12/site-packages/synapse/storage/database.py", line 488, in _do_execute
    return func(sql, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
psycopg2.errors.UndefinedFile: could not load library "/usr/lib/postgresql/plpgsql.so": /usr/lib/postgresql/plpgsql.so: undefined symbol: ReleaseAllPlanCacheRefsInOwner
2025-01-18 11:22:57,730 - synapse.storage.txn - 787 - WARNING - maybe_backfill_in_the_background-40 - [TXN OPERROR] {try_acquire_read_write_lock-f88f} could not load library "/usr/lib/postgresql/plpgsql.so": /usr/lib/postgresql/plpgsql.so: undefined symbol: ReleaseAllPlanCacheRefsInOwner
 0/5
2025-01-18 11:22:57,730 - synapse.storage.txn - 787 - WARNING - maybe_backfill_in_the_background-40 - [TXN OPERROR] {try_acquire_read_write_lock-f88f} could not load library "/usr/lib/postgresql/plpgsql.so": /usr/lib/postgresql/plpgsql.so: undefined symbol: ReleaseAllPlanCacheRefsInOwner
 1/5
2025-01-18 11:22:57,730 - synapse.storage.txn - 787 - WARNING - maybe_backfill_in_the_background-40 - [TXN OPERROR] {try_acquire_read_write_lock-f88f} could not load library "/usr/lib/postgresql/plpgsql.so": /usr/lib/postgresql/plpgsql.so: undefined symbol: ReleaseAllPlanCacheRefsInOwner
 2/5
2025-01-18 11:22:57,731 - synapse.storage.txn - 787 - WARNING - maybe_backfill_in_the_background-40 - [TXN OPERROR] {try_acquire_read_write_lock-f88f} could not load library "/usr/lib/postgresql/plpgsql.so": /usr/lib/postgresql/plpgsql.so: undefined symbol: ReleaseAllPlanCacheRefsInOwner
 3/5
2025-01-18 11:22:57,731 - synapse.storage.txn - 787 - WARNING - maybe_backfill_in_the_background-40 - [TXN OPERROR] {try_acquire_read_write_lock-f88f} could not load library "/usr/lib/postgresql/plpgsql.so": /usr/lib/postgresql/plpgsql.so: undefined symbol: ReleaseAllPlanCacheRefsInOwner
 4/5
2025-01-18 11:22:57,731 - synapse.storage.txn - 787 - WARNING - maybe_backfill_in_the_background-40 - [TXN OPERROR] {try_acquire_read_write_lock-f88f} could not load library "/usr/lib/postgresql/plpgsql.so": /usr/lib/postgresql/plpgsql.so: undefined symbol: ReleaseAllPlanCacheRefsInOwner
2025-01-18 11:22:57,731 - synapse.metrics.background_process_metrics - 253 - ERROR - maybe_backfill_in_the_background-40 - Background process 'maybe_backfill_in_the_background' threw an exception
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/synapse/metrics/background_process_metrics.py", line 251, in run
    return await func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/synapse/logging/opentracing.py", line 922, in _wrapper
    return await func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/synapse/logging/opentracing.py", line 922, in _wrapper
    return await func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/synapse/handlers/federation.py", line 235, in maybe_backfill
    async with self._worker_locks.acquire_read_write_lock(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/synapse/handlers/worker_lock.py", line 224, in __aenter__
    lock = await self.store.try_acquire_read_write_lock(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/synapse/storage/databases/main/lock.py", line 224, in try_acquire_read_write_lock
    lock = await self.db_pool.runInteraction(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/synapse/storage/database.py", line 952, in runInteraction
    return await delay_cancellation(_runInteraction())
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/twisted/internet/defer.py", line 2010, in _inlineCallbacks
    result = context.run(
             ^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/twisted/python/failure.py", line 549, in throwExceptionIntoGenerator
    return g.throw(self.value.with_traceback(self.tb))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/synapse/storage/database.py", line 918, in _runInteraction
    result: R = await self.runWithConnection(
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/synapse/storage/database.py", line 1047, in runWithConnection
    return await make_deferred_yieldable(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/twisted/python/threadpool.py", line 269, in inContext
    result = inContext.theWork()  # type: ignore[attr-defined]
             ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/twisted/python/threadpool.py", line 285, in <lambda>
    inContext.theWork = lambda: context.call(  # type: ignore[attr-defined]
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/twisted/python/context.py", line 117, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/twisted/python/context.py", line 82, in callWithContext
    return func(*args, **kw)
           ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/twisted/enterprise/adbapi.py", line 282, in _runWithConnection
    result = func(conn, *args, **kw)
             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/synapse/storage/database.py", line 1040, in inner_func
    return func(db_conn, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/synapse/storage/database.py", line 780, in new_transaction
    r = func(cursor, *args, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/synapse/storage/databases/main/lock.py", line 252, in _try_acquire_read_write_lock_txn
    self.db_pool.simple_insert_txn(
  File "/usr/local/lib/python3.12/site-packages/synapse/storage/database.py", line 1098, in simple_insert_txn
    txn.execute(sql, vals)
  File "/usr/local/lib/python3.12/site-packages/synapse/storage/database.py", line 426, in execute
    self._do_execute(self.txn.execute, sql, parameters)
  File "/usr/local/lib/python3.12/site-packages/synapse/storage/database.py", line 488, in _do_execute
    return func(sql, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
psycopg2.errors.UndefinedFile: could not load library "/usr/lib/postgresql/plpgsql.so": /usr/lib/postgresql/plpgsql.so: undefined symbol: ReleaseAllPlanCacheRefsInOwner

Anything else that would be useful to know?

No response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant