Skip to content
This repository has been archived by the owner on Nov 27, 2024. It is now read-only.

mev-inspect requires root privileges to run. Does not create relations in postgres #300

Open
gnikxela0328 opened this issue May 18, 2022 · 4 comments
Open

mev-inspect requires root privileges to run. Does not create relations in postgres #300

gnikxela0328 opened this issue May 18, 2022 · 4 comments

Comments

@gnikxela0328
Copy link

gnikxela0328 commented May 18, 2022 •
edited
Loading

Whenever I run the ./mev inspect ###### command, I am met with the issue of sql alchemy not being able to delete an entry from postgres as the entry does not exist

Inspecting block 12914944
Skipping virtualenv creation, as specified in config file.
INFO:mev_inspect.inspect_block:Block: 12914944 -- Total traces: 1058
INFO:mev_inspect.inspect_block:Block: 12914944 -- Total transactions: 222
INFO:mev_inspect.inspect_block:Block: 12914944 -- Returned 1057 classified traces
INFO:mev_inspect.inspect_block:Block: 12914944 -- Found 312 transfers
INFO:mev_inspect.inspect_block:Block: 12914944 -- Found 51 swaps
INFO:mev_inspect.inspect_block:Block: 12914944 -- Found 2 arbitrages
INFO:mev_inspect.inspect_block:Block: 12914944 -- Found 0 liquidations
INFO:mev_inspect.inspect_block:Block: 12914944 -- Found 2 sandwiches
INFO:mev_inspect.inspect_block:Block: 12914944 -- Found 0 punk snipes
INFO:mev_inspect.inspect_block:Block: 12914944 -- Found 0 nft trades
INFO:mev_inspect.inspect_block:Writing data
Traceback (most recent call last):
  File "/home/flashbot/.local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1771, in _execute_context
    self.dialect.do_execute(
  File "/home/flashbot/.local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 717, in do_execute
    cursor.execute(statement, parameters)
psycopg2.errors.UndefinedTable: relation "blocks" does not exist
LINE 2:         DELETE FROM blocks
                            ^


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/flashbot/.local/lib/python3.9/site-packages/click/core.py", line 1137, in __call__
    return self.main(*args, **kwargs)
  File "/home/flashbot/.local/lib/python3.9/site-packages/click/core.py", line 1062, in main
    rv = self.invoke(ctx)
  File "/home/flashbot/.local/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/flashbot/.local/lib/python3.9/site-packages/click/core.py", line 763, in invoke
    return __callback(*args, **kwargs)
  File "/app/mev_inspect/concurrency.py", line 18, in wrapper
    loop.run_until_complete(f(*args, **kwargs))
  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 647, in run_until_complete
    return future.result()
  File "/app/cli.py", line 45, in inspect_block_command
    await inspector.inspect_single_block(
  File "/app/mev_inspect/inspector.py", line 56, in inspect_single_block
    return await inspect_block(
  File "/app/mev_inspect/inspect_block.py", line 68, in inspect_block
    await inspect_many_blocks(
  File "/app/mev_inspect/inspect_block.py", line 173, in inspect_many_blocks
    delete_blocks(inspect_db_session, after_block_number, before_block_number)
  File "/app/mev_inspect/crud/blocks.py", line 13, in delete_blocks
    db_session.execute(
  File "/home/flashbot/.local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 1689, in execute
    result = conn._execute_20(statement, params or {}, execution_options)
  File "/home/flashbot/.local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1583, in _execute_20
    return meth(self, args_10style, kwargs_10style, execution_options)
  File "/home/flashbot/.local/lib/python3.9/site-packages/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection
    return connection._execute_clauseelement(
  File "/home/flashbot/.local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1452, in _execute_clauseelement
    ret = self._execute_context(
  File "/home/flashbot/.local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1814, in _execute_context
    self._handle_dbapi_exception(
  File "/home/flashbot/.local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1995, in _handle_dbapi_exception
    util.raise_(
  File "/home/flashbot/.local/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 207, in raise_
    raise exception
  File "/home/flashbot/.local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1771, in _execute_context
    self.dialect.do_execute(
  File "/home/flashbot/.local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 717, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedTable) relation "blocks" does not exist
LINE 2:         DELETE FROM blocks
                            ^

[SQL: 
        DELETE FROM blocks
        WHERE
            block_number >= %(after_block_number)s AND
            block_number < %(before_block_number)s
        ]
[parameters: {'after_block_number': 12914944, 'before_block_number': 12914945}]
(Background on this error at: https://sqlalche.me/e/14/f405)
command terminated with exit code 1

Upon logging into the postgres database using ./mev db, there are no visible relations in the mev_inspect database. Is there a workaround to this or am I missing a certain configuration?
@tmikulin
Copy link
Contributor

did you run the migrations?

@gnikxela0328
Copy link
Author

I ran the migrations and it fixed the issue of the missing table. However, now a new problem has occured :

Inspecting block 12914944
Skipping virtualenv creation, as specified in config file.
INFO:mev_inspect.inspect_block:Block: 12914944 -- Total traces: 1058
INFO:mev_inspect.inspect_block:Block: 12914944 -- Total transactions: 222
INFO:mev_inspect.inspect_block:Block: 12914944 -- Returned 1057 classified traces
INFO:mev_inspect.inspect_block:Block: 12914944 -- Found 312 transfers
INFO:mev_inspect.inspect_block:Block: 12914944 -- Found 51 swaps
INFO:mev_inspect.inspect_block:Block: 12914944 -- Found 2 arbitrages
INFO:mev_inspect.inspect_block:Block: 12914944 -- Found 0 liquidations
INFO:mev_inspect.inspect_block:Block: 12914944 -- Found 2 sandwiches
INFO:mev_inspect.inspect_block:Block: 12914944 -- Found 0 punk snipes
INFO:mev_inspect.inspect_block:Block: 12914944 -- Found 0 nft trades
INFO:mev_inspect.inspect_block:Writing data
Traceback (most recent call last):
  File "/home/flashbot/.local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1771, in _execute_context
    self.dialect.do_execute(
  File "/home/flashbot/.local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 717, in do_execute
    cursor.execute(statement, parameters)
psycopg2.errors.NotNullViolation: null value in column "miner_payment_usd" of relation "mev_summary" violates not-null constraint
DETAIL:  Failing row contains (null, 12914944, 2021-07-28 14:28:24, null, 0x448245bf1a507b73516c4eeee01611927dada6610bf26d403012f2e66800d8..., arbitrage, null, null, 317534, 0, 712505009991347209, 2243869979250, 0x5a0b54d5dc17e0aadc383d2db43b0a0d3e029c4c, 0, null, {uniswap_v2,uniswap_v3}).


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/flashbot/.local/lib/python3.9/site-packages/click/core.py", line 1137, in __call__
    return self.main(*args, **kwargs)
  File "/home/flashbot/.local/lib/python3.9/site-packages/click/core.py", line 1062, in main
    rv = self.invoke(ctx)
  File "/home/flashbot/.local/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/flashbot/.local/lib/python3.9/site-packages/click/core.py", line 763, in invoke
    return __callback(*args, **kwargs)
  File "/app/mev_inspect/concurrency.py", line 18, in wrapper
    loop.run_until_complete(f(*args, **kwargs))
  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 647, in run_until_complete
    return future.result()
  File "/app/cli.py", line 45, in inspect_block_command
    await inspector.inspect_single_block(
  File "/app/mev_inspect/inspector.py", line 56, in inspect_single_block
    return await inspect_block(
  File "/app/mev_inspect/inspect_block.py", line 68, in inspect_block
    await inspect_many_blocks(
  File "/app/mev_inspect/inspect_block.py", line 230, in inspect_many_blocks
    update_summary_for_block_range(
  File "/app/mev_inspect/crud/summary.py", line 156, in update_summary_for_block_range
    _insert_into_summary_for_block_range(
  File "/app/mev_inspect/crud/summary.py", line 187, in _insert_into_summary_for_block_range
    db_session.execute(
  File "/home/flashbot/.local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 1689, in execute
    result = conn._execute_20(statement, params or {}, execution_options)
  File "/home/flashbot/.local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1583, in _execute_20
    return meth(self, args_10style, kwargs_10style, execution_options)
  File "/home/flashbot/.local/lib/python3.9/site-packages/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection
    return connection._execute_clauseelement(
  File "/home/flashbot/.local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1452, in _execute_clauseelement
    ret = self._execute_context(
  File "/home/flashbot/.local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1814, in _execute_context
    self._handle_dbapi_exception(
  File "/home/flashbot/.local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1995, in _handle_dbapi_exception
    util.raise_(
  File "/home/flashbot/.local/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 207, in raise_
    raise exception
  File "/home/flashbot/.local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1771, in _execute_context
    self.dialect.do_execute(
  File "/home/flashbot/.local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 717, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.IntegrityError: (psycopg2.errors.NotNullViolation) null value in column "miner_payment_usd" of relation "mev_summary" violates not-null constraint
DETAIL:  Failing row contains (null, 12914944, 2021-07-28 14:28:24, null, 0x448245bf1a507b73516c4eeee01611927dada6610bf26d403012f2e66800d8..., arbitrage, null, null, 317534, 0, 712505009991347209, 2243869979250, 0x5a0b54d5dc17e0aadc383d2db43b0a0d3e029c4c, 0, null, {uniswap_v2,uniswap_v3}).

[SQL: 
INSERT INTO mev_summary (
    SELECT
        NULL,
        a.block_number,
        b.block_timestamp,
        NULL AS protocol,
        a.transaction_hash,
        'arbitrage' AS type,
        (
            (
                SELECT usd_price
                FROM prices
                WHERE
                    token_address = a.profit_token_address
                    AND timestamp <= b.block_timestamp
                ORDER BY timestamp DESC
                LIMIT 1
            ) * a.profit_amount / POWER(10, profit_token.decimals)
        ) AS gross_profit_usd,
        (
            (
                ((mp.gas_used * mp.gas_price) + mp.coinbase_transfer) /
                POWER(10, 18)
            ) * 
            (
                SELECT usd_price
                FROM prices p
                WHERE
                    p.timestamp <= b.block_timestamp
                    AND p.token_address = '0xeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee'
                ORDER BY p.timestamp DESC
                LIMIT 1
            )
        ) AS miner_payment_usd,
        mp.gas_used,
        mp.gas_price,
        mp.coinbase_transfer,
        mp.gas_price_with_coinbase_transfer,
        mp.miner_address,
        mp.base_fee_per_gas,
        ct.error as error,
        a.protocols
    FROM arbitrages a
    JOIN blocks b ON b.block_number = a.block_number
    JOIN tokens profit_token ON profit_token.token_address = a.profit_token_address
    JOIN classified_traces ct ON
        ct.block_number = a.block_number AND
        ct.transaction_hash = a.transaction_hash
    JOIN miner_payments mp ON
        mp.block_number = a.block_number AND
        mp.transaction_hash = a.transaction_hash
    WHERE
        b.block_number >= %(after_block_number)s
        AND b.block_number < %(before_block_number)s
        AND ct.trace_address = '{}'
        AND NOT EXISTS (
            SELECT 1
            FROM sandwiches front_sandwich
            WHERE 
                front_sandwich.block_number = a.block_number AND
                front_sandwich.frontrun_swap_transaction_hash = a.transaction_hash
        )
        AND NOT EXISTS (
            SELECT 1
            FROM sandwiches back_sandwich
            WHERE
                back_sandwich.block_number = a.block_number AND
                back_sandwich.backrun_swap_transaction_hash = a.transaction_hash
        )
)
]
[parameters: {'after_block_number': 12914944, 'before_block_number': 12914945}]
(Background on this error at: https://sqlalche.me/e/14/gkpj)
command terminated with exit code 1

Does this have to do with the alembic configuration?

@gnikxela0328
Copy link
Author

I manually dropped the not null constraint on the miner_payment_usd column using:

alter table mev_summary alter column miner_payment_usd drop not null;

If anyone has a comment on the above error, please let me know. I will close the issue.

Thanks @tmikulin

@AmbitionCX
Copy link

Hello, I met the same issue.

It seems you need to obtain the price value by command ./mev prices fetch-all

However, I got the connection refused error message when trying to fetch prices.

Have you resolved this problem?

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@tmikulin @AmbitionCX @gnikxela0328