How can I delete Dify Logs to make the system faster?

[pourmand1376]

Self Checks

• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• Please do not modify this template :) and fill in all the required fields.

1. Is this request related to a challenge you're experiencing? Tell me about your story.

We are actually using Dify in our system and It is working very good.

The only problem is that Dify tends to get really slow after logs are gathered. So, my question is, How can I delete Dify Workflow Run Logs without affecting anything else?

2. Additional context or comments

I am using Self-hosted Docker solution to host dify.

3. Can you help us with this feature?

• I am interested in contributing to this feature.

[dosubot[bot]]

I found a similar unsolved discussion that might be relevant to your question: How to discard logs ^[1].

Regarding your question on how to delete Dify logs to make the system faster, it's important to note that deleting logs can impact the ability to observe and mark the application's interactions. Logs are used to record all interaction processes and provide a visual operational experience for prompt engineers and LLM operators. Without these logs, it would be challenging to track user interactions, evaluate AI outputs, and make necessary improvements based on user and operator feedback. This could hinder the model's fine-tuning process and overall performance improvements ^[2][3].

Unfortunately, I wasn't able to find specific commands, scripts, or configuration options provided by Dify for clearing or managing logs, including log rotation or automated log management, within the repository. You might need to implement your own log management solution or look into Docker's log management capabilities to handle this.

If you decide to proceed with deleting logs, ensure you have a backup and understand the potential impact on your application's performance and functionality.

To continue talking to Dosu, mention @dosu.

[nowanti]

I have the same situation as you. My single workflow has millions of records and runs very slowly, but I can't find any entry to delete them.

[nmaniwa]

I'm experiencing the same issue. I use the POST: /v1/chat-messages API to run my chatbot. I've noticed that after running the chatbot, conversation logs are added to Dify's database. From what I've observed, the number of records in the following tables in Postgres has increased:

• conversations
• messages
• workflow_runs
• workflow_node_executions
• end_users (a record is created for each user)

I tried using the DELETE: /v1/conversations/:conversation_id API, but it seems that this API only performs a soft delete operation (it just changes the value of the is_deleted column).

We can log in to Dify's Postgres using the credentials defined in the .env file. So, I think a possible workaround would be to manually delete the records from the tables mentioned above.

• https://github.com/langgenius/dify/blob/0.8.3/api/.env.example#L35

If there are any cleaner solutions, I would really like to know them.

[pourmand1376]

Thanks for the answer.

I deleted this tables but that wasn't enough. I got into some error that I couldn't even see the logs on the front-end.

So I deleted this six tables. And the program is successfully working:

truncate public.workflow_runs;
truncate public.workflow_node_executions;
truncate public.conversations;
truncate public.messages;
truncate public.end_users;
truncate public.workflow_app_logs;

[pourmand1376]

I will also add this command that I've recently written.

DELETE FROM public.workflow_runs 
WHERE created_at < CURRENT_TIMESTAMP - INTERVAL '1 week';

DELETE FROM public.workflow_node_executions 
WHERE created_at < CURRENT_TIMESTAMP - INTERVAL '1 week';

DELETE FROM public.conversations 
WHERE created_at < CURRENT_TIMESTAMP - INTERVAL '1 week';

DELETE FROM public.messages 
WHERE created_at < CURRENT_TIMESTAMP - INTERVAL '1 week';

DELETE FROM public.end_users 
WHERE created_at < CURRENT_TIMESTAMP - INTERVAL '1 week';

DELETE FROM public.workflow_app_logs 
WHERE created_at < CURRENT_TIMESTAMP - INTERVAL '1 week';

But I will have to check it multiple times to see if works OK.

[kkjinping]

same question

[xubuild]

offical log delete or clear function is needed