Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: optimize find_all_tags_for with caching and indexing #127

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

magajh
Copy link
Contributor

@magajh magajh commented Jan 7, 2025

Description

This is a backport of the changes made in #126

This PR introduces optimizations to the find_all_tags_for method and related functionality to improve performance when working with large datasets in the Tag model. Specifically:

  1. Caching for ContentType Lookups:

    • Added cache to store ContentType objects for frequently used model names. This reduces redundant database queries for ContentType.objects.get(), especially in high-traffic scenarios or repeated calls.
  2. Database Indexes:

    • Added an index on the target_type and target_object_id fields in the Tag model.
    • This index improves query performance for operations that filter tags by target_type and target_object_id, particularly in the find_all_tags_for method.
  3. Optimized Query Logic:

    • Removed the list() call on target.values_list("id", flat=True) in the find_all_tags_for method.
    • This ensures the query is fully managed by the database, avoiding unnecessary memory overhead and improving scalability for large result sets.

Why These Changes Were Made

  • The find_all_tags_for method was identified as a performance bottleneck, particularly for large datasets in the Tag table (~6 million records).
  • Redundant ContentType lookups were causing unnecessary database queries.
  • Queries filtering by target_type and target_object_id lacked an index, leading to slower performance on large datasets.

@magajh magajh requested a review from a team as a code owner January 7, 2025 13:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant