Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: optimize find_all_tags_for with caching and indexing #126

Closed
wants to merge 5 commits into from

Conversation

magajh
Copy link
Contributor

@magajh magajh commented Jan 3, 2025

Description

This PR introduces optimizations to the find_all_tags_for method and related functionality to improve performance when working with large datasets in the Tag model. Specifically:

  1. Caching for ContentType Lookups:

    • Added cache to store ContentType objects for frequently used model names. This reduces redundant database queries for ContentType.objects.get(), especially in high-traffic scenarios or repeated calls.
  2. Database Indexes:

    • Added an index on the target_type and target_object_id fields in the Tag model.
    • This index improves query performance for operations that filter tags by target_type and target_object_id, particularly in the find_all_tags_for method.
  3. Optimized Query Logic:

    • Removed the list() call on target.values_list("id", flat=True) in the find_all_tags_for method.
    • This ensures the query is fully managed by the database, avoiding unnecessary memory overhead and improving scalability for large result sets.

Why These Changes Were Made

  • The find_all_tags_for method was identified as a performance bottleneck, particularly for large datasets in the Tag table (~6 million records).
  • Redundant ContentType lookups were causing unnecessary database queries.
  • Queries filtering by target_type and target_object_id lacked an index, leading to slower performance on large datasets.

@magajh magajh requested a review from DeimerM January 3, 2025 14:46
@magajh
Copy link
Contributor Author

magajh commented Jan 8, 2025

I'm closing this PR since it will not be merged. The purpose of this PR was to provide an easier way to visualize the changes made in patch v5.0.1.

@magajh magajh closed this Jan 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant