Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Redirect Manager: Large-Scale Import Optimization #3501

Open
YegorKozlov opened this issue Jan 3, 2025 · 0 comments
Open

Redirect Manager: Large-Scale Import Optimization #3501

YegorKozlov opened this issue Jan 3, 2025 · 0 comments
Assignees

Comments

@YegorKozlov
Copy link
Contributor

The current implementation stores redirects as direct children under the caconfig parent, creating performance issues during bulk import from Excel.

/conf/my-site/settings/redirects
  - redirect-1
  - redirect-2
  ...
  - redirect-10000

For example, importing 50K redirects from the attached sample takes approximately an hour on a standard development machine.

Proposed Solution:

Implement a sharding mechanism to optimize redirect storage and improve import performance:

/conf/my-site/settings/redirects
  + shard1
	- redirect-1
	...
	- redirect-1000
  + shard2
	- redirect-1001
	...
	- redirect-2000
  + shard3
	- redirect-2001
	...
	- redirect-3000
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant