Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DOC: What is the maximum chunk size returned from SemanticChunker.split_documents() #28250

Open
2 tasks done
abhipandey09 opened this issue Nov 21, 2024 · 0 comments
Open
2 tasks done
Labels
🤖:docs Changes to documentation and examples, like .md, .rst, .ipynb files. Changes to the docs/ folder

Comments

@abhipandey09
Copy link

URL

No response

Checklist

  • I added a very descriptive title to this issue.
  • I included a link to the documentation page I am referring to (if applicable).

Issue with current documentation:

I need to know on the maximum chunk size that can be return from SemanticChunker.split_documents() for large documents.
Can it be more than 8k token as i need to send the chunk for embedding, chunk with more than 8k token will fail with Azure embedding model.
Need Help!!

Idea or request for content:

Documentation should cleary explain this

@dosubot dosubot bot added the 🤖:docs Changes to documentation and examples, like .md, .rst, .ipynb files. Changes to the docs/ folder label Nov 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🤖:docs Changes to documentation and examples, like .md, .rst, .ipynb files. Changes to the docs/ folder
Projects
None yet
Development

No branches or pull requests

1 participant