feat: Added new max_tokens_limit values and model_names in _openai_text_completion_tokenization_details function in openai_utils.py

[tradicio]

Related Issues

• fixes Add support to gpt-3.5-turbo-0125 OpenAI model #7313

Proposed Changes:

The function _openai_text_completion_tokenization_details(model_name: str) in openai_utils.py now correctly maps the max_tokens_limit values of the gpt-3.5-turbo and gpt-4 models

How did you test it?

I have manually verified the correct max_tokens_limits in the OpenAI webpage https://platform.openai.com/docs/models/overview

Checklist

• I have read the contributors guidelines and the code of conduct
• I have updated the related issue with new insights and changes
• I added unit tests and updated the docstrings
• I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test:.
• I documented my code
• I ran pre-commit hooks and fixed any issue

[CLAassistant]

All committers have signed the CLA.

[anakin87]

Sorry, the PR should target the v1.x branch.
I changed the base to v1.x, but now there are some unrelated changes.

Can you rebase (or merge v1.x branch) in this PR?

[tradicio]

I apologize for the error. I merged v1.x branch so it should be fine now!

[anakin87]

I'm trying to fix the commit history...
give me some moments...

[anakin87]

@tradicio I managed to fix the git history, to avoid issues with the CLA.

• Some tests fail because of these new changes. Please change them.
• Please also add a release note as explained here.

@sjrl I knew you had worked on this part in the past.
May I ask for your review?

[anakin87]

Oh no! The git history is broken again.
@tradicio What I think you should do is:

git remote add upstream https://github.com/deepset-ai/haystack # maybe not needed
git rebase --interactive upstream/v1.x #here, in the editor I would drop the unrelated commits
git push --force origin v1.25.x # this is your v1.25 branch

[anakin87]

@tradicio Something unexpected happened: your commits have been dropped from the git history.

If you have your code locally, just push it again, then ping me and I'll take care of fixing git history.

[tradicio]

@tradicio Something unexpected happened: your commits have been dropped from the git history.

If you have your code locally, just push it again, then ping me and I'll take care of fixing git history.

Thanks for your help! Now I restored all the changes