Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

calls to unknown service tts.google_translate_say for queries that openai directs to a media player #245

Open
semisynthetic opened this issue Jul 26, 2024 · 1 comment

Comments

@semisynthetic
Copy link

Came across this one by accident. I'm seeing an odd issue when a sentence such as "Tell ..." is asked. For example, "Tell Kona she's a good dog" triggers a function call to speak to a media player, but it uses the service tts.google_translate_say. I definitely didn't define any function to use this, and I don't see any reference to this service anywhere in HA so it must be a function definition defined elsewhere else. Here's is the attempted function in response and the following error.

2024-07-26 01:58:30.194 INFO (MainThread) [custom_components.extended_openai_conversation] Response {'id': 'chatcmpl-9p8GDDnZuiFQpN0rvgGf8AxLTBU56', 'choices': [{'finish_reason': 'function_call', 'index': 0, 'message': {'role': 'assistant', 'function_call': {'arguments': '{"list":[{"domain":"tts","service":"google_translate_say","service_data":{"entity_id":"media_player.living_room_assist_speaker","message":"Kona, you are a good dog."}}]}', 'name': 'execute_services'}}}], 'created': 1721973509, 'model': 'gpt-3.5-turbo-1106', 'object': 'chat.completion', 'system_fingerprint': 'fp_b81a85d4e3', 'usage': {'completion_tokens': 53, 'prompt_tokens': 2435, 'total_tokens': 2488}}

2024-07-26 01:58:30.300 ERROR (MainThread) [custom_components.extended_openai_conversation] Service tts.google_translate_say not found
Traceback (most recent call last):
File "/home/homeassistant/.homeassistant/custom_components/extended_openai_conversation/init.py", line 196, in async_process
query_response = await self.query(user_input, messages, exposed_entities, 0)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/homeassistant/.homeassistant/custom_components/extended_openai_conversation/init.py", line 380, in query
return await self.execute_function_call(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/homeassistant/.homeassistant/custom_components/extended_openai_conversation/init.py", line 406, in execute_function_call
return await self.execute_function(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/homeassistant/.homeassistant/custom_components/extended_openai_conversation/init.py", line 432, in execute_function
result = await function_executor.execute(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/homeassistant/.homeassistant/custom_components/extended_openai_conversation/helpers.py", line 203, in execute
return await self.execute_service(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/homeassistant/.homeassistant/custom_components/extended_openai_conversation/helpers.py", line 278, in execute_service
await self.execute_service_single(
File "/home/homeassistant/.homeassistant/custom_components/extended_openai_conversation/helpers.py", line 253, in execute_service_single
raise ServiceNotFound(domain, service)
homeassistant.exceptions.ServiceNotFound: Service tts.google_translate_say not found

@semisynthetic
Copy link
Author

It also seems to think there is a tts.say service. Where is it getting these service definitions from? Shouldn't it be getting them from HA? Or maybe it's something in the model training? I tried adding to the prompt to direct it to the right service, but it doesn't know how to use tts.speak apparently.

"For the services domain tts, you must use the service tts.speak and must not use tts.say or tts.google_translate_say."

{'role': 'user', 'content': 'tell kona she is a good dog'}]
2024-07-26 15:07:04.046 INFO (MainThread) [custom_components.extended_openai_conversation] Response {'id': 'chatcmpl-9pKZLeElNPs76XVQthN7p4jR1PLGy', 'choices': [{'finish_reason': 'function_call', 'index': 0, 'message': {'role': 'assistant', 'function_call': {'arguments': '{"list":[{"domain":"tts","service":"speak","service_data":{"entity_id":"media_player.living_room_assist_speaker","message":"Kona, you are a good dog."}}]}', 'name': 'execute_services'}}}], 'created': 1722020823, 'model': 'gpt-3.5-turbo-1106', 'object': 'chat.completion', 'system_fingerprint': 'fp_b81a85d4e3', 'usage': {'completion_tokens': 51, 'prompt_tokens': 2459, 'total_tokens': 2510}}
2024-07-26 15:07:04.148 ERROR (MainThread) [homeassistant.components.assist_pipeline.pipeline] Unexpected error during intent recognition

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant