Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Is self reflection necessary to generate good tests? #180

Open
EmbeddedDevops1 opened this issue Oct 8, 2024 · 0 comments
Open

Is self reflection necessary to generate good tests? #180

EmbeddedDevops1 opened this issue Oct 8, 2024 · 0 comments
Labels
experiment Experimentation needed prompt engineering Involves rewording or restructuring the prompt handed to the LLM

Comments

@EmbeddedDevops1
Copy link
Collaborator

Should there be an intermediary step that the LLM takes to reflect on the code and first write tests in natural language? After it has written the natural language test it can take that and translate it into a real test.

It seems like o1 from OpenAI seems to be doing some sort of reflection/introspection first before giving a response. Maybe this is needed as well for generating unit tests or, perhaps, it is overkill?
@EmbeddedDevops1 EmbeddedDevops1 added prompt engineering Involves rewording or restructuring the prompt handed to the LLM experiment Experimentation needed labels Oct 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
experiment Experimentation needed prompt engineering Involves rewording or restructuring the prompt handed to the LLM
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@EmbeddedDevops1