Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document weirds #271

Open
JustinAzoff opened this issue Aug 8, 2024 · 1 comment
Open

Document weirds #271

JustinAzoff opened this issue Aug 8, 2024 · 1 comment

Comments

@JustinAzoff
Copy link

Weirds are generated as strings and are not documented like other enums.

This ended up on my radar recently due to popular LLMs completely hallucinating what DNS_Conn_count_too_large means. This is because that weird and many others like it are not really documented anywhere.

For each weird we should probably have:

  • When precisely is this generated?
  • What does this mean for zeek?
  • Is any action required?

Also, are all weirds tested? do we have a pcap that generates each one?

@timwoj
Copy link
Member

timwoj commented Aug 14, 2024

Also, are all weirds tested? do we have a pcap that generates each one?

A very naive survey of grepping the source for "Weird(", filtering the results a bit, and then searching through testing/btests/Baselines for matches shows we're not anywhere close to 100% test coverage on the weirds. This matches with the lack of test coverage on a lot of the error cases across the source in general though (see also zeek/zeek#2283).

This ended up on my radar recently due to popular LLMs completely hallucinating what DNS_Conn_count_too_large means.

Considering we don't have test coverage for any of the DNS wierds other than the EDNS ones, this isn't surprising.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants