Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add the otelcol syslog receiver #2263

Merged
merged 27 commits into from
Jan 3, 2025
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
1aec317
Implementation of otelcol syslog receiver wrapper
dehaansa Dec 11, 2024
69a93f3
update unmarshal code
dehaansa Dec 11, 2024
306a43f
Merge branch 'main' into feat/otelcol-syslog-receiver
dehaansa Dec 11, 2024
ad69096
Add changelog entry for syslogreceiver
dehaansa Dec 11, 2024
cefb4c4
Change field key, add converter
dehaansa Dec 11, 2024
587d4a1
Apply suggestions from code review
dehaansa Dec 11, 2024
85b2cd0
Update whitespace attribute descriptions
dehaansa Dec 11, 2024
b22ebfb
Clean up commented out code
dehaansa Dec 11, 2024
520eeea
Clean up comments
dehaansa Dec 11, 2024
ad716de
Update autogenerated docs
dehaansa Dec 11, 2024
d00f726
Update docs/sources/reference/components/otelcol/otelcol.receiver.sys…
dehaansa Dec 11, 2024
47deaad
Update docs/sources/reference/components/otelcol/otelcol.receiver.sys…
dehaansa Dec 11, 2024
f79a9ac
Put a little more maximum delay in syslog test
dehaansa Dec 11, 2024
3a438a7
clean up validation, try not to use localhost for ci tests
dehaansa Dec 11, 2024
1072d10
Update docs/sources/reference/components/otelcol/otelcol.receiver.sys…
dehaansa Dec 12, 2024
27dcf40
Fix syslogreceiver converter
dehaansa Dec 12, 2024
d0e59ac
Merge branch 'feat/otelcol-syslog-receiver' of github.com:dehaansa/al…
dehaansa Dec 12, 2024
1d0b32a
Add test sleep
dehaansa Dec 12, 2024
f2942e4
Update docs/sources/reference/components/otelcol/otelcol.receiver.sys…
dehaansa Dec 12, 2024
ad373b7
Address some PR feedback
dehaansa Dec 12, 2024
70997f7
Update docs for rfc6587 arguments
dehaansa Dec 12, 2024
f278703
Add todo to test comment
dehaansa Dec 12, 2024
548b173
Update docs/sources/reference/components/otelcol/otelcol.receiver.sys…
dehaansa Dec 12, 2024
a9fc448
Apply suggestions from code review
dehaansa Dec 12, 2024
d0a0fb8
Merge branch 'main' into feat/otelcol-syslog-receiver
dehaansa Dec 25, 2024
3ee6853
Merge branch 'main' into feat/otelcol-syslog-receiver
dehaansa Jan 2, 2025
14d1788
remote timeout from test
dehaansa Jan 2, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ The following arguments are supported:
| `enable_octet_counting` | `bool` | Whether to enable RFC6587 octet counting. | `false` | no |
| `max_octets` | `int` | The maximum octets for messages when octet counting is enabled. | `8192` | no |
| `allow_skip_pri_header` | `bool` | Allow parsing records without a priority header. | `false` | no |
| `non_transparent_framing_trailer` | `string` | The framing trailer when using RFC6587 Non-Transparent-Framing. | `false` | no |
| `non_transparent_framing_trailer` | `string` | The framing trailer when using RFC6587 Non-Transparent-Framing. | `nil` | no |

The `protocol` argument specifies the syslog format supported by the receiver.
`protocol` must be one of `rfc5424`, `rfc3164`
dehaansa marked this conversation as resolved.
Show resolved Hide resolved
Expand All @@ -53,6 +53,8 @@ See [this wikipedia entry][tz-wiki] for a non-comprehensive list.

The `non_transparent_framing_trailer` argument must be one of `LF`, `NUL`.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add some context around this? Like how can the default be nil? Even if its a string it should be "" but that seems to not be LF or NUL.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that's fair. It's a *string, so it's nil if not set. If it is set, then the character chosen is expected to terminate the message. If not set, it's expected that you're using octet counting or that the syslog frame contains only one message.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@clayton-cornell I'd appreciate any feedback you have on this docs change as well as Matt's.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's in the docs makes sense to me. I don't know the specific nuances of a string data type and a value of nil though. Whats' there seems to make sense as I read it.


The `non_transparent_framing_trailer` and `enable_octet_counting` arguments cannot be used with a UDP syslog server.
dehaansa marked this conversation as resolved.
Show resolved Hide resolved

[tz-wiki]: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones

## Blocks
Expand Down
41 changes: 22 additions & 19 deletions internal/component/otelcol/receiver/syslog/syslog.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ package syslog

import (
"fmt"
net_url "net/url"
"net"

"github.com/alecthomas/units"
"github.com/grafana/alloy/internal/component"
Expand Down Expand Up @@ -48,7 +48,7 @@ type Arguments struct {
EnableOctetCounting bool `alloy:"enable_octet_counting,attr,optional"`
MaxOctets int `alloy:"max_octets,attr,optional"`
AllowSkipPriHeader bool `alloy:"allow_skip_pri_header,attr,optional"`
NonTransparentFramingTrailer FramingTrailer `alloy:"non_transparent_framing_trailer,attr,optional"`
NonTransparentFramingTrailer *FramingTrailer `alloy:"non_transparent_framing_trailer,attr,optional"`

ConsumerRetry otelcol.ConsumerRetryArguments `alloy:"retry_on_failure,block,optional"`
TCP *TCP `alloy:"tcp,block,optional"`
Expand Down Expand Up @@ -168,27 +168,29 @@ var _ receiver.Arguments = Arguments{}
// SetToDefault implements syntax.Defaulter.
func (args *Arguments) SetToDefault() {
*args = Arguments{
Location: "UTC",
Protocol: config.SyslogFormatRFC5424,
Output: &otelcol.ConsumerArguments{},
NonTransparentFramingTrailer: LFTrailer,
Location: "UTC",
Protocol: config.SyslogFormatRFC5424,
Output: &otelcol.ConsumerArguments{},
}
args.DebugMetrics.SetToDefault()
args.ConsumerRetry.SetToDefault()
}

// Convert implements receiver.Arguments.
func (args Arguments) Convert() (otelcomponent.Config, error) {
trailer := string(args.NonTransparentFramingTrailer)

c := stanzainputsyslog.NewConfig()
c.BaseConfig = stanzaparsersyslog.BaseConfig{
Protocol: string(args.Protocol),
Location: args.Location,
EnableOctetCounting: args.EnableOctetCounting,
MaxOctets: args.MaxOctets,
AllowSkipPriHeader: args.AllowSkipPriHeader,
NonTransparentFramingTrailer: &trailer,
Protocol: string(args.Protocol),
Location: args.Location,
EnableOctetCounting: args.EnableOctetCounting,
MaxOctets: args.MaxOctets,
AllowSkipPriHeader: args.AllowSkipPriHeader,
}

if args.NonTransparentFramingTrailer != nil {
s := string(*args.NonTransparentFramingTrailer)
c.BaseConfig.NonTransparentFramingTrailer = &s
}

if args.TCP != nil {
Expand Down Expand Up @@ -274,12 +276,12 @@ func (args *Arguments) Validate() error {
}

if args.TCP != nil {
if err := validateURL(args.TCP.ListenAddress, "tcp.listen_address"); err != nil {
if err := validateListenAddress(args.TCP.ListenAddress, "tcp.listen_address"); err != nil {
errs = multierror.Append(errs, err)
}

if args.NonTransparentFramingTrailer != LFTrailer && args.NonTransparentFramingTrailer != NULTrailer {
errs = multierror.Append(errs, fmt.Errorf("invalid non_transparent_framing_trailer, must be one of 'LF', 'NUL': %s", args.NonTransparentFramingTrailer))
if args.NonTransparentFramingTrailer != nil && *args.NonTransparentFramingTrailer != LFTrailer && *args.NonTransparentFramingTrailer != NULTrailer {
errs = multierror.Append(errs, fmt.Errorf("invalid non_transparent_framing_trailer, must be one of 'LF', 'NUL': %s", *args.NonTransparentFramingTrailer))
}

_, err := decode.LookupEncoding(args.TCP.Encoding)
Expand All @@ -293,7 +295,7 @@ func (args *Arguments) Validate() error {
}

if args.UDP != nil {
if err := validateURL(args.UDP.ListenAddress, "udp.listen_address"); err != nil {
if err := validateListenAddress(args.UDP.ListenAddress, "udp.listen_address"); err != nil {
errs = multierror.Append(errs, err)
}

Expand All @@ -306,11 +308,12 @@ func (args *Arguments) Validate() error {
return errs
}

func validateURL(url string, urlName string) error {
func validateListenAddress(url string, urlName string) error {
if url == "" {
return fmt.Errorf("%s cannot be empty", urlName)
}
if _, err := net_url.Parse(url); err != nil {

if _, _, err := net.SplitHostPort(url); err != nil {
return fmt.Errorf("invalid %s: %w", urlName, err)
}
return nil
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@ func getFreeAddr(t *testing.T) string {
portNumber, err := freeport.GetFreePort()
require.NoError(t, err)

return fmt.Sprintf("localhost:%d", portNumber)
return fmt.Sprintf("127.0.0.1:%d", portNumber)
}

func TestUnmarshal(t *testing.T) {
Expand Down
Loading