Logging: print message as JSON, instead of wrapping into a string

[mriccia]

Is your feature request related to a problem? Please describe.

The Powertools Logging functionality automatically wraps the messages into string, regardless of whether they are strings or objects.
Looking through the logs with CloudWatch logs is harder with the message being printed as a single string, and printing as JSON will also unlock other options (e.g. using CloudWatch insights to automatically discover fields).

Describe the solution you'd like

Describe alternatives you've considered

Additional context

[jeromevdl]

I think this is a breaking change ==> v2

[scottgerring]

This feels quite important. Is it really breaking and not a bug?
OTOH we are redoing the logging in v2 anyway, so there's a natural synergy there.

[jeromevdl]

@jdoherty I'm taking this one. I'm working on the logging v2 anyway. Not sure that make sense you spend time on a pure log4j solution as I'm using slf4j now.

[jeromevdl]

https://stackoverflow.com/questions/77543791/slf4j-and-log4j2-json-messages-escaped

[jeromevdl]

will be part of #1435, using the PowertoolsResolver in log4j and some custom code in JsonUtils for logbak

[jeromevdl]

Discussion with Log4J: apache/logging-log4j2#2013

[scottgerring]

@jeromevdl we should probably update this with resolution once you're done 👼

[jeromevdl]

Based on the discussions with log4j guys, on this issue, and after thinking a bit more about this, we won't log messages as JSON. Instead, we'll support advanced object as additional arguments (not using MDC which is limited to Strings), and log them as JSON (when that makes sense).

Example:

Product product = new Product("id123456", "Name of the product", 234.5);

LOGGER.info("Some message", StructuredArguments.entry("key", product)); 

// can also use static import to make it lighter
LOGGER.info("Some message", entry("product", product));

will produce:

{
    "message": "Some message",
    "product": {
        "id": "id123456", 
        "name": "Name of the product", 
        "price": 234.5
    }
}

Note there was no {} in the message, if you do this:

LOGGER.info("My super product={}", entry("product", product));

will produce:

{
    "message": "My super product=[...]", // will use the object.toString()
    "product": {
        "id": "id123456", 
        "name": "Name of the product", 
        "price": 234.5
    }
}

Also working on other StructuredArguments:

• entries that will take a complete Map
• array that will take an Object... objects
• json that will take a raw json string

For the event and response, we'll use this instead of putting the event in the message.

[scottgerring]

@jeromevdl what do we need to do this - is it simply a matter of documenting this practice? I gather the user's interface here will be through SLF4j APIs.

[jeromevdl]

No I need to implement some stuff. The StructureArguments thing and how they interpreted. It is supported with the logback-logstash-encoder but that's another library to add to the project...

[mriccia]

My thoughts on this implementation are:
Yes we are achieving the objective of printing out objects as JSON structures in the logs, which is the ultimate goal.
However the onus is now on the library user to ensure that they log this way every time they intend to log an object.

What will I do if I only want to log an object, without a string message?
Will it be LOGGER.info("{}", entry("product", product));?

[jeromevdl]

You cannot log without a message. Or you can just log an empty message, but messages are always there (it's the basic of the loggers).

you should do LOGGER.info("", entry("product", product)); but you will have an empty message in the logs...

[jeromevdl]

we could actually check if the message is empty and not adding it to the log... but a log without a message is a bit weird

[mriccia]

I am OK with this (would like to hear other views), even though I don't love the syntax.

we could actually check if the message is empty and not adding it to the log... but a log without a message is a bit weird

Agree that it would be weird to print a log message, without any message