Add configurable logging system #407

enigbe · 2024-11-20T16:01:23Z

Overview

This PR introduces a flexible logging system for LDK Node by implementing a LogWriter interface that supports writing logs to different destinations.

What this PR does

Introduces a LogWriter interface, allowing Writer variants to handle log output destinations. The supported Writer variants can now:
- Write logs to the filesystem,
- Forward to a log implementer,
- Relay logs to a custom logger.
Exposes LogWriter to bindings.
Test logging to different destinations with:
- In-memory log logger,
- In-memory LogWriter logger.

Related Issue(s)

Expose logger interface #309
Cleaner implementation of closed draft feat: add customizable logging system #401

tnull

Cool, thank you for looking into this! This already looks pretty good, but I have some comments after the first round of review.

tnull · 2024-11-27T12:35:45Z

bindings/ldk_node.udl

@@ -27,6 +25,20 @@ dictionary EsploraSyncConfig {
 	u64 fee_rate_cache_update_interval_secs;
 };

+enum LdkLevel {


Let's leave this as LogLevel. I don't think we want to start prefixing any of the types we expose in the API with Ldk. The prefix is really only used internally when we want to introduce a local type with the same name as an LDK type.

tnull · 2024-11-27T12:36:32Z

bindings/ldk_node.udl

+
+dictionary FilesystemLoggerConfig {
+	string log_file_path;
+	LdkLevel level;


Let's also call the field here log_level.

tnull · 2024-11-27T12:38:22Z

...in/ldk-node-android/lib/src/androidTest/kotlin/org/lightningdevkit/ldknode/AndroidLibTest.kt

@@ -28,13 +28,11 @@ class AndroidLibTest {
        config1.storageDirPath = tmpDir1
        config1.listeningAddresses = listOf(listenAddress1)
        config1.network = Network.REGTEST
-        config1.logLevel = LogLevel.TRACE


Should we still set TRACE level, just by the new means?

tnull · 2024-11-27T12:38:46Z

Cargo.toml

@@ -75,6 +75,7 @@ libc = "0.2"
 uniffi = { version = "0.27.3", features = ["build"], optional = true }
 serde = { version = "1.0.210", default-features = false, features = ["std", "derive"] }
 serde_json = { version = "1.0.128", default-features = false, features = ["std"] }
+log = { version = "0.4.22", features = ["std"]}


Let's add default-features = false here and only include the features we really need.

tnull · 2024-11-27T12:41:08Z

src/builder.rs

-		let logger = setup_logger(&self.config)?;
+		let writer = LogWriterConfig::default();
+		let log_writer_config =
+			if let Some(config) = &self.log_writer_config { config } else { &writer };


I think you should be able to just use unwrap_or_default() here.

tnull · 2024-11-27T13:37:12Z

src/logger.rs

 pub(crate) enum Writer {
 	/// Writes logs to the file system.
 	FileWriter(FilesystemLogger),
 	/// Forwards logs to the `log` facade.
 	LogFacadeWriter(LogFacadeLogger),
+	/// Forwards logs to custom writer.


nit: s/custom writer/a custom writer/

tnull · 2024-11-27T13:37:30Z

tests/common/mod.rs

 }

 /// Simple in-memory mock `log` logger for tests.
+#[derive(Debug)]


Why is this Debug?

tnull · 2024-11-27T13:38:00Z

tests/common/mod.rs

@@ -271,9 +274,18 @@ impl MockLogger {
 	}
 }

+/// [`MockLogger`] as `log` logger - destination for [`Writer::LogFacadeWriter`]


This seems to belong in a different commit? Also, let's not add documents on trait implementations (internal ones in particular).

tnull · 2024-11-27T13:39:56Z

tests/integration_tests_rust.rs

@@ -828,13 +831,16 @@ fn generate_bip21_uri() {
 		},
 		Err(e) => panic!("Failed to generate URI: {:?}", e),
 	}
+
+	assert!(mock_logger.retrieve_logs().last().unwrap().contains("Invoice created: lnbcrt"));


As mentioned above: please avoid such potentially-flaky asserts. Also, we should really be testing the log facade separately instead of sprinkling in these asserts to unrelated testcases.

tnull · 2024-11-27T13:41:07Z

tests/integration_tests_rust.rs

 }

 #[test]
 fn unified_qr_send_receive() {
 	let (bitcoind, electrsd) = setup_bitcoind_and_electrsd();
 	let chain_source = TestChainSource::Esplora(&electrsd);

+	// Setup `log` facade logger.


Same goes for arbitrary comments: please only add comments where they add to the context. Comments like this don't provide additional information over just reading the next line, they just increase the (visual) noise when reading code.

tnull · 2024-12-09T08:14:56Z

@enigbe Is there any update on this? Please let me know if you're hitting any blockers.

This seems to need a minor rebase by now.

enigbe · 2024-12-10T03:50:34Z

No blockers on this. I'll be pushing updates later today.

* Add flexible log writer interface for multiple destinations * Implement filesystem writing capability via FilesystemLogger * Prefix LDK-based objects with 'Ldk' for consistency * Add configuration options for log file path and log level

* Add support for user-provided custom logger to write logs to, allowing users to provide any logger that implements LogWriter * Add test to cover this use case, implementing Log- Writer for the mock, in-memory MockLogger.

This commit includes several improvements aimed at simplifying code, improving readability, and optimizing performance: - Inline enum struct fields to reduce indirection and clutter. - Rename structs to improve clarity and better reflect their purpose. - Enable specific feature flags in dependencies for a more streamlined build. - Eliminate unnecessary data allocations to optimize memory usage. - Correct minor grammatical error in documentation.

- allocates Strings only when necessary with `uniffi` feature, otherwise, keeps LogRecord fileds as lifetime references.

This commit adds a CustomLogWriter class to the kotlin library test, configuring the writer via the exposed node builder and tests the ability to log to the custom writer destination. [WIP]

enigbe mentioned this pull request Nov 20, 2024

feat: add customizable logging system #401

Closed

7 tasks

G8XSU requested a review from tnull November 21, 2024 18:37

tnull reviewed Nov 27, 2024

View reviewed changes

enigbe added 9 commits December 11, 2024 13:09

feat(logger): add & impl LogWriter for Writer

abb3b44

fix(bindings): remove config log level from kotlin tests

49e914e

feat: forward logs to log facade

3dbff37

test(logwriter): forward logs to mock in-memory log logger

c1f7858

fix: correct "Forwards" spelling error

a40ccb1

feat(custom): forward logs to a custom logger

7f4b122

* Add support for user-provided custom logger to write logs to, allowing users to provide any logger that implements LogWriter * Add test to cover this use case, implementing Log- Writer for the mock, in-memory MockLogger.

fix(test): fix setting log's global logger twice

bdeb432

chore: revert the renaming of LogLevel to LdkLevel

d8eb0e1

enigbe force-pushed the 2024-11-configurable-logging-system branch from 1d57cab to d8eb0e1 Compare December 11, 2024 12:18

G8XSU mentioned this pull request Dec 12, 2024

Ldk-Server Milestone 3 lightningdevkit/ldk-server#11

Open

enigbe added 3 commits December 13, 2024 05:28

chore: remove unused CustomTlvRecord

e968562

feat: address costly log record re-allocation

0347e04

- allocates Strings only when necessary with `uniffi` feature, otherwise, keeps LogRecord fileds as lifetime references.

test(android): set custom log writer via builder method

26aff81

This commit adds a CustomLogWriter class to the kotlin library test, configuring the writer via the exposed node builder and tests the ability to log to the custom writer destination. [WIP]

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add configurable logging system #407

Add configurable logging system #407

enigbe commented Nov 20, 2024

tnull left a comment

tnull Nov 27, 2024

tnull Nov 27, 2024

tnull Nov 27, 2024

tnull Nov 27, 2024

tnull Nov 27, 2024

tnull Nov 27, 2024

tnull Nov 27, 2024

tnull Nov 27, 2024

tnull Nov 27, 2024

tnull Nov 27, 2024

tnull commented Dec 9, 2024

enigbe commented Dec 10, 2024

Add configurable logging system #407

Are you sure you want to change the base?

Add configurable logging system #407

Conversation

enigbe commented Nov 20, 2024

Overview

What this PR does

Related Issue(s)

tnull left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tnull commented Dec 9, 2024

enigbe commented Dec 10, 2024