bitswap(client/messageque): expose donthavetimeout config

[Wondertan]

Closes #701

[codecov]

Codecov Report

Attention: Patch coverage is 86.66667% with 6 lines in your changes missing coverage. Please review.

Project coverage is 60.33%. Comparing base (392493d) to head (d04bd72).
Report is 23 commits behind head on main.

Files with missing lines	Patch %	Lines
bitswap/client/client.go	20.00%	4 Missing ⚠️
...client/internal/messagequeue/donthavetimeoutmgr.go	94.44%	1 Missing and 1 partial ⚠️

@@            Coverage Diff             @@
##             main     #703      +/-   ##
==========================================
- Coverage   60.35%   60.33%   -0.03%     
==========================================
  Files         243      243              
  Lines       31038    31051      +13     
==========================================
  Hits        18734    18734              
- Misses      10637    10648      +11     
- Partials     1667     1669       +2     

Files with missing lines	Coverage Δ
...tswap/client/internal/messagequeue/messagequeue.go	83.53% <100.00%> (-1.39%)	⬇️
...client/internal/messagequeue/donthavetimeoutmgr.go	95.39% <94.44%> (+2.12%)	⬆️
bitswap/client/client.go	86.74% <20.00%> (-1.34%)	⬇️

... and 8 files with indirect coverage changes

[gammazero]

The onDontHaveTimeout function seems to accompany the config is many/most places. So, onDontHaveTimeout callback may be a candidate for moving into the DontHaveTimeoutConfig type.

[Wondertan]

I guess the question is: "Do we want users to register a custom handler there?". I would be opposed to that unless there is a prominent use case.

[lidel]

Two concerns with this approach:

1. We expose a way for overriding the defaults without exposing the implicit default values themselves
   • people will keep hardcoding current values instead of referencing const that may change in the future
2. We use a struct which forces override fo all values
   • This makes it tedious to override a single value
   • This means if we add a new knob, people who already override the struct will override default with implicit zero value for that type (likely causing bugs)

My mental model for makign things like this configurable is to

• have WithFoo configuration options which allow overriding each knob individually
• and expose DefaultFoo to allow people to programmatically refer to implicit default, rather than hardcoding them in their own code.

This may be more work in this PR upfront, but will save us from headache in the future. Thoughts?

[Wondertan]

I suggest we expose the default struct for people to override necessary knobs this way:

cfg := DefaultDontHaveTimeoutConfig()
// example of granular customization
cfg.MaxExpectedWantProcessTime = 5 * time.Second
// use the global option.
WithDontHaveTimeoutConfig(cfg)

This solves your concern without creating "options hell". This is equivalent to having multi-level options, yet it's cleaner. Also, it doesn't require forwarding all the options from client to bitswap in case you want to add more options horizontally, which I tried initially, and got lost.

[Wondertan]

Basically, the compromise I propose it expose the default configuration solving your concern while keeping things straightforward.

[Wondertan]

@gammazero, @lidel, one thing I realized about this component is that it's actually never disabled. The option to disable donthavetimeout handling exists in bitswap. However, it simply makes the handler no-op, while the component is still active and running, doing work that is costly (measuring latency, firing timeouts in routines, etc). And disabled donthavetimeout is the default behaviour in Kubo.

[lidel]

Triage notes:

• exposing defaults sgtm
  • nit: make it public? defaultDontHaveTimeoutConfig → DefaultDontHaveTimeoutConfig`
• @Wondertan what do you mean by "And disabled donthavetimeout is the default behaviour in Kubo."? Mind linking to relevant code/config?