Logistic models: Be less verbose by default + be less scary for users with less red

[DominiqueMakowski]

1. messages that are not warnings/errors and are information should be in yellow. Red really makes it look like something is wrong, and can frighten new users & students 👹
2. I would remove the message about the exponentiate. While the one above is information, that one is a _(debatable) suggestion and should be in docs

[bwiernik]

The red is just coming from your RStudio no? RStudio uses the same color for all messages, warnings, and errors.

[DominiqueMakowski]

I don't think these are messages per se, they are just part of the output, so we can specify colors as we often do elsewhere where we print info in yellow

[strengejacke]

Second is a message and only shown once per session:

parameters/R/format.R

Line 909 in 034f5df

if (!is.null(msg) && isTRUE(getOption("parameters_warning_exponentiate", TRUE))) {

[strengejacke]

First one is also a message and not shown in red:

parameters/R/format.R

Line 857 in 034f5df

}

[DominiqueMakowski]

I'd vote for:

1. Set the exponentiate warning to false by default (I don't think this info warrants an intrusive message, even once per sesh)
2. Transform the information about how CIs/ps are computed as part of the pretty printing and display them in yellow

In general, we should probably take a look at homogenizing outputs / colour schemes, we have sometimes some green, yellow, reds, etc. that we might be using inconsistently.
Arguably the problem with colours is that they must work on dark and white backgrounds, but I think if we stick to default for text, yellow for information, green/red for "passed/failed checks", and then warnings when actual warnings, we should be alright

[strengejacke]

What about model info? Yellow, too?

[DominiqueMakowski]

it's a bit much, white is fine I'd say

Maybe we could add a "Note: " before "Uncertainty intervals..." just to suggest to participants that it's not an issue per se that it's just for their information

[strengejacke]

Set the exponentiate warning to false by default (I don't think this info warrants an intrusive message, even once per sesh)

But then it's never shown.

You can always suppress all information with verbose = FALSE. And you can in general customize the amount of default information using global options:

https://easystats.github.io/parameters/reference/print.parameters_model.html#global-options-to-customize-messages-and-tables-when-printing

[DominiqueMakowski]

[strengejacke]

Note that this note is useful for people less familiar with log(it) transformations! See also:

library(parameters)
model <- lm(log(mpg) ~ hp, data = mtcars)
model_parameters(model)
#> Parameter   | Coefficient |       SE |         95% CI | t(30) |      p
#> ----------------------------------------------------------------------
#> (Intercept) |        3.46 |     0.08 | [ 3.30,  3.62] | 44.04 | < .001
#> hp          |   -3.43e-03 | 4.87e-04 | [ 0.00,  0.00] | -7.05 | < .001
#> 
#> Uncertainty intervals (equal-tailed) and p-values (two-tailed) computed
#>   using a Wald t-distribution approximation.
#> 
#> The model has a log-transformed response variable. Consider using
#>   `exponentiate = TRUE` to interpret coefficients as ratios.

library(parameters)
model <- lm(log(mpg) ~ hp, data = mtcars)
model_parameters(model, exponentiate = TRUE)
#> Parameter   | Coefficient |       SE |         95% CI | t(30) |      p
#> ----------------------------------------------------------------------
#> (Intercept) |       31.83 |     2.50 | [27.11, 37.37] | 44.04 | < .001
#> hp          |        1.00 | 4.85e-04 | [ 1.00,  1.00] | -7.05 | < .001
#> 
#> Uncertainty intervals (equal-tailed) and p-values (two-tailed) computed
#>   using a Wald t-distribution approximation.
#> 
#> The model has a log-transformed response variable, and exponentiated
#>   parameters are reported. A one-unit increase in the predictor is
#>   associated with multiplying the outcome by that predictor's coefficient.

[strengejacke]

And yes, your alerting red color is just because you have an RStudio setting that prints messages in red?!?

[DominiqueMakowski]

Note that this note is useful for people less familiar with log(it) transformations! See also:

Yes but printing this in the output is IMO not the right place for this kind of information 🙃

you have an RStudio setting that prints messages in red?!?

[strengejacke]

We may remove that messages (though I find them helpful, because I bet there are many users not very familiar with these kind of transformations), I'd say we should at least keep this message (which is a warning about possible modelling/data issues):

library(parameters)
d_sep <- data.frame(
  y = c(0, 0, 0, 0, 1, 1, 1, 1),
  x1 = c(1, 2, 3, 3, 5, 6, 10, 11),
  x2 = c(3, 2, -1, -1, 2, 4, 1, 0)
)
m_sep <- suppressWarnings(glm(y ~ x1 + x2, data = d_sep, family = binomial))
model_parameters(m_sep, exponentiate = TRUE)
#> Parameter   | Odds Ratio |       SE |      95% CI |         z |      p
#> ----------------------------------------------------------------------
#> (Intercept) |   1.97e-29 | 3.61e-24 | [0.00, Inf] | -3.60e-04 | > .999
#> x1          |   4.36e+06 | 1.19e+11 | [0.00,    ] |  5.59e-04 | > .999
#> x2          |     513.31 | 4.19e+07 | [0.00,    ] |  7.65e-05 | > .999
#> 
#> Uncertainty intervals (profile-likelihood) and p-values (two-tailed)
#>   computed using a Wald z-distribution approximation.
#> 
#> Some coefficients are very large, which may indicate issues with
#>   complete separation.

^{Created on 2024-10-27 with reprex v2.1.1}

[strengejacke]

We could move parts of the information into the docs - what would be the best place? I'd say the exponentiate argumennt? https://easystats.github.io/parameters/reference/model_parameters.default.html

[bwiernik]

I would prefer to leave the exponentiate message, but print it as regular text (white or yellow) rather than as a message()

[strengejacke]

So we have mixed opinions on this issue. How do we decide?

[strengejacke]

Reference Screenshots: https://x.com/strengejacke/status/1850483090109432136

I think the last one (large coefs) is most useful, and the one before (interpreting expt coefs) it least useful - IME it's unlikely to be clear enough to help someone who doesn't already get it, and not needed for someone who does

I think none of the information is really worth printing each time. If it was all moved into the docs, that would be just as useful.

Definitely keep that last sentence in the output for all cases you shared here.

fwiw, I think that log(outcome) warning is gold. Here is a paper in the lancet insisting it's an odds ratio in the abstract, and I had to argue at length with a colleague that it was not in fact an OR ("But Darren, it's in The Lancet!")

I prefer these informative messages and they're why I recommend the easystats packages when teaching. Students often struggle to identify the important information when it's buried in the docs. There's always summary() for "parsimonious" output. I'm not convinced how helpful the bottom left "interpretation" message is though (since it necessarily has to be fairly abstract), which makes me wonder the extent to which these messages actually improve model evaluation/interpretation. Would be interesting to test...

I think it's nice to "warn" about exponentiating in both cases but I don't think the function should be giving interpretations. That should go in the docs, imo. I also think it's nice to warn about large coefficients.

I’m wary of interpretation stuff in general because it encourages not thinking for yourself, but at the very least hide it by default and have an argument like help=T to print it.

As a non-statistician I find the messages on both left hand side images helpful.

I agree with this. {easystats} already has a {report} package for this kind of stuff. By default, I think that {parameters} should only print the quantities explicitly requested, and not give advice unless the quantities are obviously misleading.

[DominiqueMakowski]

the problem of exponentiate is that it might lead to misleading interpretations for models with interactions etc.

The last point is fair, and the idea of having a help=TRUE option to toggle actual advice could be interesting

[DominiqueMakowski]

@mattansb? also @profandyfield

[profandyfield]

The help=T suggestion was me, Dom. I'm generally with you in not wanting this sort of advice in the default output but instead think about ways to offer help to those that need it. Maybe eaststats needs the MS paperclip (monster style, obvs) .... 'It looks like you've fitted a logistic model ...' :-)

[mattansb]

I think all of these "tips" should appear as messages/warnings in the model_parameters() function (and can be disabled with verbose=FALSE, and not in the print()ed output/footer.

[strengejacke]

The "interpretation" message is now moved into the docs - I think it's fine to not show this message. Furthermore, info text is yellow. What remains is the question whether / how (often) we should show the exp-messages. Therefore, I turned this issue into a discussion...

[bwiernik]

The large value message is still shown, right ?

[strengejacke]

Yes. Only this (bottom) message is no longer shown and moved into the docs:

The exponentiate messages are also still shown.

Like: