NGBoost for Survival Probability Predictions for non-censored data

[phaniparsa]

Hello All,

Can NGBoost be used for predicting the survival probabilities for a given dataset without any right-censored events?

Thanks!

[alejandroschuler]

Sure- survival without censoring is just regression! :)

[phaniparsa]

Thank you for the reply. In survival analysis, cumulative predictions are obtained as 1-survival_probabilities. Will the CDF values from the Probability distribution output from NGBoost be equivalent to 1-survival_probabilities from interpretation point of view?

[kmedved]

I am likewise interested in this. I'd love to use Ngboost for survivorship problems, but am not sure how to interpret the results. Is it possible to use output of Ngboost to estimate the probability of surviving to an arbitrary future time?

Apologies if this is answered somewhere in the documentation and I missed it.

[alejandroschuler]

@phaniparsa yes you're basically on the right track. If your target variable Y is a vector of observed times of death/failure, then the predicted distribution Y_dist_pred= model.predict_dist(x) gives you the relative likelihood of death at any time y for an observation with features x. So, logically, if you want to compute the probability that this observation survives from 0 until a time y*, that's 1 - P(death between 0 and time y*), where P(death between 0 and time y*) is just the integral from 0 to y* of the predicted PDF of Y_dist_pred, or, in other words, its CDF at y* (assuming the CDF is 0 everywhere to the left of 0).

So, tl;dr: survival_prob(y*) = 1 - Y_dist_pred.cdf(y*)

The only thing you need to be aware of is that if you're doing strict survival, i.e. you're assuming it's impossible for someone to "die" before they are "born", then you need to use a distribution that has only positive support (e.g. Weibull, lognormal, exponential). If you use something like dist=Normal and there are some survival times in your data that are small then you could easily get cases where you're predicting non-negligible risk of instantaneous death.

[phaniparsa]

@alejandroschuler Thank you very much for the explanation, this helps a lot in understanding the concepts :)