What are the priorities for a new deepforest tree model?

[bw4sz]

Let's get a few concrete steps written down on what would make a new deepforest release model.

1. Integrate data from a variety of new sources.
   We have been gathering datasets from Integrate other large datasets. #340. There could be 50,000 or so new annotations in all of that. Hard to really know until you dig into each dataset. For each dataset, split a train and test, just as we did for the bird detector.

2. Explicit measure of robustness to patch size.
   The largest pain point about the current model is that you vary the input focal view and get wildly different results. This is most pronounced when predicting into coarser resolution data. We should have a metric of consistency across both real and simulated patch size data.

3. Bounding box to polygon using segment anything. @MarconiS has made some great inroads and we need to formalize this.
   Integrate segment anything for post model bounding box to polygons. #460

Its a pretty big list, but if we can make a new model that maintains performance on existing NEON benchmark, improves on global datasets, scales more cleanly across focal views, and produces polygons from within bounding boxes, that would be best case scenario.

@ethanwhite, thoughts?

[ethanwhite]

I really like this as a set of things to accomplish in 2.0. A few more specific thoughts:

1. This all sounds good. Hopefully some of these datasets have polygons that we can use for assessing (3).
2. As part of this I believe we've already discussed either learning the patch size or establishing a rule of thumb for guiding patch size selection.
3. Hopefully we can assess this using some of the new data from (1)

[MarconiS]

Let's do it! We can either use the boxes to anchor SAM predictions or use the 3-points approach (box centroid + 2 true-negative points). As for validation, yes, hopefully we can assess quality using data from 1. If we integrate also the 3-point approach, being it a little more flexible, it could be useful to evaluate the polygons outer-boxes on the NEON benchmark as well?

[bw4sz]

Some more thoughts on a really ambitious model.

1. Global ecosystems, The model should generate reasonable predictions anywhere in the world.
2. Robustness to resolution. The model should predict a similar number of trees in the same image regardless of small changes in resolution.
3. Robustness to focal view. The model should make a similar number of predictions on one 80m focal view as two 40m focal views.
4. Temporal constancy. The model should make a similar number of predictions for the same geographic area through time.
5. Retraining flexibility. The model should need few new samples to make improved predictions in a local area.

If we want to achieve this model, we need to identify strategies and evaluation metrics for each stage explicitly.

[bw4sz]

Some relevant contributions from users in this issue. #318