Skip to content

Commit

Permalink
remove seed from background testing
Browse files Browse the repository at this point in the history
  • Loading branch information
btupper committed Oct 17, 2023
1 parent a872328 commit 60b615b
Show file tree
Hide file tree
Showing 9 changed files with 55 additions and 19 deletions.
Binary file modified data/bkg/bkg-covariates.gpkg
Binary file not shown.
Binary file modified data/bkg/buffered-polygon.gpkg
Binary file not shown.
Binary file modified data/obs/obs-covariates.gpkg
Binary file not shown.
2 changes: 1 addition & 1 deletion docs/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -348,7 +348,6 @@ <h3 data-number="3.4" class="anchored" data-anchor-id="data-storage"><span class
<pre><code>data
├── bkg
│ ├── bkg-covariates.gpkg
│ ├── bkg-covariates.gpkg-journal
│ └── buffered-polygon.gpkg
├── model
│ ├── v1
Expand Down Expand Up @@ -1511,6 +1510,7 @@ <h3 data-number="3.4" class="anchored" data-anchor-id="data-storage"><span class
│ └── Mola_mola.gpkg
├── obs
│ ├── obs-covariates.gpkg
│ ├── obs-covariates.gpkg-journal
│ └── obs.gpkg
└── oisst
├── 2000
Expand Down
56 changes: 43 additions & 13 deletions docs/modeling-04.html
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,8 @@
<script src="site_libs/quarto-search/fuse.min.js"></script>
<script src="site_libs/quarto-search/quarto-search.js"></script>
<meta name="quarto:offset" content="./">
<link href="./about.html" rel="next">
<link href="./modeling-03.html" rel="prev">
<script src="site_libs/quarto-html/quarto.js"></script>
<script src="site_libs/quarto-html/popper.min.js"></script>
<script src="site_libs/quarto-html/tippy.umd.min.js"></script>
Expand Down Expand Up @@ -107,9 +109,7 @@
<button type="button" class="quarto-btn-toggle btn" data-bs-toggle="collapse" data-bs-target="#quarto-sidebar,#quarto-sidebar-glass" aria-controls="quarto-sidebar" aria-expanded="false" aria-label="Toggle sidebar navigation" onclick="if (window.quartoToggleHeadroom) { window.quartoToggleHeadroom(); }">
<i class="bi bi-layout-text-sidebar-reverse"></i>
</button>
<nav class="quarto-page-breadcrumbs" aria-label="breadcrumb"><ol class="breadcrumb"><li class="breadcrumb-item">
Species distribution modeling with R
</li></ol></nav>
<nav class="quarto-page-breadcrumbs" aria-label="breadcrumb"><ol class="breadcrumb"><li class="breadcrumb-item"><a href="./modeling-04.html">Background count</a></li></ol></nav>
<a class="flex-grow-1" role="button" data-bs-toggle="collapse" data-bs-target="#quarto-sidebar,#quarto-sidebar-glass" aria-controls="quarto-sidebar" aria-expanded="false" aria-label="Toggle sidebar navigation" onclick="if (window.quartoToggleHeadroom) { window.quartoToggleHeadroom(); }">
</a>
<button type="button" class="btn quarto-search-button" aria-label="" onclick="window.quartoOpenSearch();">
Expand Down Expand Up @@ -192,6 +192,12 @@
<a href="./modeling-03.html" class="sidebar-item-text sidebar-link">
<span class="menu-text">Modeling with split</span></a>
</div>
</li>
<li class="sidebar-item">
<div class="sidebar-item-container">
<a href="./modeling-04.html" class="sidebar-item-text sidebar-link active">
<span class="menu-text">Background count</span></a>
</div>
</li>
<li class="sidebar-item">
<div class="sidebar-item-container">
Expand Down Expand Up @@ -328,7 +334,7 @@ <h1 class="title">Background count</h1>
y 1 46 46.12 -0.25 WGS 84 FALSE [y]</code></pre>
</div>
</div>
<p>Now we can make the models for those months, but we need to establish the number of background points for each model. Let’s do a simple progression… from 100 to 10000.</p>
<p>Now we can make the models for those months, but we need to establish the number of background points for each model. Let’s do a simple progression… from 10 to 10000. We’ll build a model for each number of background points, make a predictive map and then compute AUC.</p>
<p>Comments in the code block below help explain the steps taken.</p>
<div class="cell">
<div class="sourceCode cell-code" id="cb12"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb12-1"><a href="#cb12-1" aria-hidden="true" tabindex="-1"></a><span class="co"># choose a sequence of background counts</span></span>
Expand All @@ -345,7 +351,7 @@ <h1 class="title">Background count</h1>
<span id="cb12-12"><a href="#cb12-12" aria-hidden="true" tabindex="-1"></a> <span class="co"># prepare all of the observations</span></span>
<span id="cb12-13"><a href="#cb12-13" aria-hidden="true" tabindex="-1"></a> obn <span class="ot">=</span> dplyr<span class="sc">::</span><span class="fu">select</span>(ob, dplyr<span class="sc">::</span><span class="fu">all_of</span>(<span class="fu">c</span>(<span class="st">"sst"</span>, <span class="st">"u_wind"</span>, <span class="st">"v_wind"</span>))) <span class="sc">|&gt;</span></span>
<span id="cb12-14"><a href="#cb12-14" aria-hidden="true" tabindex="-1"></a> sf<span class="sc">::</span><span class="fu">st_drop_geometry</span>()</span>
<span id="cb12-15"><a href="#cb12-15" aria-hidden="true" tabindex="-1"></a> <span class="co"># prepare and sample the background</span></span>
<span id="cb12-15"><a href="#cb12-15" aria-hidden="true" tabindex="-1"></a> <span class="co"># prepare and then sample the background</span></span>
<span id="cb12-16"><a href="#cb12-16" aria-hidden="true" tabindex="-1"></a> bkn <span class="ot">=</span> dplyr<span class="sc">::</span><span class="fu">select</span>(bk, dplyr<span class="sc">::</span><span class="fu">all_of</span>(<span class="fu">c</span>(<span class="st">"sst"</span>, <span class="st">"u_wind"</span>, <span class="st">"v_wind"</span>))) <span class="sc">|&gt;</span></span>
<span id="cb12-17"><a href="#cb12-17" aria-hidden="true" tabindex="-1"></a> sf<span class="sc">::</span><span class="fu">st_drop_geometry</span>() <span class="sc">|&gt;</span> </span>
<span id="cb12-18"><a href="#cb12-18" aria-hidden="true" tabindex="-1"></a> dplyr<span class="sc">::</span><span class="fu">slice_sample</span>(<span class="at">n=</span>nbk)</span>
Expand All @@ -363,15 +369,27 @@ <h1 class="title">Background count</h1>
<span id="cb12-30"><a href="#cb12-30" aria-hidden="true" tabindex="-1"></a> dplyr<span class="sc">::</span><span class="fu">tibble</span>(<span class="at">nbkg =</span> nbk, <span class="at">AUC =</span> pauc<span class="sc">$</span>area)</span>
<span id="cb12-31"><a href="#cb12-31" aria-hidden="true" tabindex="-1"></a> }, <span class="at">ob =</span> obs, <span class="at">bk =</span> bkg, <span class="at">preds =</span> predictors) <span class="sc">|&gt;</span></span>
<span id="cb12-32"><a href="#cb12-32" aria-hidden="true" tabindex="-1"></a> <span class="co"># bind all of the individual tibbles into one data frame (still a tibble)</span></span>
<span id="cb12-33"><a href="#cb12-33" aria-hidden="true" tabindex="-1"></a> dplyr<span class="sc">::</span><span class="fu">bind_rows</span>() </span>
<span id="cb12-34"><a href="#cb12-34" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb12-35"><a href="#cb12-35" aria-hidden="true" tabindex="-1"></a>ggplot2<span class="sc">::</span><span class="fu">ggplot</span>(<span class="at">data =</span> x, ggplot2<span class="sc">::</span><span class="fu">aes</span>(<span class="at">x =</span> nbkg, <span class="at">y =</span> AUC)) <span class="sc">+</span></span>
<span id="cb12-36"><a href="#cb12-36" aria-hidden="true" tabindex="-1"></a> ggplot2<span class="sc">::</span><span class="fu">geom_point</span>() <span class="sc">+</span> </span>
<span id="cb12-37"><a href="#cb12-37" aria-hidden="true" tabindex="-1"></a> ggplot2<span class="sc">::</span><span class="fu">geom_smooth</span>(<span class="at">method =</span> <span class="st">"loess"</span>, <span class="at">se =</span> <span class="cn">TRUE</span>, <span class="at">formula =</span> y <span class="sc">~</span> x) <span class="sc">+</span> </span>
<span id="cb12-38"><a href="#cb12-38" aria-hidden="true" tabindex="-1"></a> ggplot2<span class="sc">::</span><span class="fu">labs</span>(<span class="at">x =</span> <span class="st">"Number of Background Points"</span>,</span>
<span id="cb12-39"><a href="#cb12-39" aria-hidden="true" tabindex="-1"></a> <span class="at">title =</span> <span class="st">"Choosing the number of background points"</span>)</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
<span id="cb12-33"><a href="#cb12-33" aria-hidden="true" tabindex="-1"></a> dplyr<span class="sc">::</span><span class="fu">bind_rows</span>() <span class="sc">|&gt;</span></span>
<span id="cb12-34"><a href="#cb12-34" aria-hidden="true" tabindex="-1"></a> dplyr<span class="sc">::</span><span class="fu">glimpse</span>()</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
<div class="cell-output cell-output-stdout">
<pre><code>Rows: 25
Columns: 2
$ nbkg &lt;dbl&gt; 10, 20, 50, 100, 250, 500, 1000, 1500, 2000, 2500, 3000, 3500, 40…
$ AUC &lt;dbl&gt; 0.8124647, 0.8078305, 0.8071452, 0.8101962, 0.8229343, 0.8267463,…</code></pre>
</div>
</div>
<p>Now we can show these with a simple plot.</p>
<div class="cell">
<details>
<summary>Code</summary>
<div class="sourceCode cell-code" id="cb14"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb14-1"><a href="#cb14-1" aria-hidden="true" tabindex="-1"></a>ggplot2<span class="sc">::</span><span class="fu">ggplot</span>(<span class="at">data =</span> x, ggplot2<span class="sc">::</span><span class="fu">aes</span>(<span class="at">x =</span> nbkg, <span class="at">y =</span> AUC)) <span class="sc">+</span></span>
<span id="cb14-2"><a href="#cb14-2" aria-hidden="true" tabindex="-1"></a> ggplot2<span class="sc">::</span><span class="fu">geom_point</span>() <span class="sc">+</span> </span>
<span id="cb14-3"><a href="#cb14-3" aria-hidden="true" tabindex="-1"></a> ggplot2<span class="sc">::</span><span class="fu">geom_smooth</span>(<span class="at">method =</span> <span class="st">"loess"</span>, <span class="at">se =</span> <span class="cn">TRUE</span>, <span class="at">formula =</span> y <span class="sc">~</span> x) <span class="sc">+</span> </span>
<span id="cb14-4"><a href="#cb14-4" aria-hidden="true" tabindex="-1"></a> ggplot2<span class="sc">::</span><span class="fu">labs</span>(<span class="at">x =</span> <span class="st">"Number of Background Points"</span>,</span>
<span id="cb14-5"><a href="#cb14-5" aria-hidden="true" tabindex="-1"></a> <span class="at">title =</span> <span class="st">"Choosing the number of background points"</span>)</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
</details>
<div class="cell-output-display">
<p><img src="modeling-04_files/figure-html/unnamed-chunk-8-1.png" class="img-fluid" width="672"></p>
<p><img src="modeling-04_files/figure-html/unnamed-chunk-9-1.png" class="img-fluid" width="672"></p>
</div>
</div>
<p>This workflow reveals to us that for the seasonal May-Oct model that we get no improvement in AUC once we get to 2500 background points.</p>
Expand Down Expand Up @@ -612,6 +630,18 @@ <h1 class="title">Background count</h1>
}
});
</script>
<nav class="page-navigation">
<div class="nav-page nav-page-previous">
<a href="./modeling-03.html" class="pagination-link">
<i class="bi bi-arrow-left-short"></i> <span class="nav-page-text">Modeling with split</span>
</a>
</div>
<div class="nav-page nav-page-next">
<a href="./about.html" class="pagination-link">
<span class="nav-page-text">About</span> <i class="bi bi-arrow-right-short"></i>
</a>
</div>
</nav>
</div> <!-- /content -->
<footer class="footer">
<div class="nav-footer">
Expand Down
Binary file not shown.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 2 additions & 2 deletions docs/search.json

Large diffs are not rendered by default.

12 changes: 9 additions & 3 deletions modeling-04.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ predicitors = stars::st_apply(predictors, c("x", "y"), mean,
print()
```

Now we can make the models for those months, but we need to establish the number of background points for each model. Let's do a simple progression... from 100 to 10000.
Now we can make the models for those months, but we need to establish the number of background points for each model. Let's do a simple progression... from 10 to 10000. We'll build a model for each number of background points, make a predictive map and then compute AUC.

Comments in the code block below help explain the steps taken.

Expand All @@ -110,7 +110,7 @@ x = lapply(nback,
# prepare all of the observations
obn = dplyr::select(ob, dplyr::all_of(c("sst", "u_wind", "v_wind"))) |>
sf::st_drop_geometry()
# prepare and sample the background
# prepare and then sample the background
bkn = dplyr::select(bk, dplyr::all_of(c("sst", "u_wind", "v_wind"))) |>
sf::st_drop_geometry() |>
dplyr::slice_sample(n=nbk)
Expand All @@ -128,8 +128,14 @@ x = lapply(nback,
dplyr::tibble(nbkg = nbk, AUC = pauc$area)
}, ob = obs, bk = bkg, preds = predictors) |>
# bind all of the individual tibbles into one data frame (still a tibble)
dplyr::bind_rows()
dplyr::bind_rows() |>
dplyr::glimpse()
```

Now we can show these with a simple plot.

```{r}
#| code-fold: true
ggplot2::ggplot(data = x, ggplot2::aes(x = nbkg, y = AUC)) +
ggplot2::geom_point() +
ggplot2::geom_smooth(method = "loess", se = TRUE, formula = y ~ x) +
Expand Down

0 comments on commit 60b615b

Please sign in to comment.