Question: is separability method from this package compatible with tidy?

[dhombios]

I have a database in which columns represent different features of a dataset with an additional column that contains the category to which each observation belong (true or false). Due to the large amount of features available, I need to eliminate the worst ones

For that purpose, separability method provided by this project seems particularly useful, as it provides a fast coarse estimation of how useful each feature can be for classifying new observations. For that reason, I'm trying to apply it to each column of the dplyr object after separating observations according to their category

As this package seems to be compatible with tidyverse, is this function compatible with summarize? Does this package provide any abstraction that could simplify doing this?

[jeffreyevans]

Have you taken a look at the `spectral.separability` function? If I understand what you are after, this may do the trick. The `separability` function is intended to provide the individual separability statistics whereas, `spectral.separability` produces a pairwise separability matrix of target classes. You may have to mutate your format so that each column represents a target class with rows being spectra of the class samples. The help document provides an example. As to the `separability` function, it returns a vector of 5 separability metrics based on the input of 2 continuous numeric vectors. If one wanted to look conditionally, this aggregation would have to be accounted for in your code before running the statistic. Please note that a specific statistic can be indexed so that a single result is returned ie., `separability(x, y)[1]` would return the B (Bhattacharryya distance) statistic. I am sure that there is a tidy way to do this as well but, as weird as it seems in handling the resulting nested list objects, here is a solution that provides separability of all band-wise comparisons broken down by class. This is assuming a data.frame with columns representing spectra and a column representing classes. Here I create an example data.frame object “x” that illustrates how I believe you have your data set up. ``` require(MASS) d <- 6 # Number of bands n.class <- 5 # Number of classes n <- rep(1000, 5) mu <- round(matrix(rnorm(d*n.class, 128, 1), ncol=n.class, byrow=TRUE), 0) x <- matrix(double(), ncol=d, nrow=0) classes <- integer() for (i in 1:n.class) { f <- svd(matrix(rnorm(d^2), ncol=d)) sigma <- t(f$v) %*% diag(rep(10, d)) %*% f$v x <- rbind(x, mvrnorm(n[i], mu[, i], sigma)) classes <- c(classes, rep(i, n[i])) } x <- data.frame(classes = classes, x) names(x)[2:7] <- paste0("B", 1:6) ``` Now, we can use an lapply approach to iterate through all pairwise combinations (column indices are created using the spatialEco:: all_pairwise function). The second lapply calculates the separability for each class. Since we then have a nested list (which admittedly are a pain) we can use do.call with Map to rbind it into data.frames representing the classes and use do.call again to combine into a single data.frame. In this example, the resulting rownames represent the combination of class and band combination (eg., class-1.B2-3) ``` spectral.sep <- lapply(all_pairwise(2:7)[[1]], \(i) { lapply(unique(x$classes), \(j) { separability(x[x$classes == j,][,i[1]], x[x$classes == j,][,i[2]]) }) }) names(spectral.sep) <- unlist(lapply(all_pairwise(2:7)[[1]], \(i) paste0("B",paste(i,collapse="-")))) spectral.sep <- do.call(Map, c(f = rbind, spectral.sep)) names(spectral.sep) <- paste("class", 1:5) spectral.sep <- do.call(rbind, spectral.sep) ``` Sorry if I misunderstood you and this is off base. Best, Jeff Jeffrey S. Evans, Ph.D., | Senior Landscape Ecologist & Biometrician The Nature Conservancy | Global Protect, Science Visiting Professor | University of Wyoming | Ecosystem Sciences Laramie, WY | ***@***.******@***.***> | (970) 672-6766<tel:(970)%20672-6766> ORCID-ID 0000-0002-5533-7044 TNC Biography<https://www.nature.org/en-us/about-us/who-we-are/our-people/jeff-evans/> – Google Scholar<https://scholar.google.com/citations?user=FwU_ppgAAAAJ&hl=en> From: dhombios ***@***.***> Sent: Wednesday, November 13, 2024 7:58 AM To: jeffreyevans/spatialEco ***@***.***> Cc: Subscribed ***@***.***> Subject: [jeffreyevans/spatialEco] Question: is separability method from this package compatible with tidy? (Issue #67) I have a database in which columns represent different features of a dataset with an additional column that contains the category to which each observation belong (true or false). Due to the large amount of features available, I need to eliminate the worst ones For that purpose, separability method provided by this project seems particularly useful, as it provides a fast coarse estimation of how useful each feature can be for classifying new observations. For that reason, I'm trying to apply it to each column of the dplyr object after separating observations according to their category As this package seems to be compatible with tidyverse, is this function compatible with summarize? Does this package provide any abstraction that could simplify doing this? — Reply to this email directly, view it on GitHub<#67>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ACLKH76N64SDAY76LEDPMF32ANSFXAVCNFSM6AAAAABRWV7YJGVHI2DSMVQWIX3LMV43ASLTON2WKOZSGY2TKNZZGM2TENI>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.******@***.***>>

[dhombios]

Thanks for your help! That's what I was looking for. I'll try to make a wrapper for that function, so it can be integrated into dplyr workflows easily (handling that transformation internally)