Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[r] Add Seurat -> SOMA Ingestor #1146

Merged
merged 52 commits into from
Apr 20, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
9ec57ea
Implement write_soma
mojaveazure Mar 20, 2023
99d5579
Add roxygen template
mojaveazure Mar 21, 2023
d850e13
Reorganize the code
mojaveazure Mar 22, 2023
0770648
Add docs for `write_soma`
mojaveazure Mar 22, 2023
ec95f6f
Improved type testing and warnings in write_soma.data.frame
mojaveazure Mar 24, 2023
0e7dd5d
Index SOMADataFrames by soma_joinid
mojaveazure Mar 24, 2023
c48bd95
Add tests for R-object methods for `write_soma()`
mojaveazure Mar 24, 2023
f80fb0e
Matrix is annoying
mojaveazure Mar 24, 2023
b1d257f
Hide info messages
mojaveazure Mar 24, 2023
ec947a8
Adjustments to write_soma.Graph
mojaveazure Mar 24, 2023
a6a21fe
Add tests for writing Seurat subobjects
mojaveazure Mar 24, 2023
3be6c92
Add tests for writing Seurat objects
mojaveazure Mar 24, 2023
38bf272
Add logging
mojaveazure Mar 24, 2023
6eb78e4
Improve `pad_matrix` for scale.data slot
mojaveazure Mar 27, 2023
ebfcce3
Remove unnecessary code
mojaveazure Mar 27, 2023
0d05d95
Replace lambdas with functions
mojaveazure Mar 28, 2023
0724d3a
Update apis/r/R/write_soma.R
mojaveazure Mar 30, 2023
112325b
Update apis/r/R/write_soma.R
mojaveazure Mar 30, 2023
78371c0
Update apis/r/R/write_soma.R
mojaveazure Mar 30, 2023
4b24ef0
Update docs from GH changes
mojaveazure Apr 6, 2023
d10f920
Update docs for `write_soma.data.frame`
mojaveazure Apr 6, 2023
bbd78b0
Change to in
mojaveazure Apr 6, 2023
bbfb960
Rename test file with `-`
mojaveazure Apr 6, 2023
6a07588
Change `soma` param to `soma_parent`
mojaveazure Apr 6, 2023
a5e73f9
Update tests with new parameter names
mojaveazure Apr 6, 2023
9396b05
Simplify `user_dir()`
mojaveazure Apr 6, 2023
c13e8a7
Always write reductions as sparse arrays
mojaveazure Apr 6, 2023
777a61e
Don't automatically overwrite `soma_joinid` in `write_soma.data.frame()`
mojaveazure Apr 6, 2023
8265aff
Error out when trying to write out data frames with non-atomic columns
mojaveazure Apr 6, 2023
dc05dd5
Update docs to reflect always writing sparsely
mojaveazure Apr 7, 2023
3fd1023
Remove automatic URI creation for Seurat objects
mojaveazure Apr 12, 2023
f1e22c7
Remove unused function
mojaveazure Apr 12, 2023
677b716
Spell out 'layer' for clarity
mojaveazure Apr 12, 2023
76d5da3
Change default SOMADataFrame index to `obs_id` or `var_id`
mojaveazure Apr 12, 2023
df56c32
More informative variable names
mojaveazure Apr 12, 2023
a17215e
Update tests for obs_id
mojaveazure Apr 12, 2023
0e62571
Update apis/r/R/write_seurat.R
mojaveazure Apr 13, 2023
2200e55
Update apis/r/R/write_seurat.R
mojaveazure Apr 13, 2023
17605ba
Update apis/r/R/write_soma.R
mojaveazure Apr 13, 2023
e05c6dc
Update apis/r/R/write_soma.R
mojaveazure Apr 13, 2023
4a18fc3
Fix typo
mojaveazure Apr 13, 2023
bc4fccc
Remove `overwrite`
mojaveazure Apr 13, 2023
f0bccf4
Replace `overwrite` with `absolute`
mojaveazure Apr 13, 2023
e9b9079
Error when trying to write a sparse matrix densely
mojaveazure Apr 13, 2023
37d065a
Add logging when padding matrix
mojaveazure Apr 13, 2023
a1f7c0e
Proper spdl formatting
mojaveazure Apr 13, 2023
e816be9
Better variable names
mojaveazure Apr 13, 2023
d4d52d6
Remove unused code
mojaveazure Apr 13, 2023
e398057
Let SOMADataFrame validate soma_joinid
mojaveazure Apr 13, 2023
861efa7
Allow the user to set their own index when writing data frames
mojaveazure Apr 18, 2023
5f74b0c
Update variable name
mojaveazure Apr 20, 2023
cb16bda
Update changelog
mojaveazure Apr 20, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions apis/r/DESCRIPTION
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Title: TileDB SOMA
Description: Interface for working with 'TileDB'-based Stack of Matrices,
Annotated ('SOMA'): an open data model for representing annotated matrices,
like those commonly used for single cell data analysis.
Version: 0.0.0.9019
Version: 0.0.0.9020
Authors@R: c(
person(given = "Aaron",
family = "Wolen",
Expand Down Expand Up @@ -48,7 +48,8 @@ Imports:
dplyr,
data.table,
spdl,
rlang
rlang,
tools
LinkingTo:
Rcpp,
RcppSpdlog
Expand All @@ -61,7 +62,8 @@ Suggests:
withr,
testthat (>= 3.0.0),
pbmc3k.tiledb,
SeuratObject (>= 4.1.0)
SeuratObject (>= 4.1.0),
datasets
VignetteBuilder: knitr
Config/testthat/edition: 2
OS_type: unix
14 changes: 14 additions & 0 deletions apis/r/NAMESPACE
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,16 @@ S3method("[[<-",MappingBase)
S3method(as.list,MappingBase)
S3method(length,MappingBase)
S3method(names,MappingBase)
S3method(pad_matrix,default)
S3method(pad_matrix,matrix)
S3method(write_soma,Assay)
S3method(write_soma,DimReduc)
S3method(write_soma,Graph)
S3method(write_soma,Matrix)
S3method(write_soma,Seurat)
S3method(write_soma,TsparseMatrix)
S3method(write_soma,data.frame)
S3method(write_soma,matrix)
export(ConfigList)
export(EphemeralCollection)
export(EphemeralExperiment)
Expand Down Expand Up @@ -40,6 +50,7 @@ export(TileDBArray)
export(TileDBGroup)
export(TileDBObject)
export(nnz)
export(pad_matrix)
export(shape)
export(show_package_versions)
export(soma_array_reader)
Expand All @@ -51,10 +62,12 @@ export(tiledbsoma_stats_dump)
export(tiledbsoma_stats_enable)
export(tiledbsoma_stats_reset)
export(tiledbsoma_stats_show)
export(write_soma)
import(R6)
import(methods)
import(utils)
importFrom(Matrix,as.matrix)
importFrom(Matrix,sparseMatrix)
importFrom(Rcpp,evalCpp)
importFrom(arrow,RecordBatch)
importFrom(arrow,concat_arrays)
Expand All @@ -68,6 +81,7 @@ importFrom(glue,glue_collapse)
importFrom(methods,as)
importFrom(methods,new)
importFrom(methods,validObject)
importFrom(rlang,is_integerish)
importFrom(rlang,is_na)
importFrom(spdl,debug)
importFrom(spdl,info)
Expand Down
1 change: 1 addition & 0 deletions apis/r/NEWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,4 @@
* Added `PlatformConfig` and `SOMATileDBContext` classes to handle SOMA and TileDB configuration
* Add Seurat outgestors for `SOMAExperimentAxisQuery` objects
* Numeric coordinates passed to SOMADataFrame$read() are now automatically upcast to int64 when necessary
* Add ingestors to read data from `Seurat` objects
4 changes: 3 additions & 1 deletion apis/r/R/PlatformConfig.R
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,9 @@ PlatformConfig <- R6::R6Class(
f = union,
x = lapply(
X = self$platforms(),
FUN = \(p) private$.data[[p]]$keys()
FUN = function(p) {
private$.data[[p]]$keys()
}
)
)
return(params)
Expand Down
2 changes: 1 addition & 1 deletion apis/r/R/SOMATileDBContext.R
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@ SOMATileDBContext <- R6::R6Class(
}
return(tryCatch(
expr = names(as.vector(tiledb::config(private$.tiledb_ctx))),
error = \(...) NULL
error = null
))
}
)
Expand Down
114 changes: 114 additions & 0 deletions apis/r/R/pad_matrix.R
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
#' Pad a Matrix
#'
#' @param x A matrix-like object
#' @param rowidx,colidx Integer indices for rows and columns from
#' \code{x} to \code{shape}
#' @param shape Target shape to pad out to
#' @template param-dots-method
#'
#' @return A matrix containing the data of \code{x} padded out to \code{shape}
#'
#' @keywords internal
#'
#' @export
#'
pad_matrix <- function(x, ...) {
UseMethod(generic = 'pad_matrix', object = x)
}

#' Pad a sparse Matrix with additional rows or columns
#'
#' @param x A dgTMatrix
#' @param colnames,rownames A vector of column or row names
#' to add to the matrix.
#' @param returns A padded matrix containing all provided
#' row/column names
#'
#' @importFrom Matrix sparseMatrix
#'
#' @noRd
#'
#' @method pad_matrix default
#' @export
#'
pad_matrix.default <- function(x, rownames = NULL, colnames = NULL, ...) {
stopifnot(
inherits(x, "Matrix"),
is.character(colnames) || is.character(rownames)
)
# lookup table for Matrix representations
mat_rep <- switch(
EXPR = class(x),
dgTMatrix = "T",
dgCMatrix = "C",
dgRMatrix = "R",
stop("Untested Matrix object representation")

)
new_rownames <- setdiff(rownames, rownames(x))
new_colnames <- setdiff(colnames, colnames(x))
dtype <- typeof(methods::slot(object = x, name = 'x'))
if (!is_empty(new_rownames)) {
rpad <- Matrix::sparseMatrix(
i = integer(0L),
j = integer(0L),
x = vector(mode = dtype, length = 0L),
dims = c(length(new_rownames), ncol(x)),
dimnames = list(new_rownames, colnames(x)),
repr = mat_rep
)
x <- rbind(x, rpad)
}
if (!is_empty(new_colnames)) {
cpad <- Matrix::sparseMatrix(
i = integer(0L),
j = integer(0L),
x = vector(mode = dtype, length = 0L),
dims = c(nrow(x), length(new_colnames)),
dimnames = list(rownames(x), new_colnames),
repr = mat_rep
)
x <- cbind(x, cpad)
}
x
}

#' @param sparse Return a \link[Matrix::TsparseMatrix-class]{sparse matrix}
#' @rdname pad_matrix
#'
#' @method pad_matrix matrix
#' @export
#'
pad_matrix.matrix <- function(x, rowidx, colidx, shape, sparse = FALSE, ...) {
stopifnot(
rlang::is_integerish(shape, n = 2L, finite = TRUE) && all(shape > 0L),
rlang::is_integerish(rowidx, finite = TRUE) && all(rowidx > 0L),
rlang::is_integerish(colidx, finite = TRUE) && all(colidx > 0L),
all(dim(x) == c(length(rowidx), length(colidx))),
is_scalar_logical(sparse)
)
if (!all(rowidx) <= shape[1L]) {
stop('rowidx')
} else if (!all(colidx <= shape[2L])) {
stop('colidx')
}
type <- typeof(x)
type <- match.arg(arg = type, choices = c('integer', 'double', 'logical'))
mat <- if (isTRUE(sparse)) {
Matrix::sparseMatrix(
i = integer(),
j = integer(),
x = switch(EXPR = type, logical = logical(), numeric()),
dims = shape,
repr = 'T'
)
} else {
matrix(
data = vector(mode = type, length = prod(shape)),
nrow = shape[1L],
ncol = shape[2L]
)
}
mat[rowidx, colidx] <- x
return(mat)
}
5 changes: 5 additions & 0 deletions apis/r/R/utils.R
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,11 @@ rename <- function(x, names) {
if (missing(x) || is.null(x) || length(x) == 0) y else x
}

err_to_warn <- function(err, immediate. = TRUE) {
warning(conditionMessage(err), call. = FALSE, immediate. = immediate.)
return(invisible(err))
}

null <- function(...) {
return(NULL)
}
Expand Down
Loading