Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add tile statistics to mbtiles #986

Merged
merged 30 commits into from
Nov 13, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
3016d16
Add stats command
sharkAndshark Nov 1, 2023
99ffdc2
Inline page_size query
sharkAndshark Nov 8, 2023
d68f612
Use MbtType directly
sharkAndshark Nov 8, 2023
8b7ff70
Use bounds and assert_yaml_snapshot
sharkAndshark Nov 9, 2023
21b4180
fmt and clippy
sharkAndshark Nov 9, 2023
ff5fd34
Update expected stats.txt
sharkAndshark Nov 9, 2023
b9668d0
Use size-format to prettify output
sharkAndshark Nov 9, 2023
25f07b9
Fmt and clippy
sharkAndshark Nov 9, 2023
9a7e5b8
Use epsg 4326
sharkAndshark Nov 9, 2023
a0c27e7
Update statistics structure
sharkAndshark Nov 9, 2023
fb878ca
Refactor
sharkAndshark Nov 9, 2023
8a33a60
Add document about mbtiles stats
sharkAndshark Nov 10, 2023
549bfaa
Update doc link
sharkAndshark Nov 10, 2023
7befd50
Code style update
sharkAndshark Nov 10, 2023
5daea72
Update test for coord transformation
sharkAndshark Nov 10, 2023
7c3b83c
Update naming
sharkAndshark Nov 10, 2023
1030ece
Inlining
sharkAndshark Nov 10, 2023
f56193f
Update test for meter_to_lnglat
sharkAndshark Nov 10, 2023
b45285f
Update crate importing
sharkAndshark Nov 10, 2023
9d5e950
Right align numbers
sharkAndshark Nov 11, 2023
2083448
Add error msg for displaying statistics
sharkAndshark Nov 12, 2023
8865786
Make table title upcase
sharkAndshark Nov 12, 2023
7a0c9e5
Add epmty file test
sharkAndshark Nov 12, 2023
b73684a
Formar SQL
sharkAndshark Nov 13, 2023
6c63f51
Cleanup display implementation
sharkAndshark Nov 13, 2023
eefaf72
Cleanup cargo.toml
sharkAndshark Nov 13, 2023
005f035
Keep links still in document
sharkAndshark Nov 13, 2023
1e76b8a
Update stats document
sharkAndshark Nov 13, 2023
ea979b3
Cleanup
sharkAndshark Nov 13, 2023
a0f2079
Merge branch 'main' into mbstats
nyurik Nov 13, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ actix-http = "3"
actix-rt = "2"
actix-web = "4"
anyhow = "1.0"
approx = "0.5.1"
async-trait = "0.1"
bit-set = "0.5.3"
brotli = "3"
Expand Down Expand Up @@ -52,6 +53,7 @@ serde = { version = "1", features = ["derive"] }
serde_json = "1"
serde_with = "3"
serde_yaml = "0.9"
size_format = "1.0.2"
spreet = { version = "0.9", default-features = false }
sqlite-hashes = { version = "0.5", default-features = false, features = ["md5", "window", "hex"] }
sqlx = { version = "0.7", features = ["sqlite", "runtime-tokio"] }
Expand Down
19 changes: 19 additions & 0 deletions docs/src/55-mbtiles-stats.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Get a tile statistics from MBTiles file

For the concern of efficiency, you could figure out the page size and file size, tile size and bounds(represented as WGS 84 latitude and longitude values) of covered area with `mbtiles stats` command.

```shell
File: /path/to/world_cities.mbtiles
FileSize: 48.00KiB
Schema: flat
Page size: 4.00KiB
| Zoom | Count |Smallest | Largest | Average | BBox |
| 0| 1| 1.08KiB| 1.08KiB| 1.08KiB|-179.99999997494382,-85.05112877764508,180.00000015460688,85.05112879314403|
| 1| 4| 160B| 650B| 366B|-179.99999997494382,-85.05112877764508,180.00000015460688,85.05112879314403|
| 2| 7| 137B| 495B| 239B|-179.99999997494382,-66.51326042021836,180.00000015460688,66.51326049182072|
| 3| 17| 67B| 246B| 134B|-134.99999995874995,-40.9798980140281,180.00000015460688,66.51326049182072|
| 4| 38| 64B| 175B| 86B|-134.99999995874995,-40.9798980140281,180.00000015460688,66.51326049182072|
| 5| 57| 64B| 107B| 72B|-123.74999995470151,-40.9798980140281,180.00000015460688,61.60639642757953|
| 6| 72| 64B| 97B| 68B|-123.74999995470151,-40.9798980140281,180.00000015460688,61.60639642757953|
| all| 196| 64B| 1.0KiB| 96B|
```
1 change: 1 addition & 0 deletions docs/src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,4 +26,5 @@
- [MBTiles Copying / Diffing](52-mbtiles-copy.md)
- [MBTiles Validation](53-mbtiles-validation.md)
- [MBTiles Schemas](54-mbtiles-schema.md)
- [MBTiles statistics](55-mbtiles-stats.md)
- [Development](60-development.md)

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 4 additions & 2 deletions mbtiles/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,10 @@ enum-display.workspace = true
futures.workspace = true
log.workspace = true
martin-tile-utils.workspace = true
serde.workspace = true
serde_json.workspace = true
serde.workspace = true
serde_with.workspace = true
size_format.workspace = true
sqlite-hashes.workspace = true
sqlx.workspace = true
thiserror.workspace = true
Expand All @@ -37,9 +38,10 @@ tokio = { workspace = true, features = ["rt-multi-thread"], optional = true }
[dev-dependencies]
# For testing, might as well use the same async framework as the Martin itself
actix-rt.workspace = true
approx.workspace = true
ctor.workspace = true
env_logger.workspace = true
insta = { workspace = true, features = ["toml"] }
insta = { workspace = true, features = ["toml", "yaml"] }
pretty_assertions.workspace = true
rstest.workspace = true

Expand Down
10 changes: 10 additions & 0 deletions mbtiles/src/bin/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,9 @@ enum Commands {
/// MBTiles file to read from
file: PathBuf,
},
/// Gets tile statistics from MBTiels file
#[command(name = "stats")]
Stats { file: PathBuf },
/// Gets a single value from the MBTiles metadata table.
#[command(name = "meta-get")]
MetaGetValue {
Expand Down Expand Up @@ -114,6 +117,13 @@ async fn main_int() -> anyhow::Result<()> {
let mbt = Mbtiles::new(file.as_path())?;
mbt.validate(integrity_check, update_agg_tiles_hash).await?;
}
Commands::Stats { file } => {
let mbt = Mbtiles::new(file.as_path())?;
let mut conn = mbt.open_readonly().await?;

let statistics = mbt.statistics(&mut conn).await?;
println!("{statistics}");
}
}

Ok(())
Expand Down
Loading
Loading