feat(webserver): implement new storage layer for repository meta #1750

boxbeam · 2024-04-01T21:50:46Z

Relates to TAB-533

codecov · 2024-04-01T21:58:30Z

Codecov Report

Attention: Patch coverage is 0.68027% with 146 lines in your changes are missing coverage. Please review.

Project coverage is 53.17%. Comparing base (837e36d) to head (cd19700).
Report is 1 commits behind head on main.

Files	Patch %	Lines
ee/tabby-webserver/src/repositories/resolve.rs	0.00%	83 Missing ⚠️
ee/tabby-webserver/src/hub/api.rs	0.00%	21 Missing ⚠️
crates/tabby-scheduler/src/dataset.rs	0.00%	16 Missing ⚠️
ee/tabby-webserver/src/repositories/mod.rs	0.00%	6 Missing ⚠️
ee/tabby-webserver/src/service/repository.rs	0.00%	6 Missing ⚠️
ee/tabby-webserver/src/handler.rs	0.00%	5 Missing ⚠️
crates/tabby-scheduler/src/lib.rs	0.00%	4 Missing ⚠️
ee/tabby-webserver/src/path.rs	0.00%	3 Missing ⚠️
ee/tabby-webserver/src/cron/mod.rs	0.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1750      +/-   ##
==========================================
- Coverage   53.71%   53.17%   -0.54%     
==========================================
  Files         115      116       +1     
  Lines        9928     9976      +48     
==========================================
- Hits         5333     5305      -28     
- Misses       4595     4671      +76

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

crates/tabby-common/src/config.rs

crates/tabby-common/src/path.rs

ee/tabby-webserver/src/repositories/resolve.rs

crates/tabby-common/src/lib.rs

ee/tabby-db/Cargo.toml

ee/tabby-db/schema.sqlite

Cargo.toml

ee/tabby-webserver/src/repositories/resolve.rs

crates/tabby-common/src/config.rs

ee/tabby-webserver/src/hub/api.rs

ee/tabby-webserver/src/repositories/resolve.rs

crates/tabby-common/src/config.rs

crates/tabby-scheduler/src/dataset.rs

crates/tabby-common/src/config.rs

ee/tabby-webserver/src/hub/api.rs

ee/tabby-webserver/src/path.rs

wsxiaoys · 2024-04-05T16:09:36Z

ee/tabby-webserver/src/repositories/resolve.rs

-        let mut repository_lookup = self.repository_lookup.write().unwrap();
-        debug!("Reloading repositoriy metadata...");
-        *repository_lookup = load_meta(new_repositories);
+    pub fn latest_version(&self) -> Result<u64> {


Suggested change

pub fn latest_version(&self) -> Result<u64> {

fn latest_version(&self) -> Result<u64> {

ee/tabby-webserver/src/repositories/resolve.rs

wsxiaoys · 2024-04-05T16:12:58Z

ee/tabby-webserver/src/repositories/resolve.rs

+        let bucket: Bucket<_, String> = self.cache.bucket(Some(META_BUCKET))?;
+        if !bucket.contains(&META_BUCKET_VERSION_KEY.to_string())? {
+            self.update_latest_version(self.get_next_version()?)?;
+        }


Suggested change

let bucket: Bucket<_, String> = self.cache.bucket(Some(META_BUCKET))?;

if !bucket.contains(&META_BUCKET_VERSION_KEY.to_string())? {

self.update_latest_version(self.get_next_version()?)?;

}

if there's no latest version, it means the storage is simply empty.

ee/tabby-webserver/src/repositories/mod.rs

wsxiaoys · 2024-04-05T16:24:25Z

ee/tabby-webserver/src/repositories/resolve.rs

+type RepositoryBucket<'a> = Bucket<'a, String, kv::Json<RepositoryMeta>>;
+
+static META_BUCKET: &str = "meta";
+static META_BUCKET_VERSION_KEY: &str = "version";

 impl RepositoryCache {


please review visibility of functions in this struct.

boxbeam requested a review from wsxiaoys April 1, 2024 21:50

boxbeam changed the title ~~Begin implementing db for RepositoryMeta~~ feat: Implement new db for repository meta Apr 2, 2024

boxbeam marked this pull request as ready for review April 3, 2024 21:13

wsxiaoys changed the title ~~feat: Implement new db for repository meta~~ feat(webserver): Implement new storage layer for repository meta Apr 5, 2024

wsxiaoys reviewed Apr 5, 2024

View reviewed changes

crates/tabby-common/src/config.rs Outdated Show resolved Hide resolved

wsxiaoys reviewed Apr 5, 2024

View reviewed changes

crates/tabby-common/src/path.rs Outdated Show resolved Hide resolved

wsxiaoys reviewed Apr 5, 2024

View reviewed changes

ee/tabby-webserver/src/repositories/resolve.rs Outdated Show resolved Hide resolved

wsxiaoys reviewed Apr 5, 2024

View reviewed changes

crates/tabby-common/src/lib.rs Outdated Show resolved Hide resolved

wsxiaoys reviewed Apr 5, 2024

View reviewed changes

ee/tabby-db/Cargo.toml Outdated Show resolved Hide resolved

wsxiaoys reviewed Apr 5, 2024

View reviewed changes

ee/tabby-db/schema.sqlite Outdated Show resolved Hide resolved

wsxiaoys reviewed Apr 5, 2024

View reviewed changes

Cargo.toml Outdated Show resolved Hide resolved

wsxiaoys reviewed Apr 5, 2024

View reviewed changes

ee/tabby-webserver/src/repositories/resolve.rs Show resolved Hide resolved

wsxiaoys reviewed Apr 5, 2024

View reviewed changes

ee/tabby-webserver/src/repositories/resolve.rs Outdated Show resolved Hide resolved

wsxiaoys reviewed Apr 5, 2024

View reviewed changes

ee/tabby-webserver/src/repositories/resolve.rs Outdated Show resolved Hide resolved

wsxiaoys marked this pull request as draft April 5, 2024 05:23

boxbeam and others added 11 commits April 5, 2024 10:58

Begin implementing db for RepositoryMeta

c65a9d6

Add get_repository_meta

a1304e8

[autofix.ci] apply automated fixes

0617b10

[autofix.ci] apply automated fixes (attempt 2/3)

11892e8

Rewrite with kv

b630c92

[autofix.ci] apply automated fixes

04c421d

Use scheduler indexing logic for RepositoryCache

f56f9e2

Revert change to tabby-scheduler

0714b36

Rewrite

b89d1cb

Use rotating cache version

795e9de

Fix conflicts

51d5512

boxbeam force-pushed the repository-meta-db branch from 71faf5b to 51d5512 Compare April 5, 2024 15:03

Revert db cargo.toml

8ffb2c8

boxbeam marked this pull request as ready for review April 5, 2024 15:06

wsxiaoys reviewed Apr 5, 2024

View reviewed changes

crates/tabby-common/src/config.rs Outdated Show resolved Hide resolved

wsxiaoys reviewed Apr 5, 2024

View reviewed changes

ee/tabby-webserver/src/hub/api.rs Outdated Show resolved Hide resolved

wsxiaoys reviewed Apr 5, 2024

View reviewed changes

ee/tabby-webserver/src/hub/api.rs Outdated Show resolved Hide resolved

wsxiaoys requested changes Apr 5, 2024

View reviewed changes

ee/tabby-webserver/src/repositories/resolve.rs Outdated Show resolved Hide resolved

wsxiaoys marked this pull request as draft April 5, 2024 15:19

wsxiaoys changed the title ~~feat(webserver): Implement new storage layer for repository meta~~ feat(webserver): implement new storage layer for repository meta Apr 5, 2024

wsxiaoys reviewed Apr 5, 2024

View reviewed changes

ee/tabby-webserver/src/repositories/resolve.rs Outdated Show resolved Hide resolved

Apply suggestions

f6dce73

boxbeam force-pushed the repository-meta-db branch from 28c788f to f6dce73 Compare April 5, 2024 15:36

wsxiaoys reviewed Apr 5, 2024

View reviewed changes

crates/tabby-common/src/config.rs Outdated Show resolved Hide resolved

wsxiaoys reviewed Apr 5, 2024

View reviewed changes

crates/tabby-scheduler/src/dataset.rs Outdated Show resolved Hide resolved

Apply suggestions

bcf69ca

wsxiaoys reviewed Apr 5, 2024

View reviewed changes

crates/tabby-common/src/config.rs Show resolved Hide resolved

Readd start_snapshot

443b281

wsxiaoys reviewed Apr 5, 2024

View reviewed changes

ee/tabby-webserver/src/hub/api.rs Outdated Show resolved Hide resolved

wsxiaoys reviewed Apr 5, 2024

View reviewed changes

ee/tabby-webserver/src/path.rs Show resolved Hide resolved

wsxiaoys reviewed Apr 5, 2024

View reviewed changes

ee/tabby-webserver/src/path.rs Outdated Show resolved Hide resolved

Make update_latest_version

323a3c7

boxbeam force-pushed the repository-meta-db branch from 2b579c3 to 323a3c7 Compare April 5, 2024 16:02

boxbeam marked this pull request as ready for review April 5, 2024 16:04

[autofix.ci] apply automated fixes

cd19700

wsxiaoys reviewed Apr 5, 2024

View reviewed changes

wsxiaoys marked this pull request as draft April 5, 2024 16:20

wsxiaoys reviewed Apr 5, 2024

View reviewed changes

ee/tabby-webserver/src/repositories/mod.rs Show resolved Hide resolved

wsxiaoys reviewed Apr 5, 2024

View reviewed changes

boxbeam closed this Apr 5, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(webserver): implement new storage layer for repository meta #1750

feat(webserver): implement new storage layer for repository meta #1750

boxbeam commented Apr 1, 2024 •

edited

Loading

codecov bot commented Apr 1, 2024 •

edited

Loading

wsxiaoys Apr 5, 2024

wsxiaoys Apr 5, 2024

wsxiaoys Apr 5, 2024

wsxiaoys Apr 5, 2024

	pub fn latest_version(&self) -> Result<u64> {
	fn latest_version(&self) -> Result<u64> {

feat(webserver): implement new storage layer for repository meta #1750

feat(webserver): implement new storage layer for repository meta #1750

Conversation

boxbeam commented Apr 1, 2024 • edited Loading

codecov bot commented Apr 1, 2024 • edited Loading

Codecov Report

wsxiaoys Apr 5, 2024

Choose a reason for hiding this comment

wsxiaoys Apr 5, 2024

Choose a reason for hiding this comment

wsxiaoys Apr 5, 2024

Choose a reason for hiding this comment

wsxiaoys Apr 5, 2024

Choose a reason for hiding this comment

boxbeam commented Apr 1, 2024 •

edited

Loading

codecov bot commented Apr 1, 2024 •

edited

Loading