docs: add repository context management documentation (#2093)

* docs: add repository context management documentation

* update

website/blog/2023-10-16-repository-context-for-code-completion/index.md

@@ -160,5 +160,4 @@ We are incredibly enthusiastic about the potential for enhancing the quality and
 To use this repository context feature:
 
 1. Installing [tabby](/docs/installation/).
-2. Navigate to the [Configuration](/docs/configuration#repository-context-for-code-completion) page and set up your `~/.tabby/config.toml`
-3. Finally, run `tabby scheduler` to build an index and unlock the full potential of this innovative feature! 🛠️
+2. Navigate to the [Repository Context](/docs/administration/context) page and follow the instructions to set it up.

website/docs/administration/context/index.mdx

@@ -0,0 +1,42 @@
+---
+sidebar_position: 10
+---
+
+import RepositoryProvidersUrl from "./repository-providers.png";
+import GitUrl from "./git.png";
+import GitHubGitLabUrl from "./github-gitlab.png";
+import SelectRepoUrl from "./select-repo.png";
+import CodeBrowserUrl from "./code-browser.png";
+
+# Repository Context
+
+The repository context is used to connect Tabby with a source code repository from Git, GitHub, GitLab, etc. Tabby fetches the source code from the repository, parses it into an AST, and stores it in the index. During LLM inference, this context is utilized for code completion, as well as chat and search functionalities.
+
+## Adding a Repository Provider
+
+1. Navigate to the **Integrations > Repository Providers** page.
+
+<img src={RepositoryProvidersUrl} alt="Repository Providers" />
+
+2. Click **Create** to begin the process of adding a repository provider.
+
+  - For Git, you only need to fill in the name and the URL of the repository.
+
+    <img src={GitUrl} alt="Git" width="700" />
+
+  - For GitHub / GitLab, a personal access token is required to access private repositories.
+    * Check the instructions in the corresponding tab to create a token. 
+
+    <img src={GitHubGitLabUrl} alt="GitHub or GitLab" width="700" />
+
+    * Once the token is set, you can add the repository by selecting it from the dropdown list.
+
+    <img src={SelectRepoUrl} alt="select-repo" width="700" />
+
+## Verifying the Repository Provider
+
+Once connected, the indexing job will start automatically. You can check the status of the indexing job on the **Information > Jobs** page.
+
+Additionally, you can also visit the **Code Browser** page to view the connected repository.
+
+<img src={CodeBrowserUrl} alt="code browser" width="800" />

website/docs/administration/distributed/index.mdx

@@ -1,3 +1,7 @@
+---
+sidebar_position: 9
+---
+
 import CompletionWorkerUrl from "./completion-worker.png";
 import ChatPlaygroundUrl from "./chat-playground.png";
 

website/docs/administration/sso/index.mdx

@@ -1,3 +1,7 @@
+---
+sidebar_position: 8
+---
+
 import SSOCreateUrl from "./sso_create.png";
 import SSOSigninUrl from "./sso_signin.png";
 

website/docs/configuration.md

@@ -4,90 +4,6 @@ sidebar_position: 5
 
 # ⚙️ Configuration
 
-Tabby server will look for a configuration file at `~/.tabby/config.toml` for advanced features.
-
-### Repository context for code completion
-
-To enable repository level context for code completion, you can add the following to your configuration file:
-
-```toml title="~/.tabby/config.toml"
-# Index three repositories' source code as additional context for code completion.
-
-[[repositories]]
-name = "tabby"
-git_url = "https://github.com/TabbyML/tabby.git"
-
-# git through ssh protocol.
-[[repositories]]
-name = "CTranslate2"
-git_url = "[email protected]:OpenNMT/CTranslate2.git"
-
-# local directory is also supported!
-[[repositories]]
-name = "repository_a"
-git_url = "file:///home/users/repository_a"
-```
-
-Once this is set, you can run `tabby scheduler` to index the source code repository.
-
-:::tip
-By default, `tabby scheduler` runs in a daemon and processes its pipeline every 5 hours. To run the pipeline immediately, use `tabby scheduler --now`.
-:::
-
-```bash title="artifacts produced by tabby scheduler"
-~/.tabby % ls dataset
-data.jsonl
-
-~/.tabby % ls index
-1a8729fa34d844df984b444f4def1456.fast      2ed712d4a7a44ed797dd4ff5ceaf4312.fieldnorm
-b42ca53fe6f94d0c8e96f947318278ba.idx       1a8729fa34d844df984b444f4def1456.fieldnorm 
-2ed712d4a7a44ed797dd4ff5ceaf4312.idx       b42ca53fe6f94d0c8e96f947318278ba.pos
-...
-```
-
-In a code completion request, additional context from the source code repository will be attached to the prompt for better completion quality. For example:
-
-```rust title="Example prompt for code completion, with retrieval augmented enabled"
-// Path: crates/tabby/src/serve/engine.rs
-// fn create_llama_engine(model_dir: &ModelDir) -> Box<dyn TextGeneration> {
-//     let options = llama_cpp_bindings::LlamaEngineOptionsBuilder::default()
-//         .model_path(model_dir.ggml_q8_0_file())
-//         .tokenizer_path(model_dir.tokenizer_file())
-//         .build()
-//         .unwrap();
-//
-//     Box::new(llama_cpp_bindings::LlamaEngine::create(options))
-// }
-//
-// Path: crates/tabby/src/serve/engine.rs
-// create_local_engine(args, &model_dir, &metadata)
-//
-// Path: crates/tabby/src/serve/health.rs
-// args.device.to_string()
-//
-// Path: crates/tabby/src/serve/mod.rs
-// download_model(&args.model, &args.device)
-    } else {
-        create_llama_engine(model_dir)
-    }
-}
-
-fn create_ctranslate2_engine(
-    args: &crate::serve::ServeArgs,
-    model_dir: &ModelDir,
-    metadata: &Metadata,
-) -> Box<dyn TextGeneration> {
-    let device = format!("{}", args.device);
-    let options = CTranslate2EngineOptionsBuilder::default()
-        .model_path(model_dir.ctranslate2_dir())
-        .tokenizer_path(model_dir.tokenizer_file())
-        .device(device)
-        .model_type(metadata.auto_model.clone())
-        .device_indices(args.device_indices.clone())
-        .build()
-        .⮹
-```
-
 ## Usage Collection
 Tabby collects usage stats by default. This data will only be used by the Tabby team to improve its services.
 
@@ -112,4 +28,4 @@ struct HealthState {
 For an up-to-date list of the fields we have collected, please refer to [health.rs](https://github.com/TabbyML/tabby/blob/main/crates/tabby/src/services/health.rs#L11).
 
 ### How to disable it
-To disable usage collection, set the `TABBY_DISABLE_USAGE_COLLECTION` environment variable by `export TABBY_DISABLE_USAGE_COLLECTION=1`.
+To disable usage collection, set the `TABBY_DISABLE_USAGE_COLLECTION` environment variable by `export TABBY_DISABLE_USAGE_COLLECTION=1`.