rewrite - v3.0.0 - move to database and sveltekit

components.json

@@ -0,0 +1,14 @@
+{
+	"$schema": "https://shadcn-svelte.com/schema.json",
+	"style": "new-york",
+	"tailwind": {
+		"config": "tailwind.config.ts",
+		"css": "src/app.css",
+		"baseColor": "neutral"
+	},
+	"aliases": {
+		"components": "$lib/components",
+		"utils": "$lib/utils"
+	},
+	"typescript": true
+}

docker-compose.yaml

@@ -0,0 +1,9 @@
+services:
+  app:
+    build:
+      context: .
+    ports:
+      - "8090:8090"  # SvelteKit app
+      - "3000:3000"
+    env_file:
+      - .env

docs/Changelog/3.0.0.md

@@ -0,0 +1,26 @@
+---
+title: Version 3.0.0
+date: 09-24-2024
+tags:
+  - 3.0.0
+
+---
+
+## Backend Rewrite
+* **Technology Shift:** The backend has been completely rewritten, moving from Python to SwellKit.
+* **Database:** Transitioned from file-based management to using PocketBase for the backend database.
+
+This version is a full rewrite of the backend in sveltekit. The choice was made as managing both
+the frontend and backend using the same language and framework simplifies the architecture and
+management a lot. Additionally pocketbase provides a nice `js` interface which is also nice.
+
+## UI Enhancements
+- The UI has been refined using ShadeCN, and Tailwind CSS.
+
+## New Features
+* The new version introduces support for using relative paths in wiki links and images.
+* Tag Management
+  * Tag Pages: Tags now get their own dedicated pages.
+  * Menu Bar Icon: A new menu bar icon has been added to view the list of all tags.
+* Simplified Python CLI Tool
+  * The Python CLI tool has been simplified to remove most dependencies.

docs/Development/APIs.md

@@ -0,0 +1,79 @@
+---
+title: API overview
+date: 22-09-2024
+tags:
+  - api
+  - backend
+publish: true
+---
+
+This section details the core operations of the app, specifically the backend.
+The backend exposes various REST API endpoints which can serve different types of requests related
+to markdown files. Below we detail each of them.
+
+## Overview
+This document provides an overview of the API endpoints for the Marco Polo's application. The application is built using SvelteKit for the backend API and PocketBase for the database. A Python package is also used to expose a CLI for uploading files to the server.
+## Endpoints
+### Upload API
+
+> [!important]
+> **Endpoint:** ==/api/upload==
+> **Method:** *POST*
+
+- **Description** Uploads markdown files to the server, sets up the database, parses HTML, and stores the compiled results and original files in the database.
+- **Parameters**
+  - `file`: The markdown file to be uploaded.
+  - `url`: Absolute path of file from root of vault.
+
+#### Details
+
+Refer [[Markdown Rendering]]
+
+---
+### LS API
+- **Endpoint** `/api/ls`
+- **Method** GET
+- **Description** Builds a file tree from the URL field in each record of the MDBase collection.
+- **Responses**:
+  - `200 OK`: Returns a JSON response with the file tree.
+  - `500 Internal Server Error`: Error in processing the request.
+---
+### Search API
+- **Endpoint** `/api/search`
+- **Method** GET
+- **Description** Performs a fuzzy search through the content stored in the database.
+- **Parameters**:
+  - `query`: The search query string.
+- **Responses**:
+  - `200 OK`: Returns search results with snippets containing matches.
+  - `404 Not Found`: No matches found.
+---
+### Links API
+- **Endpoint** `/api/links`
+- **Method** GET
+- **Description** Retrieves all links and backlinks for a given markdown file.
+- **Parameters**
+  - `url`: The URL of the markdown file.
+- **Responses**
+  - `200 OK`: Returns forward and backward links.
+  - `404 Not Found`: File not found.
+---
+### Backlinks API
+- **Endpoint** `/api/backlinks`
+- **Method** GET
+- **Description** Retrieves only the backlinks for a given markdown file.
+- **Parameters**
+  - `url`: The URL of the markdown file.
+- **Responses**:
+  - `200 OK`: Returns backlinks.
+  - `404 Not Found`: File not found.
+---
+### Image API
+- **Endpoint** `/api/image`
+- **Method** GET
+- **Description**: Fetches images from the database.
+- **Parameters**
+  - `url`: The URL of the image file.
+- **Responses**:
+  - `200 OK`: Returns the image file.
+  - `404 Not Found`: Image not found.

docs/Development/Markdown Rendering.md

@@ -0,0 +1,35 @@
+---
+title: Rendering Markdown as HTML
+date: 09-22-2024
+tags:
+  - markdown
+  - mdsvex
+publish: true
+---
+
+This document provides an in-depth explanation of the markdown parsing process used in the Marco Polo's application. The parsing is implemented using the `md-swex` library, which allows for the integration of Svelte components within markdown files.
+## Parsing Process
+1. **Markdown to HTML Conversion**:
+   - The `md-swex` library is used to parse markdown files and convert them into HTML blocks.
+   - The `compile` function of `md-swex` is utilized to directly compile markdown content.
+2. **Database Storage**:
+   - Parsed content is stored in the PocketBase database.
+   - Fields include ID, title, parsed HTML content, URL, markdown file, front matter (as JSON), and relational fields for backlinks and forward links.
+3. **Plugins and Extensions**:
+   - **Remark Plugins**: Used for processing markdown abstract syntax trees.
+     - `Remark Math`: Renders LaTeX equations.
+     - `Remark Footnotes`: Processes footnotes.
+     - `Custom Wikilink Plugin`: Resolves relative links.
+     - `Custom Obsidian Image Plugin`: Handles inline images.
+     - `Custom Remark Mermaid Plugin`: Processes Mermaid diagrams.
+4. **Custom Wikilink Plugin**:
+   - Recognizes Wikilink syntax and evaluates relative paths.
+   - Uses front matter to access the file path and resolve links.
+5. **Handling Code Blocks**:
+   - Addresses issues with `md-swex` parsing code blocks as inline HTML.
+   - Cleanup functions remove unwanted syntax to ensure proper rendering.
+6. **Operational Checks**:
+   - Ensures the existence of necessary database collections (e.g., `mdbase`, `attachments`).
+   - Handles file uploads and updates, storing markdown and image files appropriately.
+## Conclusion
+The markdown parsing component is integral to the Marco Polo's application, enabling efficient storage and retrieval of markdown content. The use of `md-swex` and various plugins ensures robust parsing and rendering capabilities.

docs/Development/components.md

@@ -0,0 +1,36 @@
+---
+title: Components behind Markopolis
+tags:
+  - development
+  - working
+date: 09/20/2024
+publish: true
+---
+
+Markopolis is built using 2 frameworks: [sveltekit](https://svelte.dev) and [pocketbase](https://pocketbase.docs).
+Pocketbase is used for database and the backend API is implemented using sveltekit. It additionally has a
+client side convenience python CLI interface.
+The sveltekit webapp exposes APIs for interaction, pre-renders and routes the websites. The functionality of the
+app begins with the python CLI interface. Running the sync command uploads files using an API. The API on
+recieving the file, converts it to html and stores it in the database along with the original file.
+The sveltekit app then uses dynamic routing to search and fetch the file from the database and renders it as a
+webpage.
+
+On the high-level Markopolis consists of a backend and a database. The front-end interacts with the backend
+and requests things needed to render pages. The backend, according to the requests, pulls data from the database
+and returns them.
+The front-end interacts with the backend via REST APIs exposed by the backend and for webpages also uses
+Server Side Rendering(SSR). Below is a diagram illustrating the data and control flow:
+
+```mermaid
+sequenceDiagram
+  Frontend->>Backend: API calls
+  Backend-->>Database: fetch data using PocketBase API
+  Database-->>Backend: Requested data
+  Backend-)Frontend: Requested data
+```
+
+
+## Backend
+
+The backends primary function is