Skip to content

Commit

Permalink
rename crates
Browse files Browse the repository at this point in the history
  • Loading branch information
tanzaku committed May 4, 2024
1 parent 93f2344 commit 9ef3848
Show file tree
Hide file tree
Showing 52 changed files with 298 additions and 56 deletions.
10 changes: 5 additions & 5 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,14 @@ resolver = "2"
members = [
"crates/lexer-generator",
"crates/parser-generator",
"crates/postgresql-cst-parser",
"crates/postgresql-cst-parser-wasm",
"crates/postgresql-lst-parser",
"crates/postgresql-lst-parser-wasm",
]

default-members = ["crates/postgresql-cst-parser"]
default-members = ["crates/postgresql-lst-parser"]

[workspace.package]
exclude = ["crates/lexer-generator", "crates/parser-generator", "crates/postgresql-cst-parser-wasm"]
exclude = ["crates/lexer-generator", "crates/parser-generator", "crates/postgresql-lst-parser-wasm"]

[profile.release.package.postgresql-cst-parser-wasm]
[profile.release.package.postgresql-lst-parser-wasm]
opt-level = "s"
14 changes: 7 additions & 7 deletions README.ja.md
Original file line number Diff line number Diff line change
@@ -1,29 +1,29 @@
# postgresql-cst-parser
# postgresql-lst-parser

## 概要

`postgresql-cst-parser`は、Pure Rust で開発された PostgreSQL 専用の Concrete Syntax Tree(CST)パーサーです。この文書では、パーサーの特徴、動機、使用方法、および実装の詳細について説明します。
`postgresql-lst-parser`は、Pure Rust で開発された PostgreSQL 専用の Lossless Syntax Tree(LST)パーサーです。この文書では、パーサーの特徴、動機、使用方法、および実装の詳細について説明します。

## 主な特徴

- **自動生成された CST パーサー**: PostgreSQL の文法から自動的に生成されるため、幅広い文法に対応。
- **自動生成された LST パーサー**: PostgreSQL の文法から自動的に生成されるため、幅広い文法に対応。
- **部分的な制限**: スキャナーの一部の実装が不完全であるため、全ての文法に対応しているわけではありません。

## 開発の動機

1. Rust から利用でき、広範な文法をサポートする PostgreSQL の CST パーサーが不足している。
2. [pg_query.rs](https://github.com/pganalyze/pg_query.rs)はとても素晴らしいライブラリだが、CST は構築できず WebAssembly(wasm)でビルドできない。
1. Rust から利用でき、広範な文法をサポートする PostgreSQL の LST パーサーが不足している。
2. [pg_query.rs](https://github.com/pganalyze/pg_query.rs)はとても素晴らしいライブラリだが、LST は構築できず WebAssembly(wasm)でビルドできない。

## 使用方法

以下のコード例のようにして使用します。

```rust
let resolved_root = postgresql_cst_parser::parse("SELECT 1;");
let resolved_root = postgresql_lst_parser::parse("SELECT 1;");
dbg!(resolved_root);
```

さらに、このパーサーを実際に体験してみたい場合は、[こちら](https://tanzaku.github.io/postgresql-cst-parser/)でオンラインで直接試すことができます。実際のコードを入力し、パーサーがどのように動作するかを確認してみましょう。
さらに、このパーサーを実際に体験してみたい場合は、[こちら](https://tanzaku.github.io/postgresql-lst-parser/)でオンラインで直接試すことができます。実際のコードを入力し、パーサーがどのように動作するかを確認してみましょう。

## 実装方法

Expand Down
12 changes: 6 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,29 +1,29 @@
# postgresql-cst-parser
# postgresql-lst-parser

## Overview

The `postgresql-cst-parser` is a PostgreSQL-specific Concrete Syntax Tree (CST) parser developed in Pure Rust. This document describes the parser's features, motivation, usage, and details of its implementation.
The `postgresql-lst-parser` is a PostgreSQL-specific Lossless Syntax Tree (LST) parser developed in Pure Rust. This document describes the parser's features, motivation, usage, and details of its implementation.

## Key Features

- **Automatically Generated CST Parser**: Automatically generated from PostgreSQL grammar, allowing it to support a wide range of syntaxes.
- **Automatically Generated LST Parser**: Automatically generated from PostgreSQL grammar, allowing it to support a wide range of syntaxes.
- **Partial Limitations**: Due to some incomplete implementations in the scanner, it does not support all grammatical structures.

## Motivation for Development

1. There is a lack of PostgreSQL CST (Concrete Syntax Tree) parsers that can be utilized from Rust and support a wide range of syntax.
1. There is a lack of PostgreSQL LST (Lossless Syntax Tree) parsers that can be utilized from Rust and support a wide range of syntax.
1. [pg_query.rs](https://github.com/pganalyze/pg_query.rs) is an excellent library, however, it does not construct CSTs and cannot be built for WebAssembly (wasm).

## Usage

Use it as shown in the following code examples.

```rust
let resolved_root = postgresql_cst_parser::parse("SELECT 1;");
let resolved_root = postgresql_lst_parser::parse("SELECT 1;");
dbg!(resolved_root);
```

If you would like to experience this parser in action, you can try it out directly online [here](https://tanzaku.github.io/postgresql-cst-parser/). Enter your own code to see how the parser operates in real-time.
If you would like to experience this parser in action, you can try it out directly online [here](https://tanzaku.github.io/postgresql-lst-parser/). Enter your own code to see how the parser operates in real-time.


## Implementation
Expand Down
2 changes: 1 addition & 1 deletion build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@ set -eux

cargo run -p lexer-generator --release
cargo run -p parser-generator --release
cargo test --package postgresql-cst-parser --test test -- test_all --exact --show-output
cargo test --package postgresql-lst-parser --test test -- test_all --exact --show-output
2 changes: 1 addition & 1 deletion build_wasm.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

set -eux

cd crates/postgresql-cst-parser-wasm
cd crates/postgresql-lst-parser-wasm
wasm-pack build --release --target web
cp pkg/*.js pkg/*.ts pkg/*.wasm ../../docs/js
cd ../../docs
Expand Down
2 changes: 1 addition & 1 deletion crates/lexer-generator/src/lexer_generator.rs
Original file line number Diff line number Diff line change
Expand Up @@ -176,7 +176,7 @@ pub fn generate() {
.replace("{keyword_map}", &keywords.join("\n"));

let paths = [
"./crates/postgresql-cst-parser/src/lexer/generated.rs",
"./crates/postgresql-lst-parser/src/lexer/generated.rs",
"./crates/parser-generator/src/parser_generator/lexer/generated.rs",
];
for path in paths {
Expand Down
4 changes: 2 additions & 2 deletions crates/parser-generator/src/parser_generator.rs
Original file line number Diff line number Diff line change
Expand Up @@ -158,7 +158,7 @@ fn write_parser_file(
.replace("{end_rule_kind}", r#"TokenKind::RAW("$end".to_string())"#)
.replace("{end_rule_id}", &end_rule_id);

let paths = ["./crates/postgresql-cst-parser/src/parser.rs"];
let paths = ["./crates/postgresql-lst-parser/src/parser.rs"];
for path in paths {
std::fs::write(path, &parser_template).unwrap();
Command::new("rustfmt").arg(path).output().unwrap();
Expand Down Expand Up @@ -194,7 +194,7 @@ fn write_syntax_file(
kinds.join("\n\t")
);

let path = "./crates/postgresql-cst-parser/src/syntax_kind.rs";
let path = "./crates/postgresql-lst-parser/src/syntax_kind.rs";
std::fs::write(path, source).unwrap();
Command::new("rustfmt").arg(path).output().unwrap();
}
Expand Down
25 changes: 0 additions & 25 deletions crates/postgresql-cst-parser/src/lib.rs

This file was deleted.

Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[package]
name = "postgresql-cst-parser-wasm"
name = "postgresql-lst-parser-wasm"
version = "0.1.0"
authors = ["tanzaku <[email protected]>"]
edition = "2018"
Expand All @@ -15,7 +15,7 @@ default = ["console_error_panic_hook"]

[dependencies]
wasm-bindgen = "0.2.84"
postgresql-cst-parser = { path = "../postgresql-cst-parser" }
postgresql-lst-parser = { path = "../postgresql-lst-parser" }

# The `console_error_panic_hook` crate provides better debugging of panics by
# logging them with `console.error`. This is great for development, but requires
Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
mod utils;

use postgresql_cst_parser::parse;
use postgresql_lst_parser::parse;
use wasm_bindgen::prelude::*;

#[wasm_bindgen]
Expand Down
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
[package]
name = "postgresql-cst-parser"
name = "postgresql-lst-parser"
version = "0.1.0"
edition = "2021"
repository = "https://github.com/tanzaku/postgresql-cst-parser"
description = "PostgreSQL cst parser"
repository = "https://github.com/tanzaku/postgresql-lst-parser"
description = "PostgreSQL lst parser"
documentation = ""
license-file = "../../LICENSE"

Expand Down
File renamed without changes.
25 changes: 25 additions & 0 deletions crates/postgresql-lst-parser/src/lib.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
#![allow(non_camel_case_types)]

mod lexer;

mod parser;

mod lst;
pub mod syntax_kind;

pub use lst::NodeOrToken;
pub use lst::ParseError;
pub use lst::PostgreSQLSyntax;
pub use lst::ResolvedNode;
pub use lst::ResolvedToken;
pub use lst::SyntaxElement;
pub use lst::SyntaxElementRef;
pub use lst::SyntaxNode;
pub use lst::SyntaxToken;

pub fn parse(input: &str) -> Result<ResolvedNode, ParseError> {
lst::parse(input)
}

#[cfg(test)]
mod tests {}
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
use postgresql_cst_parser::parse;
use postgresql_lst_parser::parse;

#[test]
fn test_all() -> Result<(), std::io::Error> {
Expand Down
2 changes: 1 addition & 1 deletion docs/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

<body>
<script type="module">
import init, { parse_sql } from "./js/postgresql_cst_parser_wasm.js";
import init, { parse_sql } from "./js/postgresql_lst_parser_wasm.js";
init();

window.do_parse = function () {
Expand Down
Binary file modified docs/js/postgresql_cst_parser_wasm_bg.wasm
Binary file not shown.
39 changes: 39 additions & 0 deletions docs/js/postgresql_lst_parser_wasm.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
/* tslint:disable */
/* eslint-disable */
/**
* @param {string} sql
* @returns {string}
*/
export function parse_sql(sql: string): string;

export type InitInput = RequestInfo | URL | Response | BufferSource | WebAssembly.Module;

export interface InitOutput {
readonly memory: WebAssembly.Memory;
readonly parse_sql: (a: number, b: number, c: number) => void;
readonly __wbindgen_add_to_stack_pointer: (a: number) => number;
readonly __wbindgen_malloc: (a: number, b: number) => number;
readonly __wbindgen_realloc: (a: number, b: number, c: number, d: number) => number;
readonly __wbindgen_free: (a: number, b: number, c: number) => void;
}

export type SyncInitInput = BufferSource | WebAssembly.Module;
/**
* Instantiates the given `module`, which can either be bytes or
* a precompiled `WebAssembly.Module`.
*
* @param {SyncInitInput} module
*
* @returns {InitOutput}
*/
export function initSync(module: SyncInitInput): InitOutput;

/**
* If `module_or_path` is {RequestInfo} or {URL}, makes a request and
* for everything else, calls `WebAssembly.instantiate` directly.
*
* @param {InitInput | Promise<InitInput>} module_or_path
*
* @returns {Promise<InitOutput>}
*/
export default function __wbg_init (module_or_path?: InitInput | Promise<InitInput>): Promise<InitOutput>;
Loading

0 comments on commit 9ef3848

Please sign in to comment.