Skip to content

Commit

Permalink
[FIX] Variable names ending with and get split in WHERE statements (#…
Browse files Browse the repository at this point in the history
  • Loading branch information
PabloRMira authored Apr 30, 2021
1 parent fdb177e commit a5f5c5f
Show file tree
Hide file tree
Showing 8 changed files with 282 additions and 11 deletions.
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ for a custom maximum line length truncation of e.g. 50

To exemplify the formatting let's say you have a SQL query like this

```python
```
example_sql = """
create or replace table mytable as -- mytable example
seLecT a.asdf, b.qwer, -- some comment here
Expand All @@ -107,7 +107,7 @@ groUp by a.asdf
Then you can use this package to format it so that it is better readable
```python
```
from sql_formatter.core import format_sql
print(format_sql(example_sql))
```
Expand All @@ -131,7 +131,7 @@ print(format_sql(example_sql))
It can even deal with subqueries and it will correct my favourite simple careless mistake (comma at the end of SELECT statement before of FROM) for you on the flow :-)
```python
```
print(format_sql("""
select asdf, cast(qwer as numeric), -- some comment
qwer1
Expand Down Expand Up @@ -161,7 +161,7 @@ where qwer1 >= 0
The formatter is also robust against nested subqueries
```python
```
print(format_sql("""
select field1, field2 from (select field1,
field2 from (select field1, field2,
Expand All @@ -183,7 +183,7 @@ field3 from table1 where a=1 and b>=100))
If you do not want to get some query formatted in your SQL file then you can use the marker `/*skip-formatter*/` in your query to disable formatting for just the corresponding query
```python
```
from sql_formatter.format_file import format_sql_commands
print(format_sql_commands(
"""
Expand Down
3 changes: 3 additions & 0 deletions docs/_data/sidebars/home_sidebar.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,9 @@ entries:
- output: web,pdf
title: validation
url: validation.html
- output: web,pdf
title: Additional tests
url: additional_tests.html
output: web
title: sql_formatter
output: web
Expand Down
127 changes: 127 additions & 0 deletions docs/additional_tests.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
---

title: Additional tests


keywords: fastai
sidebar: home_sidebar



nb_path: "nbs/99_additional_tests.ipynb"
---
<!--
#################################################
### THIS FILE WAS AUTOGENERATED! DO NOT EDIT! ###
#################################################
# file to edit: nbs/99_additional_tests.ipynb
# command to build the docs after a change: nbdev_build_docs
-->

<div class="container" id="notebook-container">

{% raw %}

<div class="cell border-box-sizing code_cell rendered">

</div>
{% endraw %}

<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="core">core<a class="anchor-link" href="#core"> </a></h2>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h3 id="format_where">format_where<a class="anchor-link" href="#format_where"> </a></h3>
</div>
</div>
</div>
{% raw %}

<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">assert_and_print</span><span class="p">(</span>
<span class="n">format_where</span><span class="p">(</span>
<span class="s2">&quot;WHERE brand = &#39;my_brand&#39;&quot;</span>
<span class="p">),</span> <span class="s2">&quot;WHERE brand = &#39;my_brand&#39;&quot;</span>
<span class="p">)</span>
</pre></div>

</div>
</div>
</div>

<div class="output_wrapper">
<div class="output">

<div class="output_area">

<div class="output_subarea output_stream output_stdout output_text">
<pre>WHERE brand = &#39;my_brand&#39;
</pre>
</div>
</div>

</div>
</div>

</div>
{% endraw %}

<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h3 id="format_sql">format_sql<a class="anchor-link" href="#format_sql"> </a></h3>
</div>
</div>
</div>
{% raw %}

<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">assert_and_print</span><span class="p">(</span>
<span class="n">format_sql</span><span class="p">(</span><span class="s2">&quot;SELECT brand FROM table WHERE brand = &#39;my_brand&#39;&quot;</span><span class="p">),</span>
<span class="sd">&quot;&quot;&quot;</span>
<span class="sd">SELECT brand</span>
<span class="sd">FROM table</span>
<span class="sd">WHERE brand = &#39;my_brand&#39;</span>
<span class="sd">&quot;&quot;&quot;</span><span class="o">.</span><span class="n">strip</span><span class="p">()</span>
<span class="p">)</span>
</pre></div>

</div>
</div>
</div>

<div class="output_wrapper">
<div class="output">

<div class="output_area">

<div class="output_subarea output_stream output_stdout output_text">
<pre>SELECT brand
FROM table
WHERE brand = &#39;my_brand&#39;
</pre>
</div>
</div>

</div>
</div>

</div>
{% endraw %}

</div>


7 changes: 7 additions & 0 deletions docs/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,13 @@ <h3 id="Usage-with-pre-commit">Usage with <code>pre-commit</code><a class="ancho
<div class="text_cell_render border-box-sizing rendered_html">
<p><a href="https://pre-commit.com">pre-commit</a> is a nice development tool to automatize the binding of pre-commit hooks. After installation and configuration <code>pre-commit</code> will run your hooks before you commit any change.</p>
<p>To add <code>sql-formatter</code> as a hook to your <code>pre-commit</code> configuration to format your SQL files before commit, just add the following lines to your <code>.pre-commit-config.yaml</code>:</p>
<div class="highlight"><pre><span></span><span class="nt">repos</span><span class="p">:</span>
<span class="p p-Indicator">-</span> <span class="nt">repo</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">https://github.com/PabloRMira/sql_formatter</span>
<span class="nt">rev</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">master</span>
<span class="nt">hooks</span><span class="p">:</span>
<span class="p p-Indicator">-</span> <span class="nt">id</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">sql_formatter</span>
</pre></div>
<p>If you want to install <code>sql-formatter</code> locally and use that instead of using <code>pre-commit</code>'s default environment, set <code>repo: local</code> in your <code>.pre-commit-config.yaml</code> file:</p>
<div class="highlight"><pre><span></span><span class="nt">repos</span><span class="p">:</span>
<span class="p p-Indicator">-</span> <span class="nt">repo</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">local</span>
<span class="nt">hooks</span><span class="p">:</span>
Expand Down
3 changes: 2 additions & 1 deletion docs/sidebar.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
"core": "core.html",
"format_file": "format_file.html",
"utils": "utils.html",
"validation": "validation.html"
"validation": "validation.html",
"Additional tests": "additional_tests.html"
}
}
16 changes: 13 additions & 3 deletions nbs/00_core.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,16 @@
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"The autoreload extension is already loaded. To reload it, use:\n",
" %reload_ext autoreload\n"
]
}
],
"source": [
"#hide\n",
"%load_ext autoreload\n",
Expand Down Expand Up @@ -3019,8 +3028,8 @@
" # split by comment / non comment, quote / non-quote\n",
" split_s = split_comment_quote(s)\n",
" # define regex before loop\n",
" indent_and = re.compile(r\"\\s*(and)\\b\", flags=re.I)\n",
" indent_or = re.compile(r\"\\s*(or)\\b\", flags=re.I)\n",
" indent_and = re.compile(r\"\\s*\\b(and)\\b\", flags=re.I)\n",
" indent_or = re.compile(r\"\\s*\\b(or)\\b\", flags=re.I)\n",
" for d in split_s:\n",
" if not d[\"comment\"] and not d[\"quote\"]:\n",
" s_aux = d[\"string\"]\n",
Expand Down Expand Up @@ -4060,6 +4069,7 @@
"Converted 01_format_file.ipynb.\n",
"Converted 02_utils.ipynb.\n",
"Converted 03_validation.ipynb.\n",
"Converted 99_additional_tests.ipynb.\n",
"Converted index.ipynb.\n"
]
}
Expand Down
123 changes: 123 additions & 0 deletions nbs/99_additional_tests.ipynb
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#hide\n",
"%load_ext autoreload\n",
"%autoreload 2\n",
"%config Completer.use_jedi = False # workaround for buggy jedi"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Additional tests"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#hide\n",
"from nbdev.showdoc import *\n",
"from sql_formatter.utils import *\n",
"from sql_formatter.core import *"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## core"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### format_where"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"WHERE brand = 'my_brand'\n"
]
}
],
"source": [
"assert_and_print(\n",
" format_where(\n",
" \"WHERE brand = 'my_brand'\"\n",
" ), \"WHERE brand = 'my_brand'\"\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### format_sql"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"SELECT brand\n",
"FROM table\n",
"WHERE brand = 'my_brand'\n"
]
}
],
"source": [
"assert_and_print(\n",
" format_sql(\"SELECT brand FROM table WHERE brand = 'my_brand'\"),\n",
"\"\"\"\n",
"SELECT brand\n",
"FROM table\n",
"WHERE brand = 'my_brand'\n",
"\"\"\".strip()\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#hide\n",
"from nbdev.export import notebook2script\n",
"notebook2script()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python [conda env:sql-formatter-dev] *",
"language": "python",
"name": "conda-env-sql-formatter-dev-py"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
4 changes: 2 additions & 2 deletions sql_formatter/core.py
Original file line number Diff line number Diff line change
Expand Up @@ -366,8 +366,8 @@ def format_where(s, **kwargs):
# split by comment / non comment, quote / non-quote
split_s = split_comment_quote(s)
# define regex before loop
indent_and = re.compile(r"\s*(and)\b", flags=re.I)
indent_or = re.compile(r"\s*(or)\b", flags=re.I)
indent_and = re.compile(r"\s*\b(and)\b", flags=re.I)
indent_or = re.compile(r"\s*\b(or)\b", flags=re.I)
for d in split_s:
if not d["comment"] and not d["quote"]:
s_aux = d["string"]
Expand Down

0 comments on commit a5f5c5f

Please sign in to comment.