Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Table alters #7

Merged
merged 9 commits into from
Feb 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 43 additions & 2 deletions Changelog.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,51 @@
===
Dataloom **`2.0.0`**
Dataloom **`2.1.0`**
===

### Release Notes - `dataloom`

We have release the new `dataloom` Version `2.0.0` (`2024-02-12`)
We have release the new `dataloom` Version `2.1.0` (`2024-02-24`)

##### Features

- Connecting to databases using connection `uri` for all the supported dialects

```py
# postgress
pg_loom = Loom(
dialect="postgres",
connection_uri = "postgressql://root:root@localhost:5432/hi",
# ...
)
# mysql
mysql_loom = Loom(
dialect="mysql",
connection_uri = "mysql://root:root@localhost:3306/hi",
# ...
)

# sqlite
sqlite_loom = Loom(
dialect="sqlite",
connection_uri = "sqlite:///hi.db",
# ...
)
```

- updated documentation.
- enable table alterations as an option of `sync` and `connect_and_sync` function.
```py
conn, tables = pg_loom.connect_and_sync([Profile, User], alter=True)
```
> 🥇 **We recommend you to use `drop` or `force` if you are going to change or modify `foreign` and `primary` keys. This is because setting the option `alter` doe not have an effect on `primary` key columns.**

===

# Dataloom **`2.0.0`**

### Release Notes - `dataloom`

We have release the new `dataloom` Version `2.0.0` (`2024-02-21`)

##### Features

Expand Down
50 changes: 48 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@
- [Guidelines for Safe Usage](#guidelines-for-safe-usage)
- [Ordering](#ordering)
- [Filters](#filters)
- [Operators](#operators)
- [Operators](#operators)
- [Data Aggregation](#data-aggregation)
- [Aggregation Functions](#aggregation-functions)
- [Utilities](#utilities)
Expand Down Expand Up @@ -170,6 +170,18 @@ if __name__ == "__main__":
conn.close()
```

In dataloom you can use connection uris to establish a connection to the database in `postgres` as follows:

```py
pg_loom = Loom(
dialect="postgres",
connection_uri = "postgressql://root:root@localhost:5432/hi",
# ...
)
```

This will establish a connection with `postgres` with the database `hi`.

#### `MySQL`

To establish a connection with a `MySQL` database using `Loom`, you can use the following example:
Expand Down Expand Up @@ -198,6 +210,18 @@ if __name__ == "__main__":

```

In dataloom you can use connection uris to establish a connection to the database in `mysql` as follows:

```py
mysql_loom = Loom(
dialect="mysql",
connection_uri = "mysql://root:root@localhost:3306/hi",
# ...
)
```

This will establish a connection with `mysql` with the database `hi`.

#### `SQLite`

To establish a connection with an `SQLite` database using `Loom`, you can use the following example:
Expand All @@ -222,6 +246,18 @@ if __name__ == "__main__":
conn.close()
```

In dataloom you can use connection uris to establish a connection to the database in `sqlite` as follows:

```py
sqlite_loom = Loom(
dialect="sqlite",
connection_uri = "sqlite:///hi.db",
# ...
)
```

This will establish a connection with `sqlite` with the database `hi`.

### Dataloom Classes

The following are the list of classes that are available in `dataloom`.
Expand All @@ -242,13 +278,21 @@ loom = Loom(
logs_filename="logs.sql",
port=5432,
)

# OR with connection_uri
loom = Loom(
dialect="mysql",
connection_uri = "mysql://root:root@localhost:3306/hi",
# ...
)
```

The `Loom` class takes in the following options:
| Parameter | Description | Value Type | Default Value | Required |
| --------------- | --------------------------------------------------------------------------------- | --------------- | -------------- | -------- |
| `connection_uri` | The connection `uri` for the specified dialect. | `str` or `None` | `None` | `No` |
| `dialect` | Dialect for the database connection. Options are `mysql`, `postgres`, or `sqlite` | `str` or `None` | `None` | `Yes` |
| `database` | Name of the database for `mysql` and `postgres`, filename for `sqlite` | `str` or `None` | `None` | `Yes` |
| `database` | Name of the database for `mysql` and `postgres`, filename for `sqlite` | `str` or `None` | `None` | `No` |
| `password` | Password for the database user (only for `mysql` and `postgres`) | `str` or `None` | `None` | `No` |
| `user` | Database user (only for `mysql` and `postgres`) | `str` or `None` | `None` | `No` |
| `host` | Database host (only for `mysql` and `postgres`) | `str` or `None` | `localhost` | `No` |
Expand Down Expand Up @@ -620,6 +664,8 @@ The method returns a list of table names that have been created or that exist in
| `force` | Forcefully drop tables during syncing or not. | `bool` | `False` |
| `alter` | Alter tables instead of dropping them during syncing or not. | `bool` | `False` |

> 🥇 **We recommend you to use `drop` or `force` if you are going to change or modify `foreign` and `primary` keys. This is because setting the option `alter` doe not have an effect on `primary` key columns.**

> We've noticed two steps involved in starting to work with our `orm`. Initially, you need to create a connection and then synchronize the tables in another step.

#### 2. The `connect_and_sync` method.
Expand Down
20 changes: 19 additions & 1 deletion dataloom/conn/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
from os import PathLike
from typing_extensions import TypeAlias
from typing import Optional

from urllib.parse import urlparse, parse_qs
from dataloom.constants import instances


Expand Down Expand Up @@ -87,3 +87,21 @@ def get_connection_options(**kwargs):
raise UnsupportedDialectException(
"The dialect passed is not supported the supported dialects are: {'postgres', 'mysql', 'sqlite'}"
)

@staticmethod
def get_mysql_uri_connection_options(uri: str) -> dict:
components = urlparse(uri)
user = components.username
password = components.password
hostname = components.hostname
port = components.port
db = components.path.lstrip("/")

return {
"user": user,
"password": password,
"host": hostname,
"port": port,
"database": db,
**parse_qs(components.query),
}
8 changes: 8 additions & 0 deletions dataloom/exceptions/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,14 @@ class InvalidPropertyException(Exception):
pass


class InvalidDropOperationException(Exception):
pass


class InvalidConnectionURI(Exception):
pass


class TooManyPkException(Exception):
pass

Expand Down
22 changes: 22 additions & 0 deletions dataloom/keys.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
# Configuration file for unit testing.


push = True


Expand All @@ -6,18 +9,37 @@ class PgConfig:
password = "postgres"
database = "postgres"
user = "postgres"
host = "localhost"
port = 5432
connection_uri = f"postgresql://{user}:{password}@{host}:{port}/{database}"
else:
database = "postgres"
user = "postgres"
password = "root"
host = "localhost"
port = 5432
connection_uri = f"postgresql://{user}:{password}@{host}:{port}/{database}"


class MySQLConfig:
if push:
password = "testrootpass"
database = "testdb"
user = "root"
host = "0.0.0.0"
port = 3306
connection_uri = f"mysql://{user}:{password}@{host}:{port}/{database}"
else:
database = "hi"
user = "root"
password = "root"
host = "localhost"
port = 3306
connection_uri = f"mysql://{user}:{password}@{host}:{port}/{database}"


class SQLiteConfig:
if push:
connection_uri = "sqlite:///hi.db"
else:
connection_uri = "sqlite:///hi.db"
Loading