Skip to content

Commit

Permalink
Version 2.0 (#18)
Browse files Browse the repository at this point in the history
* Sum Function (#13)
* Changed columns to fields in increment function
* PostgreSQL support. (#17)
* Missing default settings variable added for connection with json file
* Version 2.0 Changelog and version update for files.
  • Loading branch information
mastizada authored Apr 2, 2017
1 parent 25c5aaf commit 961a1e3
Show file tree
Hide file tree
Showing 15 changed files with 215 additions and 74 deletions.
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
__pycache__/
venv/
*.pyc
.idea/*
build/
Expand All @@ -7,3 +8,4 @@ dist/
docs/_build/
NOTES.md
test_credentials.json
test_pg_credentials.json
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,3 +50,9 @@
* Added example for execution of custom queries
* Added Link for available connection parameters
* Flake8 improvements (PEP8)

** v 2.0 **:

* Added SUM function
* Added support for Postgresql
* Minor fixes
29 changes: 23 additions & 6 deletions README.rst
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
dbConnect: MySQL for Humans
dbConnect: Database for Humans
===========================
.. image:: https://readthedocs.org/projects/dbconnect/badge/?version=latest
:target: http://dbconnect.readthedocs.org/?badge=latest
Expand All @@ -9,11 +9,22 @@ dbConnect: MySQL for Humans
:target: https://landscape.io/github/mastizada/dbConnect/master
:alt: Code Health


WHY?
====

dbConnect was made as a little module to be used in small projects
that need to do some interactions with MySQL or PostgreSQL databases.

It's just a big time saver for developers specially for making data analyzing and data scraping
and it helps to keep your code clean and readable by using python-like structure.


Installation
=============
requirements:
^^^^^^^^^^^^^
dbConnect uses mysql.connector module, install it using:
dbConnect uses mysql.connector module for mysql, install it using:

.. code-block:: bash
Expand All @@ -22,6 +33,12 @@ dbConnect uses mysql.connector module, install it using:
Or using offical site: `https://dev.mysql.com/downloads/connector/python/`

For PostgreSQL install psycopg2 module:

.. code-block:: bash
$ pip install psycopg2
using pip:
^^^^^^^^^^

Expand All @@ -36,7 +53,7 @@ from source code:
$ git clone [email protected]:mastizada/dbConnect.git
$ cd dbConnect
$ sudo pip install -r requirements.txt --allow-external mysql-connector-python
$ # install required module for database
$ sudo python setup.py install
Usage
Expand All @@ -53,7 +70,7 @@ Documentation
=============

- Docs: http://dbconnect.readthedocs.org/
- Another Docs: https://pythonhosted.org/dbConnect/
- Alternate Docs: https://pythonhosted.org/dbConnect/
- Check generated documentation using:

.. code-block:: bash
Expand All @@ -64,9 +81,9 @@ Documentation

.. code-block:: bash
$ pydoc3 -p 1994
$ pydoc3 -p 8000
and open localhost:1994/ in browser
and open http://localhost:8000/ in browser

Enjoy
=====
3 changes: 2 additions & 1 deletion credentials.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,6 @@
"user": "",
"password": "",
"database": "",
"port": 3306
"port": 3306,
"engine": "mysql"
}
8 changes: 8 additions & 0 deletions credentials.postgres.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"host": "",
"user": "",
"password": "",
"dbname": "",
"port": 5432,
"engine": "postgres"
}
8 changes: 4 additions & 4 deletions dbConnect/__init__.py
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
"""
MySQL for Humans
Database for Humans
"""
from .dbConnect import DBConnect

__description__ = 'MySQL for Humans'
__description__ = 'Database for Humans'
__author__ = "Emin Mastizada <[email protected]>"
__version__ = '1.6.0'
__version__ = '2.0'
__license__ = "MPL 2.0"
__help__ = """MySQL for Humans"""
__help__ = """Database for Humans"""

__all__ = ['DBConnect', ]
142 changes: 108 additions & 34 deletions dbConnect/dbConnect.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,52 +2,69 @@
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
import json
try:
import mysql.connector # MySQL Connector
from mysql.connector import errorcode
except:
raise ValueError(
'Please, install mysql-connector module before using plugin.'
)


class DBConnect:
"""
Light database connection object
"""
"""Light database connection object."""
settings = {}
def _check_settings(self):
"""
Check configuration file
:return: True if all settings are correct
"""
keys = ['host', 'user', 'password', 'database']
keys = ['host', 'user', 'password']
if not all(key in self.settings.keys() for key in keys):
raise ValueError(
'Please check credentials file for correct keys: host, user, '
'password, database'
)
if self.engine == "mysql" and 'database' not in self.settings.keys():
raise ValueError(
'database parameter is missing in credentials'
)
# @NOTE PostgreSQL uses dbname and is automatically set to username

def connect(self):
"""
Creates connection to database, sets connection and cursor
Connection to database can be loosed,
if that happens you can use this function to reconnect to database
"""
try:
self.connection = mysql.connector.connect(**self.settings)
except mysql.connector.Error as err:
if err.errno == errorcode.ER_ACCESS_DENIED_ERROR:
raise ValueError("Wrong credentials, ACCESS DENIED")
elif err.errno == errorcode.ER_BAD_DB_ERROR:
if self.engine == "mysql":
try:
import mysql.connector # MySQL Connector
from mysql.connector import errorcode
self.connection = mysql.connector.connect(**self.settings)
except ImportError:
raise ValueError(
"Database %s does not exists" % (self.settings['database'])
'Please, install mysql-connector module before using plugin.'
)
else:
raise ValueError(err)
except mysql.connector.Error as err:
if err.errno == errorcode.ER_ACCESS_DENIED_ERROR:
raise ValueError("Wrong credentials, ACCESS DENIED")
elif err.errno == errorcode.ER_BAD_DB_ERROR:
raise ValueError(
"Database %s does not exists" % (self.settings['database'])
)
else:
raise ValueError(err)
elif self.engine == "postgres":
try:
import psycopg2
except ImportError:
raise ValueError(
'Please, install psycopg2 module before using plugin.'
)
self.connection = psycopg2.connect(**self.settings)
else:
raise NotImplementedError(
"Database engine %s not implemented!" % self.engine
)

self.cursor = self.connection.cursor()

def __init__(self, credentials_file=None, charset='utf8',
port=3306, **kwargs):
port=3306, engine="mysql", **kwargs):
"""
Initialise object with credentials file provided
You can choose between providing file or connection details
Expand All @@ -63,6 +80,10 @@ def __init__(self, credentials_file=None, charset='utf8',
self.settings['charset'] = charset
# Merge with kwargs
self.settings.update(**kwargs)
self.engine = self.settings.pop('engine', engine)
# @NOTE Charset parameter not supported in PostgreSQL
if self.engine == 'postgres':
self.settings.pop('charset', None)
self._check_settings()
self.connection = None
self.cursor = None
Expand Down Expand Up @@ -197,14 +218,30 @@ def insert(self, data, table, commit=True, update=None):
query = query_insert + query_value
if update and bool(update):
# bool(dict) checks if dict is not empty
query += ' ON DUPLICATE KEY UPDATE '
for key in update:
query += key + ' = '
if isinstance(update[key], int):
query += update[key] + ', '
else:
query += '"' + update[key] + '", '
query = query.rstrip(', ')
if self.engine == "mysql":
query += ' ON DUPLICATE KEY UPDATE '
for key in update:
query += key + ' = '
if isinstance(update[key], int):
query += update[key] + ', '
else:
query += '"' + update[key] + '", '
query = query.rstrip(', ')
elif self.engine == "postgres":
query += ' ON CONFLICT ON CONSTRAINT '
query += table + '_pkey'
query += ' DO UPDATE SET '
for key in update:
query = key + ' = '
if isinstance(update[key], int):
query += update[key] + ', '
else:
query += '"' + update[key] + '", '
query = query.rstrip(', ')
else:
raise NotImplementedError(
"Update on insert not implemented for choosen engine"
)
# Format, execute and send to database:
self.cursor.execute(query, data)
if commit:
Expand Down Expand Up @@ -283,22 +320,24 @@ def delete(self, table, filters=None, case='AND', commit=True):
if commit:
self.commit()

def increment(self, table, columns, steps=1, filters=None,
def increment(self, table, fields, steps=1, filters=None,
case="AND", commit=True):
"""
Increment column in table
:param table: str table name
:param columns: list column names to increment
:param fields: list column names to increment
:param steps: int steps to increment, default is 1
:param filters: dict filters for rows to use
:param case: Search case, Should be 'AND' or 'OR'
:param commit: Commit at the end or add to pool
:note: If you use safe update mode, filters should be provided
"""
if not columns:
raise ValueError("You must provide which columns to update")
if not fields:
raise ValueError(
"You must provide which columns (fields) to update"
)
query = "UPDATE %s SET " % str(table)
for column in columns:
for column in fields:
query += "{column} = {column} + {steps}, ".format(
column=column, steps=steps)
query = query.rstrip(', ')
Expand All @@ -317,6 +356,41 @@ def increment(self, table, columns, steps=1, filters=None,
self.commit()
return {'status': True, 'message': "Columns incremented"}

def value_sum(self, table, fields, filters=None, case='AND'):
"""
Get total sum of a numeric column(s)
:param table: name of the table
:type table: str
:param fields: fields to get sum of
:type fields: list
:param filters: filters to get custom results (where)
:type filters: dict
:param case: [AND, OR] for filter type
:type case: str
:return: dict with column name and value as Decimal
"""
query = 'SELECT '
for field in fields:
query += 'SUM(' + field + '), '
query = query.rstrip(', ') + ' FROM ' + str(table)
data = None
if filters:
data = {}
query += ' WHERE '
update_query, where_data = self._where_builder(filters, case)
query += update_query
for key in where_data:
data['where_' + key] = where_data[key]
if data:
self.cursor.execute(query, data)
else:
self.cursor.execute(query)
row = self.cursor.fetchone()
result = {}
for i in range(len(row)):
result[fields[i]] = row[i]
return result

def commit(self):
"""
Commit collected data for making changes to database
Expand Down
2 changes: 1 addition & 1 deletion docs/_templates/sidebarlogo.html
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
</p>

<p>
<b>dbConnect</b> is Light MySQL Database Module. If ORM is too big for your project and MySQL module looks ugly and difficult then dbConnect is for you.
<b>dbConnect</b> is Light MySQL and PostgreSQL Database Module. If ORM is too big for your project and MySQL module looks ugly and difficult then dbConnect is for you.
</p>

<p>
Expand Down
2 changes: 1 addition & 1 deletion docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
import sys
import os
import shlex
__version__ = '1.6.0'
__version__ = '2.0'

sys.path.insert(0, os.path.abspath('../dbConnect'))

Expand Down
8 changes: 5 additions & 3 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,16 +3,16 @@
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
dbConnect: MySQL for Humans
dbConnect: Database for Humans
======================================

Release v\ |version|. (:ref:`Installation <install>`)

.. image:: https://readthedocs.org/projects/dbconnect/badge/?version=latest

dbConnect is an :ref:`MPLv2 Licensed<mpl2>` Module for **little projects**
using *mysql* database. It generates mysql queries automatically,
you just send data in pythonic style and it does the rest.
using *mysql* or *postgresql* databases. It generates mysql and postgresql
queries automatically, you just send data in pythonic style and it does the rest.

>>> from dbConnect import DBConnect
>>> database = DBConnect('credentials.json')
Expand All @@ -31,6 +31,8 @@ Feature Support
- **insert** to table
- **update** row
- **delete** row
- **increment** column in table
- **sum** of a numeric column(s)
- **custom sql query**


Expand Down
Loading

0 comments on commit 961a1e3

Please sign in to comment.