Skip to content

fast flattening and unflattening of nested Python data structures

License

Notifications You must be signed in to change notification settings

acg/python-flattery

Folders and files

NameName
Last commit message
Last commit date

Latest commit

b5c31a8 · Nov 17, 2014

History

18 Commits
Mar 8, 2011
Mar 14, 2011
Mar 7, 2011
Mar 8, 2011
Mar 14, 2011
Nov 17, 2014
Mar 13, 2011
Jun 23, 2011
Mar 8, 2011
Mar 14, 2011
Mar 13, 2011
Mar 8, 2011

Repository files navigation

flattery - fast flattening and unflattening of nested Python data

This library exposes a fast C implementation for flattening and unflattening hierarchical Python data structures. A unit test suite is included.

Examples

Synopsis

#!/usr/bin/env python
from flattery import flatten, unflatten

data = {
  "x.y.0": "zero",
  "x.y.1": "one",
  "x.z" : 42
}

print unflatten(data)
>>>
{
  'x' : {
    'y' : [ 'zero', 'one' ],
    'z': 42
  }
}
<<<

assert( data == flatten(unflatten(data)) )

Processing hierarchical records

#!/usr/bin/env python
import sys
from flattery import unflatten

cols = [ "time", "request.method", "request.uri", "response.status", "response.size" ]

for line in sys.stdin:
  fields = line.rstrip('\r\n').split(len(cols)-1)
  values = dict([(cols[i],fields[i]) for i in xrange(len(cols))])
  record = unflatten(values)
  # do something with the record...
  print record
>>>
{ 'time': '12/12/2012 12:00:00',
  'request': { 'method': 'GET', 'uri': '/stuff/' },
  'response': { 'status': '200', 'size': '40391' } }
...

Web form processing: grouping data

Or suppose you have a web form for collecting several distinct blobs of data at once, like a payment form:

#!/usr/bin/html
<form method="post" action="">
  <div>
    First name: <input type="text" name="contact.firstname" value="" />
    Last name:  <input type="text" name="contact.lastname" value="" />
    Email:      <input type="text" name="contact.email" value="" />
  </div>
  <div>
    Credit Card Type: <select name="payment.cctype"> <option> ... </option> </select>
    Number:           <input type="text" name="payment.ccnumber" value="" />
    CCV:              <input type="text" name="payment.ccv" value="" />
    Expiration Month: <select name="payment.ccmonth"> <option> ... </option> </select>
    Expiration Year:  <select name="payment.ccyear"> <option> ... </option> </select>
  </div>
  <div>
    <input type="submit" name="submit" />
  </div>
</form>

In the form processing code, you can expand the key value form data pairs into a nested object:

#!/usr/bin/env python
from flattery import unflatten

params = formdata()   # however you get a dictionary of form data...
data = unflatten(params)
print data
>>>
{ 'contact':
    { 'lastname':'Doe',
      'firstname': 'John',
      'email':'[email protected]' },
  'payment':
    { 'cctype': 'amex',
      'ccnumber': '4111111111111111',
      'ccv': '4321',
      'ccmonth': '12' ,
      'ccyear' : '2020' } }

(But be careful with multiply-valued form data.)

Web form processing: tabular data

Another web example, where a user is editing tabular data:

#!/usr/bin/html
<form method="post" action="">
  <div>
    <ul class="table">
      <ul class="row">
        <li><input type="checkbox" name="rows.0.delete" /></li>
        <li><input type="text" name="rows.0.name" value="" /></li>
        <li><input type="text" name="rows.0.email" value="" /></li>
      </ul>
      <ul class="row">
        <li><input type="checkbox" name="rows.1.delete" /></li>
        <li><input type="text" name="rows.1.name" value="" /></li>
        <li><input type="text" name="rows.1.email" value="" /></li>
      </ul>
      ...
      <ul class="row">
        <li><input type="checkbox" name="rows.99.delete" /></li>
        <li><input type="text" name="rows.99.name" value="" /></li>
        <li><input type="text" name="rows.99.email" value="" /></li>
      </ul>
    </ul>
  </div>
  <div>
    <input type="submit" name="submit" />
  </div>
</form>

(You should validate that the embedded indices are below some reasonable limit to avoid a memory DoS.)

In the form processing code:

#!/usr/bin/env python
from flattery import unflatten

params = formdata()   # however you get a dictionary of form data...
data = unflatten(params)
print data
>>>
{ 'rows':
  [
    'delete': '1',
    'name': 'John Doe',
    'email': '[email protected]',
  ],
  [
    'delete': '0',
    'name': 'Suzy Q',
    'email': '[email protected]',
  ],
  ...
  [
    'delete': '0',
    'name': 'Charlie Chaplin',
    'email': '[email protected]',
  ],
}

Installation

Ubuntu / Debian users:

#!/bin/sh
fakeroot ./debian/rules binary
dpkg -i ../python-flattery*.deb

If there's no "real" packaging for your system yet:

#!/bin/sh
./setup.py build_ext --inplace
./test.py
./setup.py build
./setup.py install

About

fast flattening and unflattening of nested Python data structures

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published