Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parser #2

Open
wants to merge 11 commits into
base: parser
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGES.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
0.1, 2013-02-13 -- Initial release.

File renamed without changes.
5 changes: 5 additions & 0 deletions MANIFEST.in
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
include *.txt
include *.md
include Makefile
graft flojay

56 changes: 56 additions & 0 deletions README
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# flojay

flojay provides incremental serialization of Python data structures as
JSON documents, and an event-based JSON parser to allow for incremtnal
deserialization. It can be used to create and process JSON documents
that are larger than working memory.

It is based on [yajl](http://lloyd.github.com/yajl/) version 2, a
copy of which is included in this distribution.

# example

import flojay
import sys

# Produce JSON from a generator, printing partial results
# as they become available

encoder = flojay.JSONEncoder()
json_generator = encoder.iterencode(xrange(100))
for hunk in json_generator:
print hunk


# Read an array of JSON numbers from stdin input,
# summing the values as they are read


class ExampleCallbacks(object):
def __init__(self):
self.sum = 0

def handle_start_array(self):
pass

def handle_end_array(self):
pass

def handle_number(self, value):
self.sum += value

callbacks = ExampleCallbacks()

parser = flojay.JSONEventParser(callbacks)

while 1:
row = sys.stdin.readline()
if len(row) == 0:
break
parser.parse(row)
print "The current total is: %d" % (callbacks.sum,)


# Thank also to

Lloyd Hilaiel -- creator of yajl -- https://github.com/lloyd
6 changes: 5 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,4 +49,8 @@ copy of which is included in this distribution.
break
parser.parse(row)
print "The current total is: %d" % (callbacks.sum,)



# Thank also to

Lloyd Hilaiel -- creator of yajl -- https://github.com/lloyd
52 changes: 52 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# flojay

flojay provides incremental serialization of Python data structures as
JSON documents, and an event-based JSON parser to allow for incremtnal
deserialization. It can be used to create and process JSON documents
that are larger than working memory.

It is based on [yajl](http://lloyd.github.com/yajl/) version 2, a
copy of which is included in this distribution.

# example

import flojay
import sys

# Produce JSON from a generator, printing partial results
# as they become available

encoder = flojay.JSONEncoder()
json_generator = encoder.iterencode(xrange(100))
for hunk in json_generator:
print hunk


# Read an array of JSON numbers from stdin input,
# summing the values as they are read


class ExampleCallbacks(object):
def __init__(self):
self.sum = 0

def handle_start_array(self):
pass

def handle_end_array(self):
pass

def handle_number(self, value):
self.sum += value

callbacks = ExampleCallbacks()

parser = flojay.JSONEventParser(callbacks)

while 1:
row = sys.stdin.readline()
if len(row) == 0:
break
parser.parse(row)
print "The current total is: %d" % (callbacks.sum,)

File renamed without changes.
File renamed without changes.
35 changes: 23 additions & 12 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,23 +5,34 @@
('yajl.c', 'yajl_gen.c', 'yajl_alloc.c', 'yajl_lex.c', 'yajl_tree.c', \
'yajl_encode.c', 'yajl_version.c', 'yajl_buf.c', 'yajl_parser.c')]

flojay = Extension('flojay',
flojay_extension = Extension('flojay',
define_macros=[
('MAJOR_VERSION', '0'),
('MINOR_VERSION', '1')],
extra_compile_args=['--std=c99'],
include_dirs=['flojay/lloyd-yajl/src'],
sources=yajl_sources + ['flojay/flojay.c'])

setup(
name='flojay',
version='0.1',
description='Streaming and event-based JSON parser based on yajl',
author='Robert Church',
author_email='[email protected]',
url = "http://github/myemma/flojay/",
ext_modules=[flojay_extension],
packages=['flojay'],
install_requires=['nose==1.1.2'],
keywords = ["json", "stream", "ajax", "webapp", "website", "data", "messaging"],
classifiers = [
"Programming Language :: Python",
"Development Status :: 2 - Pre-Alpha",
"Environment :: Other Environment",
"Intended Audience :: Developers",
"License :: OSI Approved :: BSD License",
"Operating System :: OS Independent",
"Topic :: Software Development :: Libraries :: Python Modules",
"Topic :: Internet",
],
)

setup(name='flojay',
version='0.1',
description='Streaming or event-based JSON parser based on yajl',
author='Robert Church',
author_email='[email protected]',
url='',
ext_modules=[flojay],
packages=['flojay'],
install_requires=[
'nose==1.1.2',
])