-
Notifications
You must be signed in to change notification settings - Fork 4
/
Copy pathdatasetd_api.5.html
192 lines (190 loc) · 13.1 KB
/
datasetd_api.5.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
<!DOCTYPE html>
<html>
<head>
<title>Dataset Project</title>
<link href='https://fonts.googleapis.com/css?family=Open+Sans' rel='stylesheet' type='text/css'>
<link rel="stylesheet" href="https://caltechlibrary.github.io/css/site.css">
</head>
<body>
<header>
<a href="http://library.caltech.edu" title="link to Caltech Library Homepage"><img src="https://caltechlibrary.github.io/assets/liblogo.gif" alt="Caltech Library logo"></a>
</header>
<nav>
<ul>
<li><a href="/">Home</a></li>
<li><a href="index.html">README</a></li>
<li><a href="LICENSE">LICENSE</a></li>
<li><a href="install.html">INSTALL</a></li>
<li><a href="user_manual.html">User Manual</a></li>
<li><a href="about.html">About</a></li>
<li><a href="search.html">Search</a></li>
<li><a href="https://github.com/caltechlibrary/dataset">GitHub</a></li>
</ul>
</nav>
<section>
<h1 id="datasetd-rest-api">datasetd REST API</h1>
<p>datasetd provides a RESTful JSON API for working with a dataset
collection. This document describes the path expressions and to interact
with the API. Note some of the methods and paths require permissions as
set in the datasetd YAML or JSON <a href="datasetd_yaml.5.html">settings
file</a>.</p>
<h2 id="basic-path-expressions">basic path expressions</h2>
<p>There are three basic forms of the URL paths supported by the
API.</p>
<ul>
<li><code>/api/<COLLECTION_NAME>/keys</code>, get a list of all
keys in the the collection</li>
<li><code>/api/<COLLECTION_NAME>/object/<OPTIONS></code>,
interact with an object in the collection (e.g. create, read, update,
delete)</li>
<li><code>/api/<COLLECTION_NAME>/query/<FIELDS></code>,
query the collection and receive a list of objects in response</li>
</ul>
<p>The “<code><COLLECTION_NAME></code>” would be the name of the
dataset collection, e.g. “mydata.ds”.</p>
<p>The “<code><OPTIONS></code>” holds any additional parameters
related to the verb. Options are separated by the path delimiter
(i.e. “/”). The options are optional. They do not require a trailing
slash.</p>
<p>The “<code><FIELDS></code>” holds the set of fields being
passed into the query. These are delimited with the path separator like
with options (i.e. “/”). Fields are optional and they do not require a
trailing slash.</p>
<h2 id="http-methods">HTTP Methods</h2>
<p>The datasetd REST API follows the rest practices. Good examples are
POST creates, GET reads, PUT updates, and DELETE removes. It is
important to remember that the HTTP method and path expression need to
match form the actions you’d take using the command line version of
dataset. For example to create a new object you’d use the object path
without any options and a POST expression. You can do a read of an
object using the GET method along withe object style path.</p>
<h2 id="content-type-and-the-api">Content Type and the API</h2>
<p>The REST API works with JSON data. The service does not support
multipart urlencoded content. You MUST use the content type of
<code>application/json</code> when performing a POST, or PUT. This means
if you are building a user interface for a collections datasetd service
you need to appropriately use JavaScript to send content into the API
and set the content type to <code>application/json</code>.</p>
<h2 id="examples">Examples</h2>
<p>Here’s an example of a list, in YAML, of people in a collection
called “people.ds”. There are some fields for the name, sorted name,
display name and orcid. The pid is the “key” used to store the objects
in our collection.</p>
<div class="sourceCode" id="cb1"><pre
class="sourceCode yaml"><code class="sourceCode yaml"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="fu">people</span><span class="kw">:</span></span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="kw">-</span><span class="at"> </span><span class="fu">pid</span><span class="kw">:</span><span class="at"> doe-jane</span></span>
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="fu">family</span><span class="kw">:</span><span class="at"> Doe</span></span>
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="fu">lived</span><span class="kw">:</span><span class="at"> Jane</span></span>
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="fu">orcid</span><span class="kw">:</span><span class="at"> 9999-9999-9999-9999</span></span></code></pre></div>
<p>In JSON this would look like</p>
<div class="sourceCode" id="cb2"><pre
class="sourceCode json"><code class="sourceCode json"><span id="cb2-1"><a href="#cb2-1" aria-hidden="true" tabindex="-1"></a><span class="fu">{</span></span>
<span id="cb2-2"><a href="#cb2-2" aria-hidden="true" tabindex="-1"></a> <span class="dt">"people"</span><span class="fu">:</span> <span class="ot">[</span></span>
<span id="cb2-3"><a href="#cb2-3" aria-hidden="true" tabindex="-1"></a> <span class="fu">{</span></span>
<span id="cb2-4"><a href="#cb2-4" aria-hidden="true" tabindex="-1"></a> <span class="dt">"pid"</span><span class="fu">:</span> <span class="st">"doe-jane"</span><span class="fu">,</span></span>
<span id="cb2-5"><a href="#cb2-5" aria-hidden="true" tabindex="-1"></a> <span class="dt">"family"</span><span class="fu">:</span> <span class="st">"Doe"</span><span class="fu">,</span></span>
<span id="cb2-6"><a href="#cb2-6" aria-hidden="true" tabindex="-1"></a> <span class="dt">"lived"</span><span class="fu">:</span> <span class="st">"Jane"</span><span class="fu">,</span></span>
<span id="cb2-7"><a href="#cb2-7" aria-hidden="true" tabindex="-1"></a> <span class="dt">"orcid"</span><span class="fu">:</span> <span class="st">"9999-9999-9999-9999"</span></span>
<span id="cb2-8"><a href="#cb2-8" aria-hidden="true" tabindex="-1"></a> <span class="fu">}</span></span>
<span id="cb2-9"><a href="#cb2-9" aria-hidden="true" tabindex="-1"></a> <span class="ot">]</span></span>
<span id="cb2-10"><a href="#cb2-10" aria-hidden="true" tabindex="-1"></a><span class="fu">}</span></span></code></pre></div>
<h3 id="create">create</h3>
<p>The create action is formed with the object URL path, the POST http
method and the content type of “application/json”. It POST data is
expressed as a JSON object.</p>
<p>The object path includes the dataset key you’ll assign in the
collection. The key must be unique and not currently exist in the
collection.</p>
<p>If we’re adding an object with the key of “doe-jane” to our
collection called “people.ds” then the object URL path would be
<code>/api/people.ds/object/doe-jane</code>. NOTE: the object key is
included as a single parameter after “object” path element.</p>
<p>Adding an object to our collection using curl looks like the
following.</p>
<pre class="shell"><code>curl -X POST \
-H 'Content-Type: application/json' \
-H 'Accept: application/json' \
-d '{"pid": "doe-jane", "family": "Doe", "lived": "Jane", "orcid": "9999-9999-9999-9999" }' \
http://localhost:8485/api/people.ds/object/doe-jane </code></pre>
<h3 id="read">read</h3>
<p>The read action is formed with the object URL path, the GET http
method and the content type of “application/json”. There is no data
aside from the URL to request the object. Here’s what it would look like
using curl to access the API.</p>
<pre class="shell"><code>curl http://localhost:8485/api/people.ds/object/doe-jane </code></pre>
<h3 id="update">update</h3>
<p>Like create update is formed from the object URL path, content type
of “application/json” the data is expressed as a JSON object. Onlike
create we use the PUT http method.</p>
<p>Here’s how you would use curl to get the JSON expression of the
object called “doe-jane” in your collection.</p>
<pre class="shell"><code>curl -X PUT \
-H 'Content-Type: application/json' \
-H 'Accept: application/json' \
-d '{"pid": "doe-jane", "family": "Doe", "lived": "Jane", "orcid": "9999-9999-9999-9999" }' \
http://localhost:8485/api/people.ds/object/doe-jane </code></pre>
<p>This will overwrite the existing “doe-jane”. NOTE the record must
exist or you will get an error.</p>
<h3 id="delete">delete</h3>
<p>If you want to delete the “doe-jane” record in “people.ds” you
perform an http DELETE method and form the url like a read.</p>
<pre class="shell"><code>curl -X DELETE http://localhost:8485/api/people.ds/object/doe-jane </code></pre>
<h2 id="query">query</h2>
<p>The query path lets you run a predefined query from your settings
YAML file. The http method used is a POST. This is becaue we need to
send data inorder to receive a response. The resulting data is expressed
as a JSON array of object. Like with create, read, update and delete you
use the content type of “application/json”.</p>
<p>In the settings file the queries are named. The query names are
unique. One or many queries may be defined. The SQL expression
associated with the name run as a prepared statement and parameters are
mapped into based on the URL path provided. This allows you use many
fields in forming your query.</p>
<p>Let’s say we have a query called “full_name”. It is defined to run
the following SQL.</p>
<div class="sourceCode" id="cb7"><pre
class="sourceCode sql"><code class="sourceCode sql"><span id="cb7-1"><a href="#cb7-1" aria-hidden="true" tabindex="-1"></a><span class="kw">select</span> src</span>
<span id="cb7-2"><a href="#cb7-2" aria-hidden="true" tabindex="-1"></a><span class="kw">from</span> people</span>
<span id="cb7-3"><a href="#cb7-3" aria-hidden="true" tabindex="-1"></a><span class="kw">where</span> src<span class="op">->></span><span class="st">'family'</span> <span class="kw">like</span> ?</span>
<span id="cb7-4"><a href="#cb7-4" aria-hidden="true" tabindex="-1"></a> <span class="kw">and</span> src<span class="op">->></span><span class="st">'lived'</span> <span class="kw">like</span> ?</span>
<span id="cb7-5"><a href="#cb7-5" aria-hidden="true" tabindex="-1"></a><span class="kw">order</span> <span class="kw">by</span> family, lived</span></code></pre></div>
<p>NOTE: The SQL is has to retain the constraint of a single object per
row, normally this will be “src” for dataset collections.</p>
<p>When you form a query path we need to indicate that the parameter for
family and lived names need to get mapped to their respect positional
references in the SQL. This is done as following url path.</p>
<pre><code>/api/people.ds/query/full_name/family/lived</code></pre>
<p>The web form could look like this.</p>
<pre><code><form id="query_name">
<label for="family">Family</label> <input id="family" name="family" ><br/>
<label for="lived">Lived</label> <input id="lived" name="lived" ><br/>
<button type="submit">Search</button>
</form></code></pre>
<p>REMEMBER: the JSON API only supports the content type of
“application/json” so you can use the browser’s action and method in the
form.</p>
<p>You would include JavaScript in the your HTML to pull the values out
of the form and create a JSON object. If I searched for someone who had
the family name “Doe” and he lived name of “Jane” the object submitted
to query might look like the following.</p>
<div class="sourceCode" id="cb10"><pre
class="sourceCode json"><code class="sourceCode json"><span id="cb10-1"><a href="#cb10-1" aria-hidden="true" tabindex="-1"></a><span class="fu">{</span></span>
<span id="cb10-2"><a href="#cb10-2" aria-hidden="true" tabindex="-1"></a> <span class="dt">"family"</span><span class="fu">:</span> <span class="st">"Doe"</span></span>
<span id="cb10-3"><a href="#cb10-3" aria-hidden="true" tabindex="-1"></a> <span class="st">"lived"</span><span class="er">:</span> <span class="st">"Jane"</span></span>
<span id="cb10-4"><a href="#cb10-4" aria-hidden="true" tabindex="-1"></a><span class="fu">}</span></span></code></pre></div>
<p>The curl expression would look like the following simulating the form
submission would look like the following.</p>
<pre class="shell"><code>curl -X POST \
-H 'Content-Type: application/json' \
-H 'Accept: application/json' \
-d '{"family": "Doe", "lived": "Jane" }' \
http://localhost:8485/api/people.ds/query/full_name/family/lived</code></pre>
</section>
<footer>
<span>© 2022 <a href="https://www.library.caltech.edu/copyright">Caltech Library</a></span>
<address>1200 E California Blvd, Mail Code 1-32, Pasadena, CA 91125-3200</address>
<span><a href="mailto:[email protected]">Email Us</a></span>
<span>Phone: <a href="tel:+1-626-395-3405">(626)395-3405</a></span>
</footer>
</body>
</html>