Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mongo doc etc #163

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 16 additions & 16 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -113,23 +113,24 @@ The <config_file> contains:

To use the Memory back-end plug, include the following in the <config-file>:

.. code-block:: json
.. code-block:: text

{
"backend": {
"module_class": "MemoryBackend",
"filename": "<path to json file with initial data>"
"filename": <path to json file with initial data>
}
}

To use the Mongo DB back-end plug, include the following in the <config-file>:

.. code-block:: json
.. code-block:: text

{
"backend": {
"module_class": "MongoBackend",
"uri": "<Mongo DB server url> # e.g., 'mongodb://localhost:27017/'"
"uri": <Mongo DB server url> # e.g., 'mongodb://localhost:27017/'
"filename": <path to json file with initial data>
}
}

Expand All @@ -144,7 +145,7 @@ the <config_file> in plain text.

Here is an example:

.. code-block:: json
.. code-block:: text

{
"users": {
Expand All @@ -161,12 +162,12 @@ Authorization could be enhanced by changing the method "decorated" using

Configs may also contain a "taxii" section as well, as shown below:

.. code-block:: json
.. code-block:: text

{
"taxii": {
"max_page_size": 100
"interop_requirements": true
"interop_requirements": True/False # the TAXII interop document has some additional requirements
rpiazza marked this conversation as resolved.
Show resolved Hide resolved
}
}

Expand All @@ -181,23 +182,22 @@ We welcome contributions for other back-end plugins.
Docker
------

We also provide a Docker image to make it easier to run *medallion*
We also provide a Docker image to make it easier to run *medallion* with the MongoDB backend. Use the --build argument
if the code has changed.

.. code-block:: bash

$ docker build . -t medallion -f docker_utils/Dockerfile
$ docker-compose up [--build]

If operating behind a proxy, add the following option (replacing `<proxy>` with
your proxy location and port): ``--build-arg https_proxy=<proxy>``.
This uses the information in docker-compose.yml to create a Docker container with medallion, mongo db and mongo-express

Then run the image
If operating behind a proxy, add the following to the medallion:build section of docker-compose.yml:

.. code-block:: bash
.. code-block:: text

$ docker run --rm -p 5000:5000 -v <directory>:/var/taxii medallion
HTTPS_PROXY: <proxy>

Replace ``<directory>`` with the full path to the directory containing your
medallion configuration.
replacing <proxy> with your proxy location and port.

Governance
----------
Expand Down
69 changes: 41 additions & 28 deletions docs/mongodb_schema.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,22 @@ Each Mongo database contains one or more collections. The term "collection" in

It is unfortunate that the term "collection" is also used to signify something unrelated in the TAXII specification. We will use the phrase "taxii collection" to distinguish them.

An instance of this schema can be populated via the file test/data/initialize_mongodb.py. This instance will be used for examples below.
You can initialize the database with content, by specifying a json file in the backend section of the medallion configuration.

Utilities to initialize your own Mongo DB can be found in test/generic_initialize_mongodb.py.
To initialize the database for testing use mediallion/test/data/default_data.json. Use the format of that file to determine how
to initialize with your own data.

For example:

.. code-block:: text

{
"backend": {
"module_class": "MongoBackend",
"uri": <Mongo DB server url> # e.g., 'mongodb://localhost:27017/'
"filename": <path to json file with initial data>
}
}

The discovery database
----------------------
Expand Down Expand Up @@ -55,7 +68,11 @@ Here is a document from the example database:
The api root databases
----------------------

Each api root is contained in a separate Mongo DB database. It has four collections: **status**, **objects**, **manifests**, and **collections**. To support multiple taxii collections, any document in the **objects** and **manifests** contains an extra property, "collection_id", to link it to the taxii collection that it is contained in. Because "_collection_id" property is not part of the TAXII specification, it will be stripped by *medallion* before any document is returned to the client.
Each api root is contained in a separate Mongo DB database. It has four collections: **status**, **objects**,
and **collections**. To support multiple TAXII collections, any document in the **objects** contains an extra
property, "collection_id", to link it to the taxii collection that it is contained in.
Because "_collection_id" property is not part of the TAXII specification, it will be stripped by *medallion*
before any document is returned to the client.

A document from the **collections** collection:

Expand All @@ -72,23 +89,32 @@ A document from the **collections** collection:
]
}

Because the STIX objects and the manifest entries share so much information, the manifest is stored with the object.

A document from the **objects** collection:

.. code-block:: json

{
"created": "2014-05-08T09:00:00.000Z",
"id": "indicator--a932fcc6-e032-176c-126f-cb970a5a1ade",
"labels": [
"file-hash-watchlist"
],
"modified": "2014-05-08T09:00:00.000Z",
"name": "File hash for Poison Ivy variant",
"pattern": "[file:hashes.'SHA-256' = 'ef537f25c895bfa782526529a9b63d97aa631564d5d789c2b765448c8635fb6c']",
"type": "indicator",
"valid_from": "2014-05-08T09:00:00.000000Z",
"_collection_id": "91a7b528-80eb-42ed-a74d-c6fbd5a26116"
}
"created": "2017-01-27T13:49:53.997Z",
"description": "Poison Ivy",
"id": "malware--c0931cc6-c75e-47e5-9036-78fabc95d4ec",
"is_family": True,
rpiazza marked this conversation as resolved.
Show resolved Hide resolved
"malware_types": [
"remote-access-trojan",
],
"modified": "2017-01-27T13:49:53.997Z",
"name": "Poison Ivy",
"spec_version": "2.1",
"type": "malware",
"_collection_id": "91a7b528-80eb-42ed-a74d-c6fbd5a26116",
"_manifest": {
"date_added": "2017-01-27T13:49:59.997000Z",
"id": "malware--c0931cc6-c75e-47e5-9036-78fabc95d4ec",
"media_type": "application/stix+json;version=2.1",
"version": "2017-01-27T13:49:53.997Z"
}
}

A document from the **status** collection:

Expand Down Expand Up @@ -117,18 +143,5 @@ A document from the **status** collection:
]
}

A document from the **manifest** collection:

.. code-block:: json

{
"id": "indicator--a932fcc6-e032-176c-126f-cb970a5a1ade",
"date_added": "2016-11-01T10:29:05Z",
"versions": [
"2014-05-08T09:00:00.000Z"
],
"media_types": [
"application/vnd.oasis.stix+json; version=2.0"
],
"_collection_id": "91a7b528-80eb-42ed-a74d-c6fbd5a26116"
}