Skip to content

Commit

Permalink
WIP
Browse files Browse the repository at this point in the history
  • Loading branch information
WilliGautier committed Oct 4, 2024
1 parent 044f4ef commit 8c96e83
Show file tree
Hide file tree
Showing 9 changed files with 748 additions and 220 deletions.
6 changes: 3 additions & 3 deletions arlas/cli/index.py
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ def create(
@indices.command(help="Index data")
def data(
index: str = typer.Argument(help="index's name"),
files: list[str] = typer.Argument(help="List of pathes to the file conaining the data. Format: NDJSON"),
files: list[str] = typer.Argument(help="List of paths to the file(s) containing the data. Format: NDJSON"),
bulk: int = typer.Option(default=100, help="Bulk size for indexing data")
):
config = variables["arlas"]
Expand All @@ -89,9 +89,9 @@ def data(

@indices.command(help="Generate the mapping based on the data")
def mapping(
file: str = typer.Argument(help="Path to the file conaining the data. Format: NDJSON"),
file: str = typer.Argument(help="Path to the file containing the data. Format: NDJSON"),
nb_lines: int = typer.Option(default=2, help="Number of line to consider for generating the mapping. Avoid going over 10."),
field_mapping: list[str] = typer.Option(default=[], help="Overide the mapping with the provided field path/type. Example: fragment.location:geo_point. Important: the full field path must be provided."),
field_mapping: list[str] = typer.Option(default=[], help="Override the mapping with the provided field path/type. Example: fragment.location:geo_point. Important: the full field path must be provided."),
no_fulltext: list[str] = typer.Option(default=[], help="List of keyword or text fields that should not be in the fulltext search. Important: the field name only must be provided."),
push_on: str = typer.Option(default=None, help="Push the generated mapping for the provided index name"),
):
Expand Down
127 changes: 127 additions & 0 deletions docs/docs/configuration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
# Configuration


## Initial configuration
`arlas_cli` uses a yaml file for storing various ARLAS and elasticsearch configurations.

[One is automatically created for your convenience at the first launch](https://raw.githubusercontent.com/gisaia/arlas-cli/master/configuration.yaml).

<!-- termynal -->
```shell
> arlas_cli --version
X.X.X
Warning : no configuration file found, we created an empty one for
you (~/.arlas/cli/configuration.yaml).
```

By default, the file is located in `~/.arlas/cli/configuration.yaml`.
It contains the ARLAS demo endpoint and the local ARLAS and elasticsearch endpoints.

The configuration can also contain references to collection models for creating collections. A default one is provided for ARLAS EO. A reference can be an http url or a path to a local file.
It can also contain references to index mappings for creating indices. A default one is provided for ARLAS EO.

## Configurations

The command line uses the `${HOME}/.arlas/cli/configuration.yaml` configuration file:

```yaml
arlas:
local:
allow_delete: true
elastic:
headers:
Content-Type: application/json
location: http://localhost:9200
server:
headers:
Content-Type: application/json
location: http://localhost:9999/arlas
mappings:
arlas_eo:
headers: null
location: https://raw.githubusercontent.com/gisaia/ARLAS-EO/master/mapping.json
models:
arlas_eo:
headers: null
location: https://raw.githubusercontent.com/gisaia/ARLAS-EO/master/collection.json
```
The `arlas` section contains the different deployment configurations. The mapping section lists the mapping template that you can use.
Finally, the models are the templates for the collections. A [detailed description](model/README.md) of the configuration structure is provided.

### Create, describe and delete a configuration for `arlas_cli`

You can edit directly the `${HOME}/.arlas/cli/configuration.yaml` configuration file to update your configurations. You can also use the command line itself.

To list the configurations:
<!-- termynal -->
```shell
> arlas_cli confs list
+-----------+-----------------------------+
| name | url |
+-----------+-----------------------------+
| local | http://localhost:9999/arlas |
| test_conf | http://localhost:9999 |
+-----------+-----------------------------+
```

To describe a configuration:

<!-- termynal -->
```shell
arlas_cli confs describe local
allow_delete: true
authorization: null
elastic:
headers:
Content-Type: application/json
location: http://localhost:9200
server:
headers:
Content-Type: application/json
location: http://localhost:9999/arlas
```

To create a simple configuration:

<!-- termynal -->
```shell
> arlas_cli confs create dev_conf \
--server http://localhost:9999 \
--headers "Content-Type:application/json" \
--elastic http://localhost:9200 \
--elastic-headers "Content-Type:application/json" \
--no-allow-delete
```

For an arlas configuration with authentication:

<!-- termynal -->
```shell
> arlas_cli --config-file /tmp/configuration.yaml confs \
create myarlas_as_user \
--server http://myserver/arlas \
--headers "arlas-org-filter:my_org_name" \
--headers "Content-Type:application/json" \
--no-allow-delete \
--auth-token-url http://myserver/arlas_iam_server/session \
--auth-login user \
--auth-password my_password \
--auth-headers "Content-Type:application/json;charset=utf-8"\
--auth-arlas-iam
```

To delete the configuration:

<!-- termynal -->
```shell
> arlas_cli confs delete dev_conf
```

Also, it is possible to use a different configuration file than the one placed in your home directory (`$HOME/.arlas/cli/configuration.yaml`):

<!-- termynal -->
```shell
> arlas_cli --config-file /tmp/config.yaml
Warning : no configuration file found, we created an empty one for you (/tmp/config.yaml).
```
1 change: 1 addition & 0 deletions docs/docs/confs.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
# Configurations

!!! tip
At its first launch, `arlas_cli` will create a first configuration file for you (`$HOME/.arlas/cli/configuration.yaml`), with two ARLAS configurations, one pointing at a local deployment, one on ARLAS demo (without elasticsearch).
Expand Down
2 changes: 2 additions & 0 deletions docs/docs/iam.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
# Identity & Access Management

## Manage organisations, users, permissions and groups

<!-- termynal -->
Expand Down
Loading

0 comments on commit 8c96e83

Please sign in to comment.