diff --git a/docs/sources/get-started/quick-start.md b/docs/sources/get-started/quick-start.md index c6f6dcf4d21d7..9fa0e045da004 100644 --- a/docs/sources/get-started/quick-start.md +++ b/docs/sources/get-started/quick-start.md @@ -2,109 +2,214 @@ title: Quickstart to run Loki locally menuTitle: Loki quickstart weight: 200 -description: How to create and use a simple local Loki cluster for testing and evaluation purposes. +description: How to create and use a local Loki cluster for testing and evaluation purposes. +killercoda: + comment: | + The killercoda front matter and the HTML comments that start ' + # Quickstart to run Loki locally If you want to experiment with Loki, you can run Loki locally using the Docker Compose file that ships with Loki. It runs Loki in a [monolithic deployment](https://grafana.com/docs/loki//get-started/deployment-modes/#monolithic-mode) mode and includes a sample application to generate logs. -The Docker Compose configuration instantiates the following components, each in its own container: +The Docker Compose configuration runs the following components, each in its own container: + +- **flog**: which generates log lines. + [flog](https://github.com/mingrammer/flog) is a log generator for common log formats. + +- **Grafana Alloy**: which scrapes the log lines from flog, and pushes them to Loki through the gateway. +- **Gateway** (nginx) which receives requests and redirects them to the appropriate container based on the request's URL. +- **Loki read component**: which runs a Query Frontend and a Querier. +- **Loki write component**: which runs a Distributor and an Ingester. +- **Loki backend component**: which runs an Index Gateway, Compactor, Ruler, Bloom Compactor (experimental), and Bloom Gateway (experimental). +- **Minio**: which Loki uses to store its index and chunks. +- **Grafana**: which provides visualization of the log lines captured within Loki. + +{{< figure max-width="75%" src="/media/docs/loki/get-started-flog-v3.png" caption="Getting started sample application" alt="Getting started sample application" >}} + + + +## Before you begin -- **flog** a sample application which generates log lines. [flog](https://github.com/mingrammer/flog) is a log generator for common log formats. -- **Grafana Alloy** which scrapes the log lines from flog, and pushes them to Loki through the gateway. -- **Gateway** (NGINX) which receives requests and redirects them to the appropriate container based on the request's URL. -- One Loki **read** component (Query Frontend, Querier). -- One Loki **write** component (Distributor, Ingester). -- One Loki **backend** component (Index Gateway, Compactor, Ruler, Bloom Compactor (Experimental), Bloom Gateway (Experimental)). -- **Minio** an S3-compatible object store which Loki uses to store its index and chunks. -- **Grafana** which provides visualization of the log lines captured within Loki. +{{< admonition type="tip" >}} +Alternatively, you can try out this example in our interactive learning environment: [Loki Quickstart Sandbox](https://killercoda.com/grafana-labs/course/loki/loki-quickstart). -{{< figure max-width="75%" src="/media/docs/loki/get-started-flog-v3.png" caption="Getting started sample application" alt="Getting started sample application">}} +It's a fully configured environment with all the dependencies already installed. + +![Interactive](https://raw.githubusercontent.com/grafana/killercoda/prod/assets/loki-ile.svg) +{{< /admonition >}} + +- Install [Docker](https://docs.docker.com/install) +- Install [Docker Compose](https://docs.docker.com/compose/install) ## Interactive Learning Environment {{< admonition type="note" >}} -The Interactive Learning Environment is currently in trial. Please provide feedback, report bugs, and raise issues in the [Grafana Killercoda Repository](https://github.com/grafana/killercoda). +The Interactive Learning Environment is in trial. + +Provide feedback, report bugs, and raise issues in the [Grafana Killercoda repository](https://github.com/grafana/killercoda). {{< /admonition >}} -Try out this demo within our interactive learning environment: [Loki Quickstart Sandbox](https://killercoda.com/grafana-labs/course/loki/loki-quickstart) +Try out this demo within our interactive learning environment: [Loki Quickstart Sandbox](https://killercoda.com/grafana-labs/course/loki/loki-quickstart) -- A free Killercoda account is required to verify you are not a bot. -- Tutorial instructions are located on the left-hand side of the screen. Click to move on to the next section. -- All commands run inside the interactive terminal. Grafana can also be accessed via the URL links provided within the sandbox. +- You must have a free Killercoda account to verify you aren't a bot. +- Tutorial instructions are located on the left-side of the screen. + Click to move on to the next section. +- All commands run inside the interactive terminal. +- You can access Grafana with the URL links provided within the sandbox. -## Installing Loki and collecting sample logs + -Prerequisites +## Install Loki and collecting sample logs -- [Docker](https://docs.docker.com/install) -- [Docker Compose](https://docs.docker.com/compose/install) + -{{% admonition type="note" %}} +{{< admonition type="note" >}} This quickstart assumes you are running Linux. -{{% /admonition %}} +{{< /admonition >}} + + **To install Loki locally, follow these steps:** -1. Create a directory called `evaluate-loki` for the demo environment. Make `evaluate-loki` your current working directory: +1. Create a directory called `evaluate-loki` for the demo environment. + Make `evaluate-loki` your current working directory: + + - ```bash - mkdir evaluate-loki - cd evaluate-loki - ``` + ```bash + mkdir evaluate-loki + cd evaluate-loki + ``` + + 1. Download `loki-config.yaml`, `alloy-local-config.yaml`, and `docker-compose.yaml`: - ```bash - wget https://raw.githubusercontent.com/grafana/loki/main/examples/getting-started/loki-config.yaml -O loki-config.yaml - wget https://raw.githubusercontent.com/grafana/loki/main/examples/getting-started/alloy-local-config.yaml -O alloy-local-config.yaml - wget https://raw.githubusercontent.com/grafana/loki/main/examples/getting-started/docker-compose.yaml -O docker-compose.yaml - ``` + + + ```bash + wget https://raw.githubusercontent.com/grafana/loki/main/examples/getting-started/loki-config.yaml -O loki-config.yaml + wget https://raw.githubusercontent.com/grafana/loki/main/examples/getting-started/alloy-local-config.yaml -O alloy-local-config.yaml + wget https://raw.githubusercontent.com/grafana/loki/main/examples/getting-started/docker-compose.yaml -O docker-compose.yaml + ``` + + 1. Deploy the sample Docker image. - With `evaluate-loki` as the current working directory, start the demo environment using `docker compose`: + With `evaluate-loki` as the current working directory, start the demo environment using `docker compose`: + + + + ```bash + docker compose up -d + ``` + + + + {{< docs/ignore >}} + + + ```bash + docker-compose up -d + ``` - ```bash - docker compose up -d - ``` + - You should see something similar to the following: + {{< /docs/ignore >}} - ```bash - ✔ Network evaluate-loki_loki Created 0.1s - ✔ Container evaluate-loki-minio-1 Started 0.6s - ✔ Container evaluate-loki-flog-1 Started 0.6s - ✔ Container evaluate-loki-backend-1 Started 0.8s - ✔ Container evaluate-loki-write-1 Started 0.8s - ✔ Container evaluate-loki-read-1 Started 0.8s - ✔ Container evaluate-loki-gateway-1 Started 1.1s - ✔ Container evaluate-loki-grafana-1 Started 1.4s - ✔ Container evaluate-loki-alloy-1 Started 1.4s - ``` + At the end of the command, you should see something similar to the following: + + + + ```console + ✔ Network evaluate-loki_loki Created 0.1s + ✔ Container evaluate-loki-minio-1 Started 0.6s + ✔ Container evaluate-loki-flog-1 Started 0.6s + ✔ Container evaluate-loki-backend-1 Started 0.8s + ✔ Container evaluate-loki-write-1 Started 0.8s + ✔ Container evaluate-loki-read-1 Started 0.8s + ✔ Container evaluate-loki-gateway-1 Started 1.1s + ✔ Container evaluate-loki-grafana-1 Started 1.4s + ✔ Container evaluate-loki-alloy-1 Started 1.4s + ``` + + + + {{< docs/ignore >}} + + ```console + Creating evaluate-loki_flog_1 ... done + Creating evaluate-loki_minio_1 ... done + Creating evaluate-loki_read_1 ... done + Creating evaluate-loki_write_1 ... done + Creating evaluate-loki_gateway_1 ... done + Creating evaluate-loki_alloy_1 ... done + Creating evaluate-loki_grafana_1 ... done + Creating evaluate-loki_backend_1 ... done + ``` + + {{< /docs/ignore >}} 1. (Optional) Verify that the Loki cluster is up and running. - - The read component returns `ready` when you point a web browser at [http://localhost:3101/ready](http://localhost:3101/ready). The message `Query Frontend not ready: not ready: number of schedulers this worker is connected to is 0` will show prior to the read component being ready. - - The write component returns `ready` when you point a web browser at [http://localhost:3102/ready](http://localhost:3102/ready). The message `Ingester not ready: waiting for 15s after being ready` will show prior to the write component being ready. - + + - The read component returns `ready` when you browse to [http://localhost:3101/ready](http://localhost:3101/ready). + The message `Query Frontend not ready: not ready: number of schedulers this worker is connected to is 0` shows until the read component is ready. + - The write component returns `ready` when you browse to [http://localhost:3102/ready](http://localhost:3102/ready). + The message `Ingester not ready: waiting for 15s after being ready` shows until the write component is ready. + 1. (Optional) Verify that Grafana Alloy is running. - - Grafana Alloy's UI can be accessed at [http://localhost:12345](http://localhost:12345). + - You can access the Grafana Alloy UI at [http://localhost:12345](http://localhost:12345). + + + + + +## View your logs in Grafana -## Viewing your logs in Grafana +After you have collected logs, you will want to view them. +You can view your logs using the command line interface, [LogCLI](/docs/loki//query/logcli/), but the easiest way to view your logs is with Grafana. -Once you have collected logs, you will want to view them. You can view your logs using the command line interface, [LogCLI](/docs/loki//query/logcli/), but the easiest way to view your logs is with Grafana. +1. Use Grafana to query the Loki data source. -1. Use Grafana to query the Loki data source. + The test environment includes [Grafana](https://grafana.com/docs/grafana/latest/), which you can use to query and observe the sample logs generated by the flog application. - The test environment includes [Grafana](https://grafana.com/docs/grafana/latest/), which you can use to query and observe the sample logs generated by the flog application. You can access the Grafana cluster by navigating to [http://localhost:3000](http://localhost:3000). The Grafana instance provided with this demo has a Loki [data source](https://grafana.com/docs/grafana/latest/datasources/loki/) already configured. + You can access the Grafana cluster by browsing to [http://localhost:3000](http://localhost:3000). - {{< figure src="/media/docs/loki/grafana-query-builder-v2.png" caption="Grafana Explore" alt="Grafana Explore">}} + The Grafana instance in this demonstration has a Loki [data source](https://grafana.com/docs/grafana/latest/datasources/loki/) already configured. -1. From the Grafana main menu, click the **Explore** icon (1) to launch the Explore tab. To learn more about Explore, refer the [Explore](https://grafana.com/docs/grafana/latest/explore/) documentation. + {{< figure src="/media/docs/loki/grafana-query-builder-v2.png" caption="Grafana Explore" alt="Grafana Explore" >}} -1. From the menu in the dashboard header, select the Loki data source (2). This displays the Loki query editor. In the query editor you use the Loki query language, [LogQL](https://grafana.com/docs/loki//query/), to query your logs. - To learn more about the query editor, refer to the [query editor documentation](https://grafana.com/docs/grafana/latest/datasources/loki/query-editor/). +1. From the Grafana main menu, click the **Explore** icon (1) to open the Explore tab. + + To learn more about Explore, refer to the [Explore](https://grafana.com/docs/grafana/latest/explore/) documentation. + +1. From the menu in the dashboard header, select the Loki data source (2). + + This displays the Loki query editor. + + In the query editor you use the Loki query language, [LogQL](https://grafana.com/docs/loki//query/), to query your logs. + To learn more about the query editor, refer to the [query editor documentation](https://grafana.com/docs/grafana/latest/datasources/loki/query-editor/). 1. The Loki query editor has two modes (3): @@ -115,47 +220,61 @@ Once you have collected logs, you will want to view them. You can view your log 1. Click **Code** (3) to work in Code mode in the query editor. - Here are some basic sample queries to get you started using LogQL. Note that these queries assume that you followed the instructions to create a directory called `evaluate-loki`. If you installed in a different directory, you’ll need to modify these queries to match your installation directory. After copying any of these queries into the query editor, click **Run Query** (4) to execute the query. + Here are some sample queries to get you started using LogQL. + These queries assume that you followed the instructions to create a directory called `evaluate-loki`. + + If you installed in a different directory, you’ll need to modify these queries to match your installation directory. - 1. View all the log lines which have the container label "flog": + After copying any of these queries into the query editor, click **Run Query** (4) to execute the query. - ```bash - {container="evaluate-loki-flog-1"} - ``` + 1. View all the log lines which have the container label `evaluate-loki-flog-1`: - In Loki, this is called a log stream. Loki uses [labels](https://grafana.com/docs/loki//get-started/labels/) as metadata to describe log streams. Loki queries always start with a label selector. In the query above, the label selector is `container`. + ```bash + {container="evaluate-loki-flog-1"} + ``` - 1. To view all the log lines which have the container label "grafana": + In Loki, this is a log stream. - ```bash - {container="evaluate-loki-grafana-1"} - ``` + Loki uses [labels](https://grafana.com/docs/loki//get-started/labels/) as metadata to describe log streams. - 1. Find all the log lines in the container=flog stream that contain the string "status": + Loki queries always start with a label selector. + In the previous query, the label selector is `{container="evaluate-loki-flog-1"}`. - ```bash - {container="evaluate-loki-flog-1"} |= `status` - ``` + 1. To view all the log lines which have the container label `evaluate-loki-grafana-1`: - 1. Find all the log lines in the container=flog stream where the JSON field "status" is "404": + ```bash + {container="evaluate-loki-grafana-1"} + ``` - ```bash - {container="evaluate-loki-flog-1"} | json | status=`404` - ``` + 1. Find all the log lines in the `{container="evaluate-loki-flog-1"}` stream that contain the string `status`: - 1. Calculate the number of logs per second where the JSON field "status" is "404": + ```bash + {container="evaluate-loki-flog-1"} |= `status` + ``` - ```bash - sum by(container) (rate({container="evaluate-loki-flog-1"} | json | status=`404` [$__auto])) - ``` + 1. Find all the log lines in the `{container="evaluate-loki-flog-1"}` stream where the JSON field `status` has the value `404`: - The final query above is a metric query which returns a time series. This will trigger Grafana to draw a graph of the results. You can change the type of graph for a different view of the data. Click **Bars** to view a bar graph of the data. + ```bash + {container="evaluate-loki-flog-1"} | json | status=`404` + ``` -1. Click the **Builder** tab (3) to return to Builder mode in the query editor. - 1. In Builder view, click **Kick start your query**(5). - 1. Expand the **Log query starters** section. - 1. Select the first choice, **Parse log lines with logfmt parser**, by clicking **Use this query**. - 1. On the Explore tab, click **Label browser**, in the dialog select a container and click **Show logs**. + 1. Calculate the number of logs per second where the JSON field `status` has the value `404`: + + ```bash + sum by(container) (rate({container="evaluate-loki-flog-1"} | json | status=`404` [$__auto])) + ``` + + The final query is a metric query which returns a time series. + This makes Grafana draw a graph of the results. + + You can change the type of graph for a different view of the data. + Click **Bars** to view a bar graph of the data. + +1. Click the **Builder** tab (3) to return to builder mode in the query editor. + 1. In builder mode, click **Kick start your query** (5). + 1. Expand the **Log query starters** section. + 1. Select the first choice, **Parse log lines with logfmt parser**, by clicking **Use this query**. + 1. On the Explore tab, click **Label browser**, in the dialog select a container and click **Show logs**. For a thorough introduction to LogQL, refer to the [LogQL reference](https://grafana.com/docs/loki//query/). @@ -166,7 +285,7 @@ Here are some more sample queries that you can run using the Flog sample data. To see all the log lines that flog has generated, enter the LogQL query: ```bash -{container="evaluate-loki-flog-1"}|= `` +{container="evaluate-loki-flog-1"} ``` The flog app generates log lines for simulated HTTP requests. @@ -189,7 +308,7 @@ To see every log line with a 401 status (unauthorized error), enter the LogQL qu {container="evaluate-loki-flog-1"} | json | status="401" ``` -To see every log line that does not contain the value 401: +To see every log line that doesn't contain the text `401`: ```bash {container="evaluate-loki-flog-1"} != "401" @@ -197,8 +316,12 @@ To see every log line that does not contain the value 401: For more examples, refer to the [query documentation](https://grafana.com/docs/loki//query/query_examples/). + + ## Complete metrics, logs, traces, and profiling example -If you would like to use a demo that includes Mimir, Loki, Tempo, and Grafana, you can use [Introduction to Metrics, Logs, Traces, and Profiling in Grafana](https://github.com/grafana/intro-to-mlt). `Intro-to-mltp` provides a self-contained environment for learning about Mimir, Loki, Tempo, and Grafana. +If you would like to run a demonstration environment that includes Mimir, Loki, Tempo, and Grafana, you can use [Introduction to Metrics, Logs, Traces, and Profiling in Grafana](https://github.com/grafana/intro-to-mlt). +It's a self-contained environment for learning about Mimir, Loki, Tempo, and Grafana. -The project includes detailed explanations of each component and annotated configurations for a single-instance deployment. Data from `intro-to-mltp` can also be pushed to Grafana Cloud. +The project includes detailed explanations of each component and annotated configurations for a single-instance deployment. +You can also push the data from the environment to [Grafana Cloud](https://grafana.com/cloud/).