Skip to content

Commit

Permalink
Merge pull request #19172 from newrelic/revert-19091-NR-310861
Browse files Browse the repository at this point in the history 
Revert "update fluent bit output plugin document"
  • Loading branch information
@nbaenam
nbaenam authored Nov 4, 2024
2 parents 9f778d6 + febce65 commit a3398b7
Show file tree
Hide file tree
Showing 2 changed files with 80 additions and 80 deletions.
1 change: 0 additions & 1 deletion src/content/docs/alerts/create-alert/create-alert-condition/alert-conditions.mdx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@ redirects:
- /docs/alerts-applied-intelligence/new-relic-alerts/advanced-alerts/advanced-techniques/select-product-targets-alert-condition
- /docs/alerts/create-alert/create-alert-condition/create-alert-conditions
- /docs/alerts/create-alert/create-alert-condition/update-or-disable-policies-conditions
- /docs/alerts-applied-intelligence/new-relic-alerts/alert-conditions/create-alert-conditions
- /docs/alerts/new-relic-alerts-beta/configuring-alert-policies/define-alert-conditions
- /docs/alerts/new-relic-alerts/configuring-alert-policies/define-alert-conditions
- /docs/alerts/new-relic-alerts-beta/configuring-alert-policies/select-product-targets-alert-condition
Expand Down
159 changes: 80 additions & 79 deletions src/content/docs/logs/forward-logs/fluent-bit-plugin-log-forwarding.mdx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@ tags:
- Logs
- Enable log management in New Relic
- Enable log monitoring in New Relic
- Fluent Bit
translate:
- kr
metaDescription: 'Install and configure the New Relic logging plugin for Fluent Bit, so you can use enhanced log management capabilities.'
Expand All @@ -14,113 +13,117 @@ redirects:
- /docs/logs/enable-logs/enable-logs/fluent-bit-plugin-logs
- /docs/logs/enable-log-monitoring-new-relic/enable-log-monitoring-new-relic/fluent-bit-plugin-log-forwarding
- /docs/logs/enable-log-management-new-relic/enable-log-monitoring-new-relic/fluent-bit-plugin-log-forwarding
freshnessValidatedDate: 2024-10-30
freshnessValidatedDate: never
---

If your log data is already being monitored by [Fluent Bit](https://fluentbit.io/), you can use our Fluent Bit output plugin to forward and enrich your log data in New Relic.

Forwarding your Fluent Bit logs to New Relic will give you enhanced <InlinePopover type="logs"/> capabilities to collect, process, explore, query, and alert on your log data. You have these options to install the Fluent Bit:
Forwarding your Fluent Bit logs to New Relic will give you enhanced <InlinePopover type="logs"/> capabilities to collect, process, explore, query, and alert on your log data.

* [Kubernetes installation](#k8s-installation)
* [On-host installation](#on-host-installation)
## Basic process [#compatibility-requirements]

## Kubernetes installation [#k8s-installation]
We have published a container with the plugin installed. It serves as a base image to be used by our Kubernetes integration. We recommend you use this base image and layer your own custom configuration files.

New Relic has a [Fluent Bit output plugin](https://github.com/newrelic/newrelic-fluent-bit-output) to forward your logs to New Relic log management. You can install this plugin as a standalone Docker image in a Kubernetes cluster, where it functions as a DaemonSet, also known as the Kubernetes plugin.
To forward your logs from Fluent Bit to New Relic:

You can install it in your cluster using our [Helm chart](https://github.com/newrelic/helm-charts/tree/master) in two ways:
1. Make sure you have:
* A New Relic <InlinePopover type="licenseKey"/>
* Fluent Bit 1.0 or higher (recommended), although v0.12 or higher is supported
* Fluent Bit Windows install directions can be found [here](https://docs.fluentbit.io/manual/installation/windows)
* Fluent Bit Linux install directions can be found [here](https://docs.fluentbit.io/manual/installation/linux)
2. [Install](#fluentbit-plugin) the Fluent Bit plugin.
3. [Configure](#configure-plugin) the Fluent Bit plugin.
4. [Test](#test-plugin) the Fluent Bit plugin.
5. Generate some traffic and wait a few minutes, then [check your account](#find-data) for data.

* Through our guided install
* Through a Docker image
## Install the Fluent Bit plugin [#fluentbit-plugin]

### Using our guided install [#helm-guided-install]
To install the Fluent Bit plugin:

Although the [`newrelic-logging`](https://github.com/newrelic/helm-charts/tree/master/charts/newrelic-logging) chart works as a standalone, we recommend installing it as part of the [`nri-bundle`](https://github.com/newrelic/helm-charts/tree/master/charts/nri-bundle) chart.
1. Navigate to New Relic's [Fluent Bit plugin repository on GitHub](https://github.com/newrelic/newrelic-fluent-bit-output).
2\. From the repository page, [clone or download the repository](https://help.github.com/en/articles/cloning-a-repository).
3\. Run the following command to build your plugin:

The best way to install this is through our [guided installation](/install/kubernetes/) process. This guided install generates the Helm commands required to install it.
```sh
cd newrelic-fluent-bit-output && make all
```

Alternatively, and not recommended, you can install it manually using Helm. Run this command to install the repo:
2. Store `out_newrelic.so` or `out_newrelic_winXX.dll` at a location that can be accessed by the `fluent-bit` daemon.

```bash
helm repo add newrelic https://helm-charts.newrelic.com
```
<Callout variant="tip">
If you'd rather not compile the plugin yourself, you can download pre-compiled versions from our [GitHub repository's releases page](https://github.com/newrelic/newrelic-fluent-bit-output/releases).
</Callout>

To update the repo, run this command:
## Upgrade the Fluent Bit plugin [#upgrade-plugin]

```bash
helm repo update newrelic
```
Before you upgrade your Fluent Bit plugin, run the following NRQL query to find the current versions of the output plugin being used in your system:

See [Uninstall Kubernetes integration](/docs/kubernetes-pixie/kubernetes-integration/installation/uninstall-kubernetes/) if you want to uninstall it.
```sql
FROM K8sContainerSample
SELECT latest(containerImage)
WHERE podName like '%newrelic-logging%'
FACET clusterName
```

### Using a Docker image [#helm-docker-image]
<Callout variant="caution">
Fluent Bit output plugin versions 1.16.0 through 1.19.2 are affected by a [security vulnerability (CVE-2024-4323)](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-4323). If you're using one of these versions, upgrade to version 2.0.0 or higher. For more information on this, see our [security bulletin NR24-01 - Fluent Bit](/docs/security/new-relic-security/security-bulletins/security-bulletin-nr24-01/).
</Callout>
Once the container is published with the plugin installed, it serves as a base image for our Kubernetes integration. If you've your own custom kubernetes integration, we recommend using our [Docker image](https://hub.docker.com/r/newrelic/newrelic-fluentbit-output) that comes with the [`newrelic-fluent-bit-output`](https://github.com/newrelic/newrelic-fluent-bit-output) plugin. Or you can use the Docker image as a base image and layer your own custom configuration files.
To upgrade, follow the [installation instructions](#fluentbit-plugin) or grab the latest pre-compiled version from our [GitHub repository](https://github.com/newrelic/newrelic-fluent-bit-output/releases).
## On-host installation [#on-host-installation]
## Install Fluent Bit output plugin [#fluent-bit]
To an on-host installation of the Fluent Bit plugin, follow these steps:
New Relic has a [Fluent Bit](https://fluentbit.io/) [output plugin](https://github.com/newrelic/newrelic-fluent-bit-output) to forward your logs to New Relic log management. This plugin is also provided in a standalone Docker image that can be installed in a Kubernetes cluster in the form of a DaemonSet, also known as the Kubernetes plugin.
1. Open the New Relic's [Fluent Bit plugin repository](https://github.com/newrelic/newrelic-fluent-bit-output) on GitHub.
You can install it in your cluster using our Helm chart in two ways.
2. From the repository page, [clone or download the repository](https://docs.github.com/en/repositories/creating-and-managing-repositories/cloning-a-repository).
### Use our guided install [#helm-guided-install]
3. Run this command to build your plugin:
Although the [newrelic-logging](https://github.com/newrelic/helm-charts/tree/master/charts/newrelic-logging) chart works as a standalone, we recommend installing it as part of the [nri-bundle](https://github.com/newrelic/helm-charts/tree/master/charts/nri-bundle) chart.
```sh
cd newrelic-fluent-bit-output && make all
```
The best way to install this is through our [guided installation](/docs/kubernetes-pixie/kubernetes-integration/installation/kubernetes-integration-install-configure/) process. This guided install can generate the Helm 3 commands required to install it ([see "Helm 3"](/docs/kubernetes-pixie/kubernetes-integration/installation/kubernetes-integration-install-configure/#finish-your-install)).
4. Store `out_newrelic.so` or `out_newrelic_winXX.dll` at a location that can be accessed by the `fluent-bit` daemon.
### Manual installation [#manual-helm-install]
<Callout variant="tip">
If you'd rather not compile the plugin yourself, you can download pre-compiled versions from our [GitHub repository's releases page](https://github.com/newrelic/newrelic-fluent-bit-output/releases).
</Callout>
Alternately, you can install it manually using Helm, by running this command to install the repo:
### Upgrade the Fluent Bit plugin [#upgrade-plugin]
```bash
helm repo add newrelic https://helm-charts.newrelic.com
```
Before you upgrade your Fluent Bit plugin, run the following NRQL query to find the current versions of the output plugin being used in your system:
To update the repo you can run:
```sql
FROM K8sContainerSample
SELECT latest(containerImage)
WHERE podName like '%newrelic-logging%'
FACET clusterName
```bash
helm repo update newrelic
```
<Callout variant="caution">
Note that the [security vulnerability (CVE-2024-4323)](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-4323) affects Fluent Bit output plugin versions 1.16.0 through 1.19.2. If you're using one of these versions, update to version 2.0.0 or higher. For more information on this, see our security bulletin [NR24-01 - Fluent Bit](/docs/security/new-relic-security/security-bulletins/security-bulletin-nr24-01/).
</Callout>
To update, follow the installation instructions or grab the latest pre-compiled version from our [GitHub repository](https://github.com/newrelic/newrelic-fluent-bit-output/releases).
Go here for [uninstallation instructions](/docs/kubernetes-pixie/kubernetes-integration/uninstall-kubernetes/).
## Configure the Fluent Bit plugin [#configure-plugin]
Fluent Bit needs to know the location of the New Relic plugin and the New Relic <InlinePopover type="licenseKey"/> to output data to New Relic.
Fluent Bit needs to know the location of the New Relic plugin and the New Relic <InlinePopover type="licenseKey"/> to output data to New Relic. To configure your Fluent Bit plugin:
<Callout variant="important">
Pay attention to white space when editing your config files. Be sure to use four spaces to indent and one space between keys and values.
</Callout>
To configure your Fluent Bit plugin, follow these steps:
1. Locate or create the `plugins.conf` file in your plugins directory.
1. Locate or create a `plugins.conf` file in your plugins directory.
2. In the `plugins.conf` file, add a reference to `out_newrelic.so`, adjacent to your `fluent-bit.conf` file:
```ini
[PLUGINS]
Path /PATH/TO/newrelic-fluent-bit-output/out_newrelic.so
```
```ini
[PLUGINS]
Path /PATH/TO/newrelic-fluent-bit-output/out_newrelic.so
```
3. In the `fluent-bit.conf` file, add the following line under the `service` block:
```ini
[SERVICE]
# This is the main configuration block for fluent bit.
# Ensure the follow line exists somewhere in the SERVICE block
Plugins_File plugins.conf
```
```ini
[SERVICE]
# This is the main configuration block for fluent bit.
# Ensure the follow line exists somewhere in the SERVICE block
Plugins_File plugins.conf
```
4. At the bottom of the `fluent-bit.conf` file, add the following to set up the input, filter, and output sections. Replace the placeholder text with your <InlinePopover type="licenseKey"/>:
Expand Down Expand Up @@ -159,29 +162,30 @@ To configure your Fluent Bit plugin, follow these steps:
licenseKey YOUR_LICENSE_KEY
```

5. Restart your Fluent Bit instance by running this command:
5. Restart your Fluent Bit instance with the following command:

```sh
fluent-bit -c /PATH/TO/fluent-bit.conf
```

<InstallFeedback/>

## Test the Fluent Bit plugin [#test-plugin]

To test if your Fluent Bit plugin is receiving input from a log file:

1. Run this command to append a test log message to your log file:
1. Run the following command to append a test log message to your log file:

```sh
echo "test message" >> /PATH/TO/YOUR/LOG/FILE
```
2. Search our [logs UI](https://one.newrelic.com/launcher/logger.log-launcher) for `test message`.

2. Search for `test message` in our [logs UI](https://one.newrelic.com/launcher/logger.log-launcher) .

For more options, see the [the modify filter on the Fluent Bit documentation](https://docs.fluentbit.io/manual/pipeline/filters/modify). See also our documentation to [forward your logs using the infrastructure agent](/docs/logs/forward-logs/forward-your-logs-using-infrastructure-agent/#automatically-inserted-attributes).
For more options, see the [Fluent Bit modify filter documentation](https://docs.fluentbit.io/manual/pipeline/filters/modify) and our documentation to [forward your logs using the infrastructure agent](/docs/logs/enable-log-management-new-relic/enable-log-monitoring-new-relic/forward-your-logs-using-infrastructure-agent/#automatically-inserted-attributes).

## Optional: Configure plugin attributes [#instrument-plugin]

Once you have installed and [configured](#configure-plugin) the Fluent Bit plugin, you can use the following attributes to configure how the plugin sends data to New Relic:
Once you have [installed](#fluentbit-plugin) and [configured](#configure-plugin) the Fluent Bit plugin, you can use the following attributes to configure how the plugin sends data to New Relic:

<table>
<thead>
Expand Down Expand Up @@ -233,7 +237,7 @@ Once you have installed and [configured](#configure-plugin) the Fluent Bit plugi
</td>

<td>
<DNT>**Deprecated.**</DNT> Takes a New Relic [insights insert key](/docs/insights/insights-data-sources/custom-data/send-custom-events-event-api#register), but using the `licenseKey` attribute is preferred. Use either `licenseKey` or `apiKey`, not both.
<DNT>**Deprecated.**</DNT> Takes a New Relic [Insights insert key](/docs/insights/insights-data-sources/custom-data/send-custom-events-event-api#register), but using the `licenseKey` attribute is preferred. Use either `licenseKey` or `apiKey`, not both.
</td>
</tr>

Expand All @@ -243,36 +247,33 @@ Once you have installed and [configured](#configure-plugin) the Fluent Bit plugi
</td>

<td>
Defaults to `https://log-api.newrelic.com/log/v1`. If you're using an EU key, you need set it to `https://log-api.eu.newrelic.com/log/v1`
Defaults to `https://log-api.newrelic.com/log/v1` - If using an EU key needs to be set to `https://log-api.eu.newrelic.com/log/v1`
</td>
</tr>
</tbody>
</table>

## View log data [#find-data]

If everything is configured correctly and New Relic collects your data, you should see log data in both of these places:
If everything is configured correctly and your data is being collected, you should see log data in both of these places:

* Our [logs UI](https://one.newrelic.com/launcher/logger.log-launcher)
* Our tools for running [NRQL queries](/docs/query-your-data/explore-query-data/query-builder/use-advanced-nrql-mode-query-data/). For example, you can run a query like this:
* Our tools for running [NRQL queries](/docs/chart-builder/use-chart-builder/choose-data/use-advanced-nrql-mode-specify-data). For example, you can execute a query like this:

```sql
SELECT * FROM Log
```

If don't get any data after you enable our log management capabilities, follow our standard log [troubleshooting procedures](/docs/logs/troubleshooting/no-log-data-appears-ui/).
If no data appears after you enable our log management capabilities, follow our [standard log troubleshooting procedures](/docs/logs/log-management/troubleshooting/no-log-data-appears-ui/).

## What's next? [#what-next]

Explore logging data across your platform with our [logs UI](/docs/logs/ui-data/use-logs-ui/).
Explore logging data across your platform with our [logs UI](/docs/logs/log-management/ui-data/use-logs-ui).

* Get deeper visibility into both your application and your platform performance data by forwarding your logs with our [logs in context](/docs/logs/enable-log-management-new-relic/configure-logs-context/configure-logs-context-apm-agents/) capabilities.

* Set up [alerts](/docs/alerts/create-alert/create-alert-condition/alert-conditions/).

* Set up [alerts](/docs/alerts-applied-intelligence/new-relic-alerts/alert-conditions/create-alert-conditions/).
* [Query your data](/docs/query-your-data/explore-query-data/get-started/introduction-querying-new-relic-data/) and [create dashboards](/docs/query-your-data/explore-query-data/dashboards/introduction-dashboards/).

## Disable log forwarding [#disable]

To disable log forwarding capabilities, follow standard procedures in [Fluent Bit documentation](https://fluentbit.io/). You don't need to do anything else in New Relic.
To disable log forwarding capabilities, follow standard procedures in [Fluent Bit documentation](https://fluentbit.io/). You do not need to do anything else in New Relic.

0 comments on commit a3398b7

Please sign in to comment.