tap-newrelic

tap-newrelic is a Singer tap for NewRelic.

Build with the Singer SDK.

It uses the NerdGraph API to fetch data using NRQL.

Currently, only synthetics checks data (SyntheticCheck) and logs (Log) are natively supported, however the tap supports custom queries for any query that is orderable by timestamp. In addition it should be streightforward to add other data sources, PRs accepted.

Installation

pip install tap-newrelic

Configuration

Accepted Config Options

{
  // required:
  "api_key": "ABCD-XXXXXXXXXXXXXXXXXXXXXXXXXXX",
  "account_id": 12345678,
  "start_date": "2021-01-00T00:00:00Z",
  // optional, defaults to https://api.newrelic.com/graphql
  "api_url": "https://api.eu.newrelic.com/graphql",
  // optional, a list of custom queries to run
  "custom_queries": {
    "name": "my_custom_event",
    "query": "SELECT * FROM my_custom_event",
    // optional, override the primary keys for your event
    "key_properties": ["my_event_id"]
  }
}

A full list of supported settings and capabilities for this tap is available by running:

tap-newrelic --about

Source Authentication and Authorization

Use of the API requires an API Key.

Usage

You can easily run tap-newrelic by itself or in a pipeline using Meltano.

Executing the Tap Directly

tap-newrelic --version
tap-newrelic --help
tap-newrelic --config CONFIG --discover > ./catalog.json

Initialize your Development Environment

pipx install poetry
poetry install

Create and Run Tests

Create tests within the tap_newrelic/tests subfolder and then run:

poetry run pytest

You can also test the tap-newrelic CLI interface directly using poetry run:

poetry run tap-newrelic --help

Testing with Meltano

Note: This tap will work in any Singer environment and does not require Meltano. Examples here are for convenience and to streamline end-to-end orchestration scenarios.

Next, install Meltano (if you haven't already) and any needed plugins:

# Install meltano
pipx install meltano
# Initialize meltano within this directory
cd tap-newrelic
meltano install

Now you can test and orchestrate using Meltano:

# Test invocation:
meltano invoke tap-newrelic --version
# OR run a test `elt` pipeline:
meltano elt tap-newrelic target-jsonl

Singer SDK Dev Guide

See the dev guide for more instructions on how to use the Singer SDK to develop your own taps and targets.