Instrumentation of popular libraries

[tombruijn]

• Outgoing HTTP requests (HTTPoison, Tesla)
• Other frameworks
  • Absinthe - Add absinthe support #751
• Background job libraries (verk, exq, Toniq (Toniq errors are not tracked #38) etc.)
• etc

Recommend us libraries :)

[johnhamelink]

I'd love to see job queue integration (verk, exq, etc), much how appsignal monitors Sidekiq in Ruby.

[tombruijn]

Hi @johnhamelink

Thanks for the suggestions!
We're currently working on a 1.2 and 1.3 release of the Elixir library. We've not planned adding support for more libraries so far, but we're going to put your suggestions on our list :)

[teamon]

Thanks for adding tesla to the list! 💃

The simplest integration would be via middleware:

defmodule Appsignal.Tesla do
  import Appsignal.Instrumentation.Helpers, only: [instrument: 3]

  def call(env, next, opts) do
    verb = env.method |> to_string |> String.upcase
    instrument "net.http", "#{verb} #{env.url}", fn ->
      Tesla.run(env, next)
    end
  end
end

used like this:

defmodule MyApi do
  use Tesla
  # ...
  plug Appsignal.Tesla
end

[tombruijn]

I've added a page to our docs website about Absinthe support - https://docs.appsignal.com/elixir/integrations/absinthe.html .
This does not mean we officially support Absinthe and all of its features, but adds some basic support we had some time to look into when someone reported an issue with using our Plug in an Absinthe app.
Thought we'd share it, as it might be useful to someone running into this issue.

[quatermain]

any progress with exq? Maybe some suggestions how to do it temporary? We would greatly appreciate it

[tombruijn]

Hi @quatermain , we don't have any plans for exq yet. We're still working our way to a the best setup for the AppSignal integration for Elixir. When we're happy with that we'll start adding more integrations.

For now you can try and see if you can add the integration with an exq middleware. I've based the example below on the exq logger middleware and it's completely untested, but hopefully it will help you on your way :)

# For more information
# https://github.com/akira/exq#middleware-support

# Sample exq middleware of AppSignal
defmodule Appsignal.Exq.Middleware do
  @behaviour Exq.Middleware.Behaviour

  alias Exq.Middleware.Pipeline
  alias Appsignal.Transaction
  import Pipeline

  def before_work(pipeline) do
    Transaction.start(pipeline.assigns.job.jid, :background)
    pipeline
  end

  def after_processed_work(pipeline) do
    complete_transaction(Transaction.lookup, pipeline)
    pipeline
  end

  def after_failed_work(pipeline) do
    transaction = Transaction.lookup
    # Store error on transaction
    # TODO: Maybe you can get the reason and stacktrace from the pipeline as well?
    transaction.set_error("reason", to_string(pipeline.assigns.error_message), System.stacktrace)
    complete_transaction(transaction, pipeline)
    pipeline
  end

  defp complete_transaction(transaction, pipeline) do
    # TODO: Get job name from pipeline
    Transaction.set_action(pipeline.assigns.worker_module)

    if Transaction.finish(transaction) == :sample do
      # Fetch data from pipeline and add it to this function call
      Transaction.set_sample_data(transaction, "environment", %{other: "data"})
    end

    :ok = Transaction.complete(transaction)
  end
end

# TODO: Add this middleware to the Exq configuration:
# middleware: [Appsignal.Exq.Middleware, Exq.Middleware.Logger]

[nirev]

This is what I'm using in production for ~1year.
It works like a charm and very similar to what you posted!

defmodule Exq.Middleware.AppSignal do
  @behaviour Exq.Middleware.Behaviour

  alias Exq.Middleware.Pipeline
  import Pipeline

  def before_work(pipeline) do
    # Start an AppSignal transaction
    transaction = Appsignal.Transaction.start(
      Appsignal.Transaction.generate_id,
      :background_job
    )
    |> Appsignal.Transaction.set_action("Exq/#{pipeline.assigns.worker_module}")
    |> Appsignal.Transaction.set_sample_data(
      "environment", %{job_id: pipeline.assigns.job.jid}
    )
    assign(pipeline, :appsignal_transaction, transaction)
  end

  def after_processed_work(pipeline) do
    transaction = pipeline.assigns.appsignal_transaction
    Appsignal.Transaction.finish(transaction)
    Appsignal.Transaction.complete(transaction)

    pipeline
  end

  def after_failed_work(pipeline) do
    transaction = pipeline.assigns.appsignal_transaction

    Appsignal.Transaction.set_error(
      transaction,
      "Exq job failed with exception",
      pipeline.assigns.error_message,
      System.stacktrace
    )

    Appsignal.Transaction.finish(transaction)
    Appsignal.Transaction.complete(transaction)

    pipeline
  end
end

[tombruijn]

That's amazing @nirev ! Thanks for sharing 👍

[bgentry]

Still works great @nirev, thank you! 🎉 The main caveat is that it has to be placed towards the end of the middleware list, or else some expected pipeline.assigns are not available.

AppSignal team, it'd really be great to include a middleware like this in the library. Even if it's not automatically configured, it's much easier just to add a middleware to my Exq config than to copy the code myself. Plus that way I don't have to feel the burden of maintaining it myself 😉

[bgentry]

I also expanded on the Exq middleware by @nirev to add more detailed job metadata:

def before_work(pipeline) do
    # Start an AppSignal transaction
    transaction =
      Appsignal.Transaction.start(
        Appsignal.Transaction.generate_id(),
        :background_job
      )
      |> Appsignal.Transaction.set_action("Exq/#{pipeline.assigns.worker_module}")
      |> Appsignal.Transaction.set_sample_data(
        "environment",
        metadata_from_job(pipeline.assigns.job)
      )

    assign(pipeline, :appsignal_transaction, transaction)
  end

# skip over code from above


  defp metadata_from_job(%{} = job) do
    %{
      args: job.args,
      class: job.class,
      enqueued_at: iso_datetime_from_unix_float(job.enqueued_at),
      job_id: job.jid,
      queue: job.queue,
      retry: job.retry
    }
  end

  defp iso_datetime_from_unix_float(unix_float) when is_float(unix_float) do
    microseconds_int = trunc(unix_float * 1_000_000.0)

    case DateTime.from_unix(microseconds_int, :microseconds) do
      {:ok, datetime} -> DateTime.to_iso8601(datetime)
      _ -> "unable to parse"
    end
  end

[hosh]

It looks like the next minor version of Absinthe will get Telemetry 0.4 support: absinthe-graphql/absinthe#663

I take it once this is out, we'd just use something similar for pushing Ecto telemetry into appsignal.

[QuinnWilton]

Absinthe v1.5 just came out, with built in telemetry support!

Here's the docs on the events they emit: https://github.com/absinthe-graphql/absinthe/blob/master/guides/telemetry.md

[rossvz]

Any word on seeing absinthe telemetry, now that 1.5 is out?

[lstrzebinczyk]

Seconded, I would love proper native support for absinthe.

[franzejr]

I'd love to see it too!

[Stefano1990]

Proper Absinthe support please.

[haizop]

Absinthe please!

[tombruijn]

I've split off Absinthe support in a new issue: #751
Please subscribe to that issue to receive updates on its progress.

[coladarci]

Apologies for the questionable use of this thread but the google brings me here when trying to figure out why Tesla external calls aren't showing up anywhere despite adding plug Tesla.Middleware.Telemetry - my understanding was the focus on OpenTelemetry would mean these would make their way "for free".. This thread speaks of targeting a 1.2 release and you are past 2.0 now so just curious if I'm missing something here.

Thanks for all the great work!

[RodolfoSilva]

Add support for Absinthe Dataloader, without this we can't know what's going on in the application:

What kind of operation is taking so long to run? We don't know :( .