Build failed when configuring BP_NEW_RELIC_EXT_URI/BP_NEW_RELIC_EXT_SHA256/BP_NEW_RELIC_EXT_VERSION

[zhoufenqin]

When I didn't add BP_NEW_RELIC_EXT_{xxx} during the build, an app can be built successfully.

......
Paketo Buildpack for New Relic 5.1.0
  https://github.com/paketo-buildpacks/new-relic
  Build Configuration:
    $BP_NEW_RELIC_EXT_SHA256      the SHA256 hash of the New Relic extensions archive
    $BP_NEW_RELIC_EXT_STRIP    0  the number of directory components to strip from the New Relic extensions archive
    $BP_NEW_RELIC_EXT_URI         the download location of the New Relic extensions
    $BP_NEW_RELIC_EXT_VERSION     the version of the New Relic extensions
  New Relic Java Agent 7.9.0: Contributing to layer
    Reusing cached download from buildpack
    Copying to /layers/paketo-buildpacks_new-relic/new-relic-java
    Writing env.launch/JAVA_TOOL_OPTIONS.append
    Writing env.launch/JAVA_TOOL_OPTIONS.delim
    Writing env.launch/NEW_RELIC_LOG.default
  Launch Helper: Contributing to layer
    Creating /layers/paketo-buildpacks_new-relic/helper/exec.d/properties
......

The agent shown in container

cnb@app-newre-default-9-547c5d9c8c-99nh4:/layers/paketo-buildpacks_new-relic/new-relic-java$ ls
env.launch  newrelic-agent-7.9.0.jar  newrelic.yml

But when I specify a new relic agent version, even if it is the same version as the buildpack.toml in new-relic buildpacks, the build will be failed, and the build logs show

Build reason(s): CONFIG
CONFIG:
        + env:
        + - name: BP_NEW_RELIC_EXT_URI
        +   value: https://repo1.maven.org/maven2/com/newrelic/agent/java/newrelic-agent/7.9.0/newrelic-agent-7.9.0.jar
        + - name: BP_NEW_RELIC_EXT_VERSION
        +   value: 7.9.0
        + - name: BP_NEW_RELIC_EXT_SHA256
        +   value: 0bfa5a55533fb13b2eda612f9c392c3b8237b766b5379d5256f0c5f657915e7f
        resources:
.....
Paketo Buildpack for New Relic 5.1.0
  https://github.com/paketo-buildpacks/new-relic
  Build Configuration:
    $BP_NEW_RELIC_EXT_SHA256   0bfa5a55533fb13b2eda612f9c392c3b8237b766b5379d5256f0c5f657915e7f                                      the SHA256 hash of the New Relic extensions archive
    $BP_NEW_RELIC_EXT_STRIP    0                                                                                                     the number of directory components to strip from the New Relic extensions archive
    $BP_NEW_RELIC_EXT_URI      https://repo1.maven.org/maven2/com/newrelic/agent/java/newrelic-agent/7.9.0/newrelic-agent-7.9.0.jar  the download location of the New Relic extensions
    $BP_NEW_RELIC_EXT_VERSION  7.9.0                                                                                                 the version of the New Relic extensions
  New Relic Java Agent 7.9.0: Reusing cached layer
  New Relic Extensions 7.9.0: Contributing to layer
    Downloading from https://repo1.maven.org/maven2/com/newrelic/agent/java/newrelic-agent/7.9.0/newrelic-agent-7.9.0.jar
    Verifying checksum
    Expanding to /layers/paketo-buildpacks_new-relic/new-relic-extensions
unable to invoke layer creator
unable to expand extensions
unable to create GZIP reader
gzip: invalid header
ERROR: failed to build: exit status 1

https://github.com/paketo-buildpacks/appdynamics also has this problem, have I misconfigured something, or is there a bug?

Expected Behavior

The configured new-relic version can work

Current Behavior

The configured new-relic version can't work

Possible Solution

Steps to Reproduce

Motivations

[pivotal-david-osullivan]

Thanks @zhoufenqin!

This might be a bug in that it should not assume the extension library is a GZIP archive, however I don't believe the extensions mechanism is intended to be able to override the New Relic Agent jar itself.

The docs describe this functionality as a way to provide support for additional features/libraries via supported modules - see https://docs.newrelic.com/docs/apm/agents/java-agent/instrumentation/extension-additional-instrumentation-modules/#install

I will label this as a bug to track the fix that is required to support extensions as .jar files, since the above doc lists all supported modules as this type.

[dmikusa]

I don't believe the extensions mechanism is intended to be able to override the New Relic Agent jar itself.

100%

The docs describe this functionality as a way to provide support for additional features/libraries via supported modules - see https://docs.newrelic.com/docs/apm/agents/java-agent/instrumentation/extension-additional-instrumentation-modules/#install
https://github.com/paketo-buildpacks/appdynamics also has this problem, have I misconfigured something, or is there a bug?

Sorry, this is not your fault, it is not documented so it's an issue on our end. This functionality is kind of similar to external Tomcat config. The buildpack expects a tar of the New Relic extensions (App Dynamics extensions work the same way). It extracts that out as a layer in the container, then it sets -Dnewrelic.config.extensions.dir which allows the New Relic agent to find that extension. There is an expectation that the downloaded file is a tar file. What is extracted from the TAR File ends up in the layer and is essentially the root of -Dnewrelic.config.extensions.dir.

I will say that this method of downloading a TAR, extracting, and installing is basically a carry-over from the older Cloud Foundry Java buildpack. I don't personally think this method of providing additional configuration fits well in the K8s world. It requires you bundle and host files on an HTTP server, those are then downloaded and installed at buildtime. It can work, but it's not particularly convenient on K8s. In K8s, it's easier and more flexible to use config maps or volume mounts to bring additional configuration and extensions into the container. That maps better into K8s primitives and it's also more flexible as you can do it at runtime, instead of build time. We don't have any specific plans but I hope we can transition away from using these extension TARs. For now, you are welcome to use them if it suits your purpose and we'll obviously provide adequate transition periods if we do end up making changes.

Having said all that, if you want to override from where the New Relic agent is installed please take a look at dependency mappings. A dependency mapping will allow you to override the location from which a dependency is installed.

[ketronkowski]

@pivotal-david-osullivan Any progress in supporting .jar extensions? I would like to employ it for pulling in https://github.com/newrelic/newrelic-java-kotlin-coroutines release artifacts.