Issue with data-orch.list and URLs Containing Parameters in Apache Hop

[oussamachaouachii]

Hello,

I’m currently working with Apache Hop and facing an issue with the data-orch.list file. Here’s the scenario:

I override the data-orch.list file with the path of the pipeline I want to open in Apache Hop.
When the URL of the .hpl file does not contain parameters, everything works perfectly.
However, if the URL contains parameters, I encounter the following error:

For example:
URL without parameters (works): http://domain/path/to/pipeline.hpl
URL with parameters (fails): http://path/to/pipeline.hpl?param1=value1&param2=value2
Has anyone else encountered this issue? Is there a workaround or a specific configuration needed to handle URLs with parameters?

Thank you in advance for your help!

[hansva]

Interesting case. I have not seen anyone try this before.

Does the URL with parameters work when you use "open file" and paste the path directly in the filepath box?
And if it doesn't with some luck it will give us a stacktrace.

If there's an easy way for us to reproduce this using s3 or another blob storage service, you could make a ticket with the reproduction path. I would not classify it as a bug but as a feature because it falls a bit in the non-normal use category.

[oussamachaouachii]

When I use "open file" and paste the url, it surprisingly removes one of the "/"s in front of "http:" and can't access it .

Steps to reproduce:
Successful File Reading

1. Having an hpl file stored in minio. The bucket needs to be public.
2. Adding the file url ( without any parameter) in data-orch.list and checking hop ui. The file will be charged successfully.

Failed

1. Make the bucket private, and click on share file and generate a dynamic url from minio ui
2. Adding the file url ( with the expiration parameters) in data-orch.list and checking hop ui. There will the error provided in the description of this discussion.

[bamaer]

Afaik, the data-orch.list is only used to keep track of the opened files per project and doesn't have any functionality to deal with parameters, variables or anything but the basic URL. That said, it is an interesting approach. Feel free to create a feature request so we can keep track of this discussion and take a closer look.