infinitely looped path for my jobs app

[prod-feng]

Hello,

I found there's might be a bug that handles the path, when user uses "Job Composer-> From Specified Path". By dafault OnDemand will use $HOME as the DATAROOT for "myjobs" app. So when user use $HOME(like ~/) as the "Source path" for the "From Specified Path), it will cause  infinitely looped path of ~/ondemand/data folder rsynced. OOD only checks the size of the "Source folder", and uses "du" command timeout fo safe copy check, which can not absolutely avoid this situation.

Th finitely looped path rsynced looks like the following:

/home/user1/ondemand/data/sys/myjobs/projects/default/2/ondemand/data/sys/myjobs/projects/default/2/ondemand/data/sys/myjobs/projects/default/2/ondemand/data/sys/myjobs/projects/default/2/ondemand/data/sys/myjobs/projects/default/2/ondemand/data/sys/myjobs/projects/default/2/ondemand/data/sys/myjobs/projects/default/2/ondemand/data/sys/myjobs/projects/default/2/ondemand/data/sys/myjobs/projects/default/2/ondemand/data/sys/myjobs/projects/default/2/ondemand/data/sys/myjobs/projects/default/2/ondemand/data/sys/myjobs/projects/default/2/ondemand/data/sys/myjobs/projects/default/2/ondemand/data/sys/myjobs/projects/default/2/ondemand/data/sys/myjobs/projects/default/2/ondemand/data/sys/myjobs/projects/default/2/ondemand/data/sys/myjobs/projects/default/2/ondemand/data/sys/myjobs/projects/default/2/ondemand/data/sys/myjobs/projects/default/2/ondemand/data/sys/myjobs/projects/default/2/

I have prepared a solution for this issue, by adding 3 lines of codes in app/models/filesystem.rb, to block the use of $HOME by users and give warning message:

 #
 # Exclude user's $HOME folder, avoid infinitely looped path rsynced, when the the DATAROOT is not set, or set to ~/ondemand/data/sys/myjobs/
 #
 if ENV['HOME'].to_s == Pathname.new(path).expand_path.realpath.to_s
    return false, "Please DO NOT use your HOME folder as your Specified Source path! Try to use other paths, sub-paths, like ~/mywork, etc."
 end
 # FIXME: consider using http://ruby-doc.org/stdlib-2.2.0/libdoc/timeout/rdoc/Timeout.html

The patching codes can be found here in my fork of OnDemand:

prod-feng@b03f091

Best,

Feng

[prod-feng]

Follow up:

Added some more codes to handle more general situation when the DATAOOT is set.

#Check if the Source path is a subfolder of the OOD_DATAROOT. If so, stop it.

    if Pathname.new(ENV['OOD_DATAROOT']).fnmatch?(File.join(path,'**'))
       return false, "Please DO NOT use #{path} as the Source path! This will cause infinitive loop of files/paths copying when OOD stages the folder for your job. Please choose other path!"
    end

Please see the link here : cf4cb9b

[johrstrom]

Thanks for the ticket and the followup. I'll take a look at this this week.

[johrstrom]

Hey, I'm just circling back to this to get it fixed in the next release. I can't however replicate the issue. If I specify ~ or ~/ it successfully times out and in-fact when the next page is rendered it translates ~ to my actual real path.

Here I have a screenshot illustrating the same with my network tab open to confirm I sent ~ as a parameter.

[johrstrom]

Is this still an issue @prod-feng? I can't replicate, can you? If so, what version of OnDemand can you replicate this with? (I tried 3.1.x and the master branch).

[prod-feng]

Hi Jeff, It is 3.0.3. Rocky Linux 8.8, kernel 4.18.0 Best, Feng

…

On Fri, Nov 8, 2024 at 11:39 AM Jeff Ohrstrom ***@***.***> wrote: Is this still an issue @prod-feng <https://github.com/prod-feng>? I can't replicate, can you? If so, what version of OnDemand can you replicate this with? (I tried 3.1.x and the master branch). — Reply to this email directly, view it on GitHub <#3426 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABVLAZP6APKXA3CTPCJYXADZ7TSNXAVCNFSM6AAAAABQBUMZ42VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINRVGIZTMOJUGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[prod-feng]

Jeff, The reason you could not repeat might be because your HOME has huge data sets there, and it caused the timeout... Best, Feng

…

On Fri, Nov 8, 2024 at 12:55 PM Feng Zhang ***@***.***> wrote: Hi Jeff, It is 3.0.3. Rocky Linux 8.8, kernel 4.18.0 Best, Feng On Fri, Nov 8, 2024 at 11:39 AM Jeff Ohrstrom ***@***.***> wrote: > Is this still an issue @prod-feng <https://github.com/prod-feng>? I > can't replicate, can you? If so, what version of OnDemand can you replicate > this with? (I tried 3.1.x and the master branch). > > — > Reply to this email directly, view it on GitHub > <#3426 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/ABVLAZP6APKXA3CTPCJYXADZ7TSNXAVCNFSM6AAAAABQBUMZ42VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINRVGIZTMOJUGE> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> >

[prod-feng]

to repeat the issue, you might need to use a different user account with a small HOME folder, or, increase the timeout threshold. Best, Feng

…

On Fri, Nov 8, 2024 at 1:01 PM Feng Zhang ***@***.***> wrote: Jeff, The reason you could not repeat might be because your HOME has huge data sets there, and it caused the timeout... Best, Feng On Fri, Nov 8, 2024 at 12:55 PM Feng Zhang ***@***.***> wrote: > Hi Jeff, > > It is 3.0.3. Rocky Linux 8.8, kernel 4.18.0 > > Best, > > Feng > > > On Fri, Nov 8, 2024 at 11:39 AM Jeff Ohrstrom ***@***.***> > wrote: > >> Is this still an issue @prod-feng <https://github.com/prod-feng>? I >> can't replicate, can you? If so, what version of OnDemand can you replicate >> this with? (I tried 3.1.x and the master branch). >> >> — >> Reply to this email directly, view it on GitHub >> <#3426 (comment)>, >> or unsubscribe >> <https://github.com/notifications/unsubscribe-auth/ABVLAZP6APKXA3CTPCJYXADZ7TSNXAVCNFSM6AAAAABQBUMZ42VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINRVGIZTMOJUGE> >> . >> You are receiving this because you were mentioned.Message ID: >> ***@***.***> >> >