Not working on yggtorrent.cool

[NathanIfinity]

Been using the extension forever with the same setting but last couple days it has stopped working but only on that site all the other torrent site work just as they should.

here is the url for a torrent download: https://www3.yggtorrent.cool/engine/download_torrent?id=1172880
link catcher doesn't work and right click add to torrent client either.

use to send it to rutorrent on my seedbox

[bogenpirat]

seems to require a login, so i can't test it. what actually happens when you click the link?

if it just downloads a link, it might be that the link is generated dynamically, after RTA's content script is injected into the website. in that case, it can't register an action on the link. but then, right-clicking and sending it via RTA that way should work. do you see any error messages in the extension's background page? -> chrome://extensions/?id=oabphaconndgibllomdcjbfdghcmenci, developer mode on, background page, then try adding a torrent via context menu from the page

[NathanIfinity]

It actually never worked by clicking on that site always used the right-click menu and addto remote torrent but for a few days now even that doesn't work.

This is what I see in the dev mode

[NathanIfinity]

Vs if I try with iptorrent that works

[bogenpirat]

well, hard to say why exactly the server won't let us fetch the file. might be that some particular header needs to be set. i assume the torrent file downloads fine if you just click on it? if so, it might be worth it to compare two requests:

1. the one made by the extension in that window you have open - opened from the extension's settings (it'll show in the network tab)
2. the one made by chrome when you're just left-clicking a link on the website (f12 while you're on it)

that's typically where you can see differences.

[NathanIfinity]

Yeah if I click on it it downloads fine! I'll look the 2 request and report back
Thanks

[NetixQc]

I'm having the same issue as described by Nathan. Not sure what I could link to help the proccess.

[bogenpirat]

@NathanIfinity i've taken the liberty of deleting that post since its screenshots may have contained some of your authentication data. i saw however that the extension doesn't send a referer field - that might be it. perhaps i can write something together for that in a bit.

[NathanIfinity]

Perfect ! glad it could shed some light, let me know if you need me to test anything

[ImTheEZ3KieL]

Exactly the same issue :) Did you happen to be able to find a workaround?
Thanks all.

[geeknessfr]

Hi, same issue here, I don't know what they changed to their website, but it broke the plugin :(

[campones]

and absolutely no feedback from the admin there... of course

[NathanIfinity]

Not sure if it can help ( because I like torrent adder more) but Torrent Control seems to work fine on Firefox if you need any log I could send them

[NetixQc]

Yggtorrent switched domain again. They went private to ygg.re and the problem is now solved. Remote Torrent Adder works like before.

[bogenpirat]

huh, procrastination clearly solved the issue. can the others confirm that it's fixed? if not, i'll have time on the weekend to finally zip a test build

[NathanIfinity]

I confirm that on the new private domain it works as expected

[NathanIfinity]

It started happening again since yesterday, any chance you had time to implement the fix so it doesn't break everytime they make a change :P

Thanks

[NetixQc]

Can confirm what Nathan said. It's broken again.

[bogenpirat]

alright, i fear i need to ask for a sampling of this again - i had deleted the prior one. make sure to censor any header that looks like a session hash or credentials. in the cookies header, feel free to censor what you wish, but leaving the cookie names might help (format is name1=value1; name2=value2)

well, hard to say why exactly the server won't let us fetch the file. might be that some particular header needs to be set. i assume the torrent file downloads fine if you just click on it? if so, it might be worth it to compare two requests:

1. the one made by the extension in that window you have open - opened from the extension's settings (it'll show in the network tab)
2. the one made by chrome when you're just left-clicking a link on the website (f12 while you're on it)

that's typically where you can see differences.

NB: it's not the referer or origin fields as i originally suspected - i checked and the extension actually already sets those on requests.

[NathanIfinity]

Hopefully this helps and I blocked all the correct info
let me know if you need more or need to test anything

[bogenpirat]

thanks, that's a good start. now could you open a tab to the torrent site, hit f12/network tab, then normally download the torrent file (i presume since RTA doesn't properly register the left-click action, you can just left-click? if not, do ctrl+leftclick) and please show me what the request headers look like for that request (so like images 2+3 from your post)

[NathanIfinity]

let me know if this is ok

[bogenpirat]

at first glance, it seems that RTA isn't sending most of the cookies that the browser does. this should happen though, because RTA's requests towards the server are just executed through the browser which already has all of the cookie data. if this part didn't work, no website requiring a login should work, and i'm certain that they do.

but there's something fishy here - your response headers for the browser request show a content-type of text/html - you wouldn't be getting a torrent file from that response. can you take a look at the "response" tab right above the headers - it should show you a bunch of garbage symbols if it is a torrent file. i'm suspecting you might be seeing some html/javascript code instead.

fwiw, these two problems are completely separate. if my suspicion is correct, i'm not certain that RTA can circumvent this... "behavior" to download the torrent file. let me know what you find.

[NathanIfinity]

yeah in the response tab I have html/java I don't see anything when i click or right click on the torrent file

!DOCTYPE html.txt

[NetixQc]

at first glance, it seems that RTA isn't sending most of the cookies that the browser does. this should happen though, because RTA's requests towards the server are just executed through the browser which already has all of the cookie data. if this part didn't work, no website requiring a login should work, and i'm certain that they do.

but there's something fishy here - your response headers for the browser request show a content-type of text/html - you wouldn't be getting a torrent file from that response. can you take a look at the "response" tab right above the headers - it should show you a bunch of garbage symbols if it is a torrent file. i'm suspecting you might be seeing some html/javascript code instead.

fwiw, these two problems are completely separate. if my suspicion is correct, i'm not certain that RTA can circumvent this... "behavior" to download the torrent file. let me know what you find.

Thanks for taking the time to troubleshoot this issue. Users at Ygg.re are reporting that everything works on Firefox using the extension Torrent Control so not sure if it does something different then RTA.

[bogenpirat]

yeah in the response tab I have html/java I don't see anything when i click or right click on the torrent file

!DOCTYPE html.txt

sorry, this is the response you get when you click on the download link for a torrent file in the browser?

[bogenpirat]

Thanks for taking the time to troubleshoot this issue. Users at Ygg.re are reporting that everything works on Firefox using the extension Torrent Control so not sure if it does something different then RTA.

doesn't seem to be doing anything special, so it may be down to how the browsers are handling things.

[NathanIfinity]

is this what you wanted ?

[bogenpirat]

yep, that's your browser downloading the torrent file.

can you show me the (redacted, again) request headers for that?

[NathanIfinity]

[bogenpirat]

that looks correct. so i think the central issue is that the extension doesn't automatically send the cookies for the site along with the request that gets the torrent file, resulting in a 403. i'm not sure why that is, i presume it's got to do with the configuration of the server. i'm taking a stab in the dark at circumventing that with this build:
remote-torrent-adder.git.zip
please check https://github.com/bogenpirat/remote-torrent-adder/wiki/Try-new-code on how to try it out.

[NathanIfinity]

Installed the new package and tried and seem to be getting the same error

[bogenpirat]

please try this one: remote-torrent-adder.git.zip

it sets 3 new headers in the fetch request, but idk if it will work. please pay attention to the request headers: it should set Sec-Fetch-Dest, Sec-Fetch-Mode, Sec-Fetch-Site with values document, navigate, same-origin. i'm suspecting that chrome only sends cookies when these are set to roughly those values.

[NathanIfinity]

Ok this is the result

the value don't seem to correspond to Document, navigate, same-origin

what i get is

empty, cors, none

let me know if I can get you anything more

[bogenpirat]

alright, seems like we cannot change those in code -> https://developer.mozilla.org/en-US/docs/Glossary/Forbidden_header_name

i'm afraid i'm out of ideas on this one.

[NathanIfinity]

Oh well thanks for trying ! maybe eventually something on the site will change again and it's going to start working again :P

[esperlu3tte]

Hello,

It seems related to the cookie "cf_clearance" from third party CloudFlare which is protecting the website. This cookie is stored using the Chrome's "Independent Partitioned State" cookies sandbox (CHIPS), still experimental. It appears that a fetch from an extension, contrary as one from a page, whether with same-origin or include credentials, doesn't include the partitioned cookies, no matter what I try. I don't know if it's a bug in Chrome or done by design, but that's the reason behind the 403. All the other cookies are well sent:

Here a comparison with a fetch on the page:

Unfortunately, as the "Cookie" header is not editable, the only way I see to solve this would be by doing the fetch through code injection (chrome.tabs.executeScript()).

--

Edit: It appears to work with code injection.
I've quickly modified the getTorrent function as such:

RTA.getTorrent = function(server, url, label, dir, referer, tabId) {
  if (url.substring(0, 7) == "magnet:" || server.rutorrentalwaysurl) {
    RTA.dispatchTorrent(server, url, "", label, dir);
  } else {
    RTA.getTorrentLink = url;
    RTA.getTorrentLinkReferer = referer;

    new Promise((resolve) => {
        const funcId = crypto.randomUUID();
        let funcTimeout;
        const messageListener = (message) => {
          if (message.funcId !== funcId) {
            return;
          }
          chrome.runtime.onMessage.removeListener(messageListener);
          clearTimeout(funcTimeout);
          return resolve(message.response);
        };
        chrome.runtime.onMessage.addListener(messageListener);
        chrome.tabs.executeScript(tabId ?? undefined, {
          code: `
(async function() {
  const message = {
    funcId: "${funcId}"
  };
  try {
    const response = await fetch("${url}", {
      credentials: "include"
    });

    // mangling it as text so it works with the older (xhr-reliant) code.
    // could probably modernize the webui parts at some point.
    const responseBodyBlob = await response.blob();
    const responseBodyArrayBuffer = await responseBodyBlob.arrayBuffer();
    const ui8a = new Uint8Array(responseBodyArrayBuffer);
    const chunksize = 0x8000;
    const chunks = [];
    for (let i = 0; i < ui8a.length; i += chunksize) {
      chunks.push(String.fromCharCode.apply(null, ui8a.subarray(i, i + chunksize)));
    }
    const responseBody = chunks.join("");

    const responseHeaders = {};
    response.headers.forEach((value, name) => {
      responseHeaders[name] = value;
    });

    message.response = {
      body: responseBody,
      headers: responseHeaders,
      ok: response.ok,
      redirected: response.redirected,
      status: response.status,
      statusText: response.statusText,
      type: response.type,
      url: response.url,
    };
  } catch (err) {
    message.error = {
      message: err.message,
      arguments: err.arguments,
      type: err.type,
      name: err.name,
      stack: err.stack
    };
  } finally {
    chrome.runtime.sendMessage(message);
  }
})();
          `
        }, () => {
          funcTimeout = setTimeout(() => {
            chrome.runtime.onMessage.removeListener(messageListener);
            return resolve(null);
          }, 30000)
        });
      })
      .then(RTA.handleFetchError)
      .then(async function(response) {
        var name = "file.torrent";
        if (response.url.match(/\/([^\/]+.torrent)$/)) {
          name = response.url.match(/\/([^\/]+.torrent)$/)[1];
        }

        RTA.dispatchTorrent(server, response.body, name, label, dir);
      })
      .catch(error => {
        RTA.displayResponse("Failure", "Could not download torrent file.\nError: " + error.message, true);
      });
  }
}

Here the full change, if someone want to test:
remote-torrent-adder.git.zip

@bogenpirat, if you are ok with the principle of fetch by code injection, I'm happy to improve/refactor this and to make a PR.

[NathanIfinity]

@esperlu3tte

Version tested and functional ! just needed to update the manifest

before: "version": "1.3.9-test-credentials-with-fetchfn-injection", = doesn't work ( won't add the extension)
Now: "version": "1.3.9", = Works ( adds the extension)

and the torrents gets added to the remote client !

there is just something odd when I look at the logs though, the cookie is alway the same and the request URL as well no matter the file I download, but it does download the correct file ??

for reference this was the torrent that I was downloading: https://www.ygg.re/torrent/filmvid%C3%A9o/film/1186567-hit+man+2023+multi+vfq+2160p+sdr+web-dl+h265-slay3r+tueurs+a+gages

you can see that the ID (1186567) is not the one in the logs which is for another torrent I manually downloaded a few days ago
from another tracker completely? (ex.to)

_UserID=xxxxx; _SessionID=3CAxxxxxx06D2; browser_timezone=America/Toronto; magnetic_name=%5Bext.to%5D%20Vikings%20S01E08%20HDTV%20x264-2HD%20%5Beztv%5D

but it works so not sure what is going on here..

Thanks alot !!!

[NetixQc]

@esperlu3tte

Version tested and functional ! just needed to update the manifest

before: "version": "1.3.9-test-credentials-with-fetchfn-injection", = doesn't work ( won't add the extension) Now: "version": "1.3.9", = Works ( adds the extension)

and the torrents gets added to the remote client !

there is just something odd when I look at the logs though, the cookie is alway the same and the request URL as well no matter the file I download, but it does download the correct file ??

for reference this was the torrent that I was downloading: https://www.ygg.re/torrent/filmvid%C3%A9o/film/1186567-hit+man+2023+multi+vfq+2160p+sdr+web-dl+h265-slay3r+tueurs+a+gages

you can see that the ID (1186567) is not the one in the logs which is for another torrent I manually downloaded a few days ago from another tracker completely? (ex.to)

_UserID=xxxxx; _SessionID=3CAxxxxxx06D2; browser_timezone=America/Toronto; magnetic_name=%5Bext.to%5D%20Vikings%20S01E08%20HDTV%20x264-2HD%20%5Beztv%5D

but it works so not sure what is going on here..

Thanks alot !!!

How did you update the manifest. Doesn't seem to work for me when I press update in the extension tabs. Same error you got the first time.

[bogenpirat]

@esperlu3tte cool find! i'm not a big fan of using your fix generally. it's such a hack and the errors seem to only affect one website for now. i could see it as an opt-in option in the settings, specifically targeted at users of yggtorrent and perhaps future CF-guarded sites. would you be comfortable implementing something to that effect? if not, i can take a shot at it when i find the time for it.

i'm not generally a big fan of this solution/remote code execution. i'm not knowledgable on js security, but this looks like it could be prone to abuse as well as not working on some sites that do funky stuff with their javascript scope. if google's drive to abolish third party cookie access keeps up, this might become the only somewhat reliable solution to grab torrents going forward. in that case, it'd certainly be beneficial to already have an implementation in the codebase.

in any case, good work.

@NathanIfinity
that's a different kind of request you're looking at - there, RTA is sending the torrent to your torrent client. since @esperlu3tte's fix is injecting code directly into the context of the page, the request is also executed in the context of the page. so you can no longer find the RTA-downloads-torrent-file requests in the script's background page context. instead, it will be in the website's context and look pretty similar to the one you see when you just click on the link without using RTA.

[NathanIfinity]

Oh ok awesome ! well it works so thats what is important for now :)

@esperlu3tte
Version tested and functional ! just needed to update the manifest
before: "version": "1.3.9-test-credentials-with-fetchfn-injection", = doesn't work ( won't add the extension) Now: "version": "1.3.9", = Works ( adds the extension)
and the torrents gets added to the remote client !
there is just something odd when I look at the logs though, the cookie is alway the same and the request URL as well no matter the file I download, but it does download the correct file ??

for reference this was the torrent that I was downloading: https://www.ygg.re/torrent/filmvid%C3%A9o/film/1186567-hit+man+2023+multi+vfq+2160p+sdr+web-dl+h265-slay3r+tueurs+a+gages
you can see that the ID (1186567) is not the one in the logs which is for another torrent I manually downloaded a few days ago from another tracker completely? (ex.to)
_UserID=xxxxx; _SessionID=3CAxxxxxx06D2; browser_timezone=America/Toronto; magnetic_name=%5Bext.to%5D%20Vikings%20S01E08%20HDTV%20x264-2HD%20%5Beztv%5D
but it works so not sure what is going on here..
Thanks alot !!!

How did you update the manifest. Doesn't seem to work for me when I press update in the extension tabs. Same error you got the first time.

Juste using Notepad++ and editing the manifest.json were it says version (remove the text)

[NathanIfinity]

@esperlu3tte cool find! i'm not a big fan of using your fix generally. it's such a hack and the errors seem to only affect one website for now. i could see it as an opt-in option in the settings, specifically targeted at users of yggtorrent and perhaps future CF-guarded sites. would you be comfortable implementing something to that effect? if not, i can take a shot at it when i find the time for it.

i'm not generally a big fan of this solution/remote code execution. i'm not knowledgable on js security, but this looks like it could be prone to abuse as well as not working on some sites that do funky stuff with their javascript scope. if google's drive to abolish third party cookie access keeps up, this might become the only somewhat reliable solution to grab torrents going forward. in that case, it'd certainly be beneficial to already have an implementation in the codebase.

in any case, good work.

@NathanIfinity that's a different kind of request you're looking at - there, RTA is sending the torrent to your torrent client. since @esperlu3tte's fix is injecting code directly into the context of the page, the request is also executed in the context of the page. so you can no longer find the RTA-downloads-torrent-file requests in the script's background page context. instead, it will be in the website's context and look pretty similar to the one you see when you just click on the link without using RTA.

Oh I see thats why ! well anyway it works ! thanks everyone for all your time on this weird issue :)

[esperlu3tte]

@esperlu3tte cool find! i'm not a big fan of using your fix generally. it's such a hack and the errors seem to only affect one website for now.

@bogenpirat, I'm not particularly a fan either, it's just the only way I see, at the moment, to solve the issue with yggtorrent.

i could see it as an opt-in option in the settings, specifically targeted at users of yggtorrent and perhaps future CF-guarded sites. would you be comfortable implementing something to that effect? if not, i can take a shot at it when i find the time for it.

Good idea. I can probably spend some time on it during next week.
Do you plan on staying with v2 manifest? Or a migration to v3 is planned at some point? I'm asking because there are changes in the APIs between v2 and v3, and v3 scripting.executeScript() is natively supporting promises which would remove the need to use runtime.onMessage() & runtime.sendMessage().

This looks like it could be prone to abuse as well as not working on some sites that do funky stuff with their javascript scope.

I didn't do it in the POC fix, but the torrent download URL needs to be well sanitized to prevent malicious code injection. If you see other threats, please let me know.
About sites with possibly strange js scope, do you think they'd mess up the window object enough to affect the native fetch function? If that happens, I imagine it can be advised at that moment.

[bogenpirat]

Do you plan on staying with v2 manifest? Or a migration to v3 is planned at some point? I'm asking because there are changes in the APIs between v2 and v3, and v3 scripting.executeScript() is natively supporting promises which would remove the need to use runtime.onMessage() & runtime.sendMessage().

it's not so much planned as it is in the back of my head because they'll eventually disable v2 extensions and i'll have no choice but to migrate to still have it running. iirc there were some changes in functionality wrt header manipulation(?), which is a problem and will likely make migrating a bit of a bigger deal. so naturally, i'm procrastinating on that, which has been a successful strategy since google does the same thanks to the adblock uproar.

if you invest time at all, feel free to invest as little as possible and stay within v2. if the api (executeScript) still exists, i can still switch to promises when it's no longer avoidable.

do you think they'd mess up the window object enough to affect the native fetch function?

i've seen exactly that happen. iirc they just were replacing fetch with an implementation of XHR because at that time you could do some thing with XHR that you couldn't yet with fetch... can't remember what. wasn't a torrent site either. but i trust js devs to be extremely adventurous even today.

[ghost58400]

Hello, any news on this issue ?
Do you plan to implement this fix (as an option) ?