Can I listen CDP events in thirtyfour?

[DCjanus]

We've able to execute cdp command in thirtyfour, but I've not found a way to listen CDP event.

Sometimes, I hope to use fetch.enable to catch all network request in a page. When page send a network request, a requestPaused event is issued, and I should capture this event and send fulfillRequest or failRequest.

Like this: https://stackoverflow.com/questions/59420493/how-to-use-chrome-devtools-protocol-in-selenium-using-python-for-capturing-htt

[stevepryde]

I don't think this is possible in thirtyfour currently. Would you be willing to contribute this feature?

[DCjanus]

Sounds good to me.

[DCjanus]

I am not very familiar with chrome driver special protocol, like /session/{session_id}/goog/cdp/execute, is there any place that I can find it,

[stevepryde]

Have a look here: https://chromedevtools.github.io/devtools-protocol/

Edit: use the navigation on the side to view docs for each command

[stevepryde]

You can also check out other selenium libraries such as the python one to see how they do it

[DCjanus]

It looks like that selenium open a websocket connection to accept CDP event, which means we need to introduce a websocket library to thirtyfour.

Which websocket library would you like to use?

I'm not familiar with this domain, my friends told me tokio-tungstenite is the best chose, but it doesn't support async-std, and there is a fork to support different async runtime.

[stevepryde]

Yeah have a look at https://github.com/sdroege/async-tungstenite which supports both runtimes via feature flags

Edit: just realised that's the one you linked to 😂

[audioXD]

I thought that most libraries just use a proxy to intercept all those requests :p

[stevepryde]

You can use a proxy, however you can use CDP to do it, which I would prefer over a proxy since it's coming from the browser itself. It's more like listening to the browser's network tab

[notdanilo]

Any success on listening the events from a websocket connection? I am trying this with no success yet:

    let response = reqwest::get("http://localhost:9222/json/version")
        .await?
        .json::<HashMap<String, String>>()
        .await?;
    let ws_debugger_url = response.get("webSocketDebuggerUrl").expect("Couldn't get ws debugger url.");

    let (mut socket, _) = connect_async(Url::parse(ws_debugger_url)?).await?;
    
        let dev_tools = ChromeDevTools::new(driver.session());
    let version_info = dev_tools.execute_cdp("Browser.getVersion").await?;
    println!("{:#?}", version_info);

    std::thread::sleep(Duration::from_secs_f64(3.0));
    println!("{:#?}", dev_tools.execute_cdp("HeapProfiler.enable").await?);
    println!("{:#?}", dev_tools.execute_cdp_with_params("HeapProfiler.takeHeapSnapshot", json!({"reportProgress":true,"treatGlobalObjectsAsRoots":true,"captureNumericValue":false})).await?);

    while let Some(Ok(message)) = socket.next().await {
        println!("{}", message);
    }

[notdanilo]

Maybe I am missing the session ID or something that binds the websocket to the client sending the requests?

[notdanilo]

I just found this link, which describes the protocol process pretty well: https://github.com/aslushnikov/getting-started-with-cdp

[audioXD]

@notdanilo I saw that the OP made a project seperate from thirtyfour, chromiumoxide made for CDP. To be honest it looks quite similar (but works only for chrome)

EDIT: Yust realized it was a fork from @mattsse

[notdanilo]

@audioXD I urgently created a CDP client myself because I couldn't get chromiumoxide working. The reason it didnt work was because I was simply misunderstanding how the protocol works and chromiumoxide didn't provide a high level API for what I wanted (getting a memory heap snapshot).

I am still using thirtyfour for collecting information from html although.

[stevepryde]

For anyone else looking for CDP support, start here: https://docs.rs/thirtyfour/latest/thirtyfour/extensions/cdp/struct.ChromeDevTools.html

CDP itself is quite large, so we may not have high-level wrappers around it in the short term, but you can implement it yourself without needing to modify thirtyfour.
CDP docs are here: https://chromedevtools.github.io/devtools-protocol/