[Help Needed] wait for site finish loading

[huynguyen230892]

Hello, I'm really sorry to bother you, but how can I get the contents of a website after it has finished loading? I've searched everywhere and found nothing. Please help me. :(

[phimage]

not enough information, will close Le mer. 28 juin 2023 à 08:29, huynguyen230892 ***@***.***> a écrit :

…

Hello, I am really sorry to bother you, but how can I get the contents when the website is finish loading? I search everywhere and find nothing. Please help me :( — Reply to this email directly, view it on GitHub <#61>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACDW56AWH6GDIIL72W2ZL23XNPFN7ANCNFSM6AAAAAAZWR2RH4> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[robertlude]

I, too, would like this functionality.

In other headless browsers I have used, there is an option to wait for the page to completely load before scraping or manipulating, including the page's JavaScript updating more "dynamic" pages before querying for information.

A proper signal that says "the page is actually fully loaded and ready to parse" would be really helpful. A lot of web sites use code to immediately load in data, but it's not immediate.

I think other headless browsers do it by detecting when DOM changes really stop after X threshold of time, or maybe detecting that no immediate JavaScript is running, I'm not really sure. But this would be a fantastic feature.

[e-marchand]

I, too, would like this functionality.

In other headless browsers I have used, there is an option to wait for the page to completely load before scraping or manipulating, including the page's JavaScript updating more "dynamic" pages before querying for information.

A proper signal that says "the page is actually fully loaded and ready to parse" would be really helpful. A lot of web sites use code to immediately load in data, but it's not immediate.

I think other headless browsers do it by detecting when DOM changes really stop after X threshold of time, or maybe detecting that no immediate JavaScript is running, I'm not really sure. But this would be a fantastic feature.

Could be implemented here if api allow it
https://github.com/phimage/Erik/blob/92c7081ce6f5b4416cc238791e14960bc44fdbf7/Sources/LayoutEngine.swift#L65C17-L65C34

my way to do it is after loading wait for a specific dom element created, or some js variable intialized, and if not I want and retry to get it
So it's depend on each page. I do it a lot in browser "userscript"

[tkantor81]

Hello, I am also receiving different responses, because page have JS and it is not fully loaded. So could you give me a clue how to use this library? How to wait for page is loaded? Thank you