Site fixes

Hi and thanks for posting!

I'll start the response with some bad news: this site uses reCAPTCHAv3. You might recall Google reCAPTCHA challenges having humans select images from a set and stuff like that. It was all reCAPTCHAv2 (or older v1). reCAPTCHAv3 works differently. It monitors person's actions on a site and sends this data to Google to analyze without ever showing any challenge. No chance of getting away without it (unless we reimplement the entire thing).

At some point we definitely want to implement a libre reCAPTCHAv2 script as something like this has already been proven possible. It might also be possible to implement libre reCAPTCHAv3 script and we'd like to do so. However, even the libre reCAPTCHAv3 client we'll eventually come up with might be considered by some to be unacceptably privacy-hostile.

Anyway, any reCAPTCHA-related work is unlikely to be started before we:

• finish Haketilo 1.0 that I am working on all the time
• add mechanism to allow Haketilo scripts to bypass CORS (and maybe also spoof some request headers)

You might want to keep the 100$ for later :)

End of bad news

As to LibreJS compatibility, it is assumed the user would disable LibreJS on the site that is meant to be handled by Haketilo. When we say those 2 extensions are compatible, we mean they won't interfere in some other, more serious ways (e.g. causing redirect loops, deleting HTTP headers the other one relies on, etc.).

You want to write a fix from scratch? I usually start by playing with the page in Developer Tools. You know, the interface you open with Ctrl+Shift+I. Look at page's structure, inspect network traffic, etc. Once you know what the new script should do to make the page work, you use the Developer Tools' Console to run the code while having site's own scripts disabled (either by Haketilo or some other extension). I found it most convenient to only write very short snippets in the Console and put anything bigger in a text editor. I'd then verify the code is working ok by copying it from Emacs to the Console (and reloading the page when necessary). Once you think it's ready, go to Haketilo settings page and add it as a new "script". Then, in "pages" tab, click to add a new "page", choose the new script as its payload and type in the URL pattern (explained here) for which this replacement script should be used. Once these are added, reload the page again to see if everything works.

If you want to make the script usable in Hydrilla, just use the same format as those here. Or, simpler, send it to us to do this and include the script in the repo (this way we'll also maintain it and translate to the new Hydrilla format once we introduce it)

I thought the reCAPTHCAs that relied on normal user activity had a fallback on more traditional (picture) CAPTCHAs, which I thought would mean implementing v2 would be enough in this case (or any case that uses v3).

When writing fixes, you have to run the original, proprietary, scripts in order to see the complete original network traffic, right? Do you use a different browser instance or somehow separate the activity from your normal browsing? I'm not totally sure how browser sandboxing works, but sites can't see the data of other sites, right? So maybe it doesn't matter if I use the same browser instance?