Haketilo

09/10/2021

10:07 PM Feature #90: Make the 0.1 release

I started documenting Hachette usage. I uploaded the screenshots I made, so if you happen to come there while I sleep... koszko

05:15 PM Feature #90: Make the 0.1 release

"allow" option, CSP behavior and URL length limits are now on `koszko` branch koszko

08:49 PM Feature #13: find some way not to require each chrome user to modify manifest.json

I found details regarding the CRX file format:
http://www.dre.vanderbilt.edu/~schmidt/android/android-4.0/external/c... koszko

05:47 PM Support #75: ServiceWorkers

I added unregistering code on `koszko` branch. It needs testing koszko

05:34 PM Feature #32: Process HTML files in data: URLs instead of just blocking them

It might be possible to utilize this API:
https://developer.mozilla.org/en-US/docs/Web/API/Navigator/registerProto... koszko

05:07 PM Feature #88: [Roadmap 6][Milestone] Allow payloads to also specify CSP rules that should be used instead of the original ones served by page

As this is somehow related, I'll write an update regarding our recent CSP change (where we are no longer modifying ex... koszko

09/09/2021

05:35 PM Support #75: ServiceWorkers

jahoti wrote:
> perhaps we could present some version of [this information](https://www.ghacks.net/2016/03/02/manage... koszko

01:52 PM Feature #66: Write tests

Have you considered using UML (no, not that diagraming language, I mean User Mode Linux) to run tests inside? I'm sug... koszko

12:51 PM Feature #34 (Closed): improve CSP injection blocking

Can be considered done as part of #78 koszko

12:15 PM Support #78: Investigate into how browsers handle files that are not HTML

> I am going to continue with this tomorrow. Btw, I realized some mistakes (including being unaware of what I just de... koszko

09/07/2021

10:31 PM Support #78: Investigate into how browsers handle files that are not HTML

I now realize what is the problem with all XMLs, including SVGs. Any XML can include elements from other XML namespac... koszko

10:52 AM Support #78: Investigate into how browsers handle files that are not HTML

I suppose it's the same as with SVG, although I need to make sure it's really the case koszko

09/06/2021

12:05 AM Feature #90: Make the 0.1 release

That leaves me with 4, I suppose, which is probably just as well; the current (limited) state of the testing suite is... jahoti

08:51 PM Feature #90: Make the 0.1 release

`3`. is now ready, as noted in #78 koszko

04:54 PM Feature #90: Make the 0.1 release

Instead of implementing 2. as specified in the description, I did something else. Effect is as wanted - build.sh gene... koszko

02:39 PM Feature #90 (Closed): Make the 0.1 release

Right now what we have left to do is:
1. ~~Make it impossible to check "allow" option for page with payload, as sugg... koszko

12:02 AM Support #78: Investigate into how browsers handle files that are not HTML

> I came up with code that should do with blocking for now. On koszko branch. Could do with more testing
Doing thi... jahoti

08:49 PM Support #78: Investigate into how browsers handle files that are not HTML

I came up with code that should do with blocking for now. On `koszko` branch. Could do with more testing koszko

06:55 PM Support #78: Investigate into how browsers handle files that are not HTML

Now we know why NoScript included special code for SVGs and XMLs:
https://developer.mozilla.org/en-US/docs/Web/SVG/E... koszko

02:57 PM Support #78: Investigate into how browsers handle files that are not HTML

> > While server might not be able to make user's browser execute scripts in a non-HTML page, we are. Should we restr... koszko

11:48 AM Support #78: Investigate into how browsers handle files that are not HTML

> While server might not be able to make user's browser execute scripts in a non-HTML page, we are. Should we restrai... jahoti

09:56 AM Support #78: Investigate into how browsers handle files that are not HTML

> > Now it would make sense to make content script not try to inject payload if document.contentType is not of proper... koszko

12:00 AM Feature #13: find some way not to require each chrome user to modify manifest.json

> The "key" manifest property was required by Chromium to be an actual key in PEM format
Thank you for explaining!... jahoti

04:53 PM Feature #13: find some way not to require each chrome user to modify manifest.json

> > Wouldn't that still require each user to build the extension themselves?
>
> It would. It would just be less h... koszko

11:45 PM Feature #28: split options_main.js into several smaller files

> Right now I can quickly make this little change you suggested since I already know that code. And you could instead... jahoti

02:20 PM Feature #28: split options_main.js into several smaller files

Discussion moved from #15
>>>>Since long-term we're not really planning to allow our scripts to run together with ... koszko

11:41 AM Feature #15: make sure page's own csp in <head> doesn't block our scripts

> Keep in mind, however, options_main.js is currntly the most tangled script file in Hachette
Perhaps I'll start ... jahoti

10:24 AM Feature #15: make sure page's own csp in <head> doesn't block our scripts

> > Since long-term we're not really planning to allow our scripts to run together with page's ones (i.e. "allow site... koszko

11:41 AM Feature #7: [Roadmap 34][Milestone] find some convenient way to automatically re-add intrinsic javascript

> You mean re-allowing the actual intrinsics as they appear on the page they came with?
I did, having not really t... jahoti

10:37 AM Feature #7: [Roadmap 34][Milestone] find some convenient way to automatically re-add intrinsic javascript

> A hacky and flawed solution to this might be to simply scan the nodes and rever event-handler attribute blocking.
... koszko

11:29 AM Support #75: ServiceWorkers

> Unfortunately, it seems a page reload is required for this to take effect.
>
> Additionally, is there a way servi... jahoti

10:50 AM Support #75: ServiceWorkers

Unfortunately, it seems a page reload is required for this to take effect.
Additionally, is there a way service w... koszko

09:51 AM Feature #70: [Roadmap 7][Milestone] Add facility to replace sites' original HTML with custom one

Together with this, we could allow scripts to access the original, raw HTML code of the page in question. I am mentio... koszko

09/05/2021

10:50 AM Feature #26 (Closed): besides blocking scripts through csp, also block connections that needlessly fetch those scripts

Tentatively closed; the bug is no longer reproduceable on IceCat, LibreWolf, or Ungoogled Chromium (version to be not... jahoti

04:38 AM Feature #26: besides blocking scripts through csp, also block connections that needlessly fetch those scripts

I'll check if this is even an issue either today or in the next few days (if live scripts are never added to the acti... jahoti

05:12 AM Support #75: ServiceWorkers

The following script will deregister all service workers in a page (courtesy of <https://love2dev.com/blog/how-to-uni... jahoti

04:52 AM Feature #14: test with more browser forks (Abrowser, newest Parabola IceWeasel, LibreWolf)

> I am not so sure. Official mobile releases stopped at 38.6.0.
That complicates things. I'll see if I can find w... jahoti

04:50 AM Feature #7: [Roadmap 34][Milestone] find some convenient way to automatically re-add intrinsic javascript

A hacky and flawed solution to this might be to simply scan the nodes and rever event-handler attribute blocking. jahoti

04:44 AM Feature #16 (Closed): create a repository to host scripts

See project:Hydrilla and the instance at [[https://api-demo.hachette-hydrilla.org]]. jahoti

04:29 AM Feature #66: Write tests

The basic infrastructure to support creating a "virtual network" in now in the `jahoti` branch, and can be used on it... jahoti

02:20 AM Feature #15: make sure page's own csp in <head> doesn't block our scripts

> If any part of Hachette can be considered infrastructure trap, it's surely this CSP stuff. Having already done so m... jahoti

09/04/2021

01:40 AM Support #78: Investigate into how browsers handle files that are not HTML

> Btw, I noticed cookies don't work on non-HTML pages. This doesn't seem to be an issue as long as we assume the conc... jahoti

09:05 PM Bug #89 (Closed): Restore, to the extent necessary, the script sanitizing functionality

Merged to `master` koszko

08:50 PM Bug #89 (Closed): Restore, to the extent necessary, the script sanitizing functionality

Sanitizing of `<script>` tags was recently dropped because it seemed sufficient to rely on CSP rules being injected. ... koszko

07:36 PM Feature #88 (New): [Roadmap 6][Milestone] Allow payloads to also specify CSP rules that should be used instead of the original ones served by page

Note that this concerns CSP rules other than those for scripts. For scripts we always use a nonce
[Roadmap](/proje... koszko

07:33 PM Bug #65 (Closed): When a site fails to load, for example due to its IP address not being found, the injected value with settings remains in the URL

Merged to `master` koszko

12:36 PM Feature #11 (Closed): add some nice styling to settings page

Merged to `master` koszko

12:35 PM Feature #15 (Closed): make sure page's own csp in <head> doesn't block our scripts

Merged to `master` koszko

12:35 PM Feature #23 (Closed): also implement support for whitelisting of non-https urls

Merged to `master` koszko

12:34 PM Feature #31 (Closed): add an option to disable script blocking globally

Merged to `master` koszko

12:34 PM Feature #49 (Closed): add some nice styling to popup

Merged to `master` koszko

09/03/2021

07:19 PM Support #78: Investigate into how browsers handle files that are not HTML

Modified StreamFilter code is now on `koszko-rethinked-meta-sanitizing`. The `policy` object now also contains inform... koszko

12:36 PM Support #78: Investigate into how browsers handle files that are not HTML

No, since under Chromium I've never actually seen our "document_start" content scripts start with DOM partially or fu... koszko

12:19 PM Support #78: Investigate into how browsers handle files that are not HTML

> Perhpas we could instead, in StreamFilter, just try running DOMParser over the first chunk of data and examining th... jahoti

11:17 AM Support #78: Investigate into how browsers handle files that are not HTML

Heuristics. That's bad... For us.
Even mere parsing of response headers is already risky because of some subtletie... koszko

10:21 AM Support #78: Investigate into how browsers handle files that are not HTML

According to <https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types#mime_sniffing>:
> In th... jahoti

12:52 PM Feature #85: Make Haketilo use the same format as Hydrilla for import and export of settings

jahoti wrote:
> Is the Hydrilla format stable? If not, is it worth waiting for that first or should this be easy eno... koszko

12:27 PM Feature #85: Make Haketilo use the same format as Hydrilla for import and export of settings

Is the Hydrilla format stable? If not, is it worth waiting for that first or should this be easy enough to do now? jahoti

12:50 PM Feature #14: test with more browser forks (Abrowser, newest Parabola IceWeasel, LibreWolf)

jahoti wrote:
> I suspect IceCat can be built on FSDG-compliant distros.
I am not so sure. Official mobile releas... koszko

12:25 PM Feature #14: test with more browser forks (Abrowser, newest Parabola IceWeasel, LibreWolf)

I suspect IceCat can be built on FSDG-compliant distros. Ungoogled Chromium might have that option, yet it's pointles... jahoti

12:23 PM Feature #15: make sure page's own csp in <head> doesn't block our scripts

If any part of Hachette can be considered infrastructure trap, it's surely this CSP stuff. Having already done so muc... koszko

11:59 AM Feature #15: make sure page's own csp in <head> doesn't block our scripts

> So we still need workarounds under Mozilla :/
How easy life would be if everything worked reasonably well!
> ... jahoti

10:32 AM Feature #15: make sure page's own csp in <head> doesn't block our scripts

> - On Chromium, nodes injected by content scripts are CSP-exempt, meaning CSP filtering is unnecessary (albeit harml... koszko

09:51 AM Feature #15: make sure page's own csp in <head> doesn't block our scripts

Sorry I didn't see your question! I distracted myself with researching around the topic (in the midst of general busy... jahoti

12:18 PM Feature #83: Also add ability to selectively block other types of content (e.g. fonts)

> I am not entirely sure the actual fetching of resources is also prevented by CSP. What I am sure would work, though... jahoti

11:44 AM Feature #83: Also add ability to selectively block other types of content (e.g. fonts)

I am not entirely sure the actual fetching of resources is also prevented by CSP. What I am sure would work, though, ... koszko

10:16 AM Feature #83: Also add ability to selectively block other types of content (e.g. fonts)

To summarise from the [full list of CSP directives](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content... jahoti

09/02/2021

09:37 PM Feature #15: make sure page's own csp in <head> doesn't block our scripts

I pushed some code for this to new `koszko-rethinked-meta-sanitizing` branch. I am not yet 100% sure this will work. ... koszko

09:05 PM Feature #85 (In Progress): Make Haketilo use the same format as Hydrilla for import and export of settings

I just realized it should be possible to access entire directories:
https://developer.mozilla.org/en-US/docs/Web/API... koszko

06:37 PM Feature #31 (Resolved): add an option to disable script blocking globally

On `koszko-smuggle-policy` branch koszko

09/01/2021

02:18 PM Feature #11: add some nice styling to settings page

Import dialog is now also styled. All that's left is merging to `master` koszko

02:18 PM Feature #49: add some nice styling to popup

Install dialog is now also styled. All that's left is merging to `master` koszko

11:48 AM Feature #49: add some nice styling to popup

This is now also on `koszko-smuggle-policy` branch, except for the install dialog koszko

01:49 PM Feature #15: make sure page's own csp in <head> doesn't block our scripts

Did you have any success? koszko

08/31/2021

01:36 PM Feature #83 (New): Also add ability to selectively block other types of content (e.g. fonts)

Google uses fonts sites load from its servers for snooping. Blocking them causes relatively little issues (compared t... koszko

01:32 PM Feature #11: add some nice styling to settings page

Forgot to mention: this has been ready (except for settings import window) on `koszko-smuggle-policy` branch since ye... koszko

08/30/2021

12:13 PM Feature #14: test with more browser forks (Abrowser, newest Parabola IceWeasel, LibreWolf)

## Mobile version considerations
I don't have any Android device to test on, although it might be possible to use ... koszko

12:02 PM Feature #82 (New): Style extension's pages for mobile usage

koszko

12:00 PM Feature #80 (New): Make Haketilo work with mobile versions of browsers

This is mostly the matter of apropriately styling extension's pages. Unfortunately, a libre mobile browser to test on... koszko

08/28/2021

08:56 AM Support #78: Investigate into how browsers handle files that are not HTML

> As for making sure we only filter relevant data, do any browsers try to guess mime types?
By guessing you mean a... koszko

03:00 AM Support #78: Investigate into how browsers handle files that are not HTML

For the second point at least, I know NoScript operates on XML (and will check uBlock Origin for similar behavior). W... jahoti

08:48 AM Feature #13: find some way not to require each chrome user to modify manifest.json

> Wouldn't that still require each user to build the extension themselves?
It would. It would just be less hacky t... koszko

02:54 AM Feature #13: find some way not to require each chrome user to modify manifest.json

> Using a synchronous AJAX call from the content script might allow us to use a bundled file as a secret
Wouldn't ... jahoti

08/27/2021

10:58 AM Feature #79 (Closed): Improve the build script by using awk

Since writing `build.sh` I realized some things could be done a lot easier using awk koszko

10:56 AM Feature #23 (Resolved): also implement support for whitelisting of non-https urls

koszko

10:55 AM Feature #23: also implement support for whitelisting of non-https urls

`ftp://` is now also ready and pushed to this temporary branch. Changes will be merged together with completed Featur... koszko

10:12 AM Feature #23: also implement support for whitelisting of non-https urls

Support for the `file://` protocol is now on the `koszko-smuggle-policy` branch. I re-used the temporarily-unused app... koszko

10:32 AM Support #78 (Rejected): Investigate into how browsers handle files that are not HTML

Our tampering with HTML pages, including rewriting parts of them using the StreamFilter API, might cause problems whe... koszko

10:26 AM Feature #77: Check LibreJS is compatible with this extension.

# History before copying
koszko wrote:
> I assume by compatibility you mean the ability to run side-by-side with ... koszko

10:26 AM Feature #77 (Closed): Check LibreJS is compatible with this extension.

Many swfreedom supporters prefer LibreJS' blocking mechanism. As there's good reason to expect compatability, it woul... koszko

10:07 AM Feature #13: find some way not to require each chrome user to modify manifest.json

Using a synchronous AJAX call from the content script might allow us to use a bundled file as a secret koszko

08/26/2021

03:55 PM Bug #65: When a site fails to load, for example due to its IP address not being found, the injected value with settings remains in the URL

> We could use webRequest to remove our cookies from request headers in case they happen to get there
Committed to... koszko

09:54 AM Feature #15: make sure page's own csp in <head> doesn't block our scripts

> I'll try and do this today.
If it turns out to work, you should be able to use StreamFilter code from 6b53d6c840... koszko

08/25/2021

12:07 AM Feature #15: make sure page's own csp in <head> doesn't block our scripts

> I instead implemented a hacky way that uses multiple invocations of DOMParser to find where page's <head> ends so t... jahoti

09:55 PM Feature #15: make sure page's own csp in <head> doesn't block our scripts

I instead implemented a hacky way that uses multiple invocations of DOMParser to find where page's `<head>` ends so t... koszko

08/23/2021

11:56 PM Feature #15: make sure page's own csp in <head> doesn't block our scripts

> I hate web browsers. It all grows waaaay more complex than I expected.
Which then wastes half one's energy remem... jahoti

06:18 PM Feature #15: make sure page's own csp in <head> doesn't block our scripts

> I'll investigate possible workarounds for Mozilla.
I did.
* We can make a HTML on-the-fly "parser" by creating ... koszko

11:14 AM Feature #15: make sure page's own csp in <head> doesn't block our scripts

The code that uses StreamFilter is now on my branch. The remaining issues are worth mentioning:
1. Under Chromium ... koszko

11:17 AM Bug #65: When a site fails to load, for example due to its IP address not being found, the injected value with settings remains in the URL

> > Now there is real danger cookie will not get deleted for some reason and will get sent to server. Anyway, I think... koszko

08/21/2021

08:55 PM Feature #15: make sure page's own csp in <head> doesn't block our scripts

Had some issues again (document created with `DOMParser` can be written to under Chromium but not under IceCat 60). A... koszko

08/20/2021

01:04 PM Feature #15 (In Progress): make sure page's own csp in <head> doesn't block our scripts

> Maybe the *extension* should have been named Hydrilla- whenever one path gets cut off, two more grow in its place :... koszko

11:06 AM Bug #65: When a site fails to load, for example due to its IP address not being found, the injected value with settings remains in the URL

Thanks for pointing out. I'll fix it together with some bigger changes for issue 15 https://hachettebugs.koszko.org/i... koszko

07:20 AM Bug #65: When a site fails to load, for example due to its IP address not being found, the injected value with settings remains in the URL

> EDIT: Newest commit on my branch restores compatibility with IceCat 60. Testing on other browsers still welcome :)
... jahoti

08/19/2021

01:49 AM Bug #65: When a site fails to load, for example due to its IP address not being found, the injected value with settings remains in the URL

> Now there is real danger cookie will not get deleted for some reason and will get sent to server. Anyway, I think t... jahoti

08/18/2021

08:57 PM Support #75 (Rejected): ServiceWorkers

Investigate into Service Workers. Find out if some additional measures need to be taken against them koszko

06:10 PM Bug #65: When a site fails to load, for example due to its IP address not being found, the injected value with settings remains in the URL

> Sounds like a winner (and much safer than dealing with the URL fragment)!
It is indeed way more convenient. Safe... koszko

08/17/2021

01:19 AM Feature #15: make sure page's own csp in <head> doesn't block our scripts

> Sad that I already wrote the toughest parts of that one :/
*Sigh* :/
At least you've got something to start w... jahoti

07:50 PM Feature #15: make sure page's own csp in <head> doesn't block our scripts

UPDATE
Bad news (but read on!) - we cannot use `document.write()` this way from content script nor from any `<script... koszko

01:13 AM Bug #65: When a site fails to load, for example due to its IP address not being found, the injected value with settings remains in the URL

Sounds like a winner (and much safer than dealing with the URL fragment)! That said, is there any way to deal with a ... jahoti

07:41 PM Bug #65: When a site fails to load, for example due to its IP address not being found, the injected value with settings remains in the URL

We should investigate if we can use `Set-Cookie` header instead of URL for policy smuggling
EDIT: Looks very promi... koszko

08/16/2021

11:24 AM Feature #15: make sure page's own csp in <head> doesn't block our scripts

> I think it will take me a little while to understand exactly what magic you've pulled :).
All that's needed is t... koszko

08/15/2021

12:47 PM Feature #15: make sure page's own csp in <head> doesn't block our scripts

That is genuine genius- I think it will take me a little while to understand exactly what magic you've pulled :). jahoti

09:08 AM Bug #53: Interference with existing CSP headers

No- feel free to delete the csp-PoC branch. jahoti

08/14/2021

01:03 PM Feature #15: make sure page's own csp in <head> doesn't block our scripts

> A spurious `</script>` at the beginning of the document could cause serious issues with my method. There are, howev... koszko

09:42 AM Feature #15: make sure page's own csp in <head> doesn't block our scripts

> > So, in the end, this will not only allow us to modify the offending csp rules but also impose script-blocking and... koszko

03:10 AM Feature #15: make sure page's own csp in <head> doesn't block our scripts

> I started looking for a solution and found out a very good thing. In Chromium at `document_start` we could stop unw... jahoti

10:21 AM Feature #17 (Closed): enable the extension to automatically fetch script substitutes from the repo

Tested with Google Drive fixes. Closing. Documentation will be added at some point. koszko

10:10 AM Bug #53 (Closed): Interference with existing CSP headers

Merged to master. You no longer need the `csp-PoC` branch, do you? koszko

02:25 AM Bug #53: Interference with existing CSP headers

> From what I tested today and yesterday[1], the experimental code in csp-PoC that's responsible for removing the CSP... jahoti

08/13/2021

06:03 PM Feature #29 (Closed): validate settings data on import

I did it as part of https://hachettebugs.koszko.org/issues/17
For now, it's on `koszko` branch koszko

05:23 PM Bug #53: Interference with existing CSP headers

From what I tested today and yesterday[1], the experimental code in csp-PoC that's responsible for removing the CSP `... koszko

05:13 PM Feature #15: make sure page's own csp in <head> doesn't block our scripts

I see you tried to remove the offending `<meta>` csp tags in the csp-PoC branch. Unfortunately, to the extent I teste... koszko

12:51 PM Feature #34: improve CSP injection blocking

Update: we might be able to just inject `<meta>` at the very beginning of the document. Browsers seem to be able to d... koszko