Haketilo

09/25/2021

03:49 AM Feature #94 (Rejected): Add support for extension auto-updating

See
https://developer.chrome.com/docs/extensions/mv3/linux_hosting/#update
https://extensionworkshop.com/documentat... jahoti

09/18/2021

06:19 AM Feature #90 (Closed): Make the 0.1 release

"andyprough" has offered some outlets for spreading the news at <https://trisquel.info/en/forum/announcing-haketilo-0... jahoti

09/15/2021

02:45 PM Feature #90: Make the 0.1 release

> Note: the 0.1 release is missing the default repository :/.
My fault. Updated [[Releases]].
> If you haven't ... koszko

12:23 PM Feature #90: Make the 0.1 release

After a somewhat embarrassing length of time, I've come to the realization the script you posted doesn't actually do ... jahoti

06:49 AM Feature #90: Make the 0.1 release

**Note**: the 0.1 release is missing the default repository :/. jahoti

09/14/2021

11:48 PM Support #78: Investigate into how browsers handle files that are not HTML

Rough estimate of progress (it's hard to tell without knowing in advance what the solution will involve) jahoti

11:09 PM Feature #90: Make the 0.1 release

> > As a rather unimportant aside, however, we have yet to establish a clear difference between "Haketilo" and "Haket... jahoti

08:22 PM Feature #90: Make the 0.1 release

OK, it seems all that's important is ready. Documentation will never be perfect but it's already sufficiently good. I... koszko

07:20 PM Feature #90: Make the 0.1 release

In case anyone's wondering how I automatized the generation of Chromium builds with different secres, it's this scrip... koszko

04:24 PM Feature #90: Make the 0.1 release

> As a rather unimportant aside, however, we have yet to establish a clear difference between "Haketilo" and "Haketil... koszko

03:25 AM Feature #90: Make the 0.1 release

> Right, in the documentation (at the end of Mozilla installation instructions, perhaps also in some other place(s)) ... jahoti

03:59 AM Feature #92: Replace cookie smuggling with some safer approach

> Actually, I thought about simply redirecting to an extension-packaged file. For basic functionality we only need 3 ... jahoti

09/13/2021

09:46 AM Feature #90: Make the 0.1 release

jahoti wrote:
> OK, the Firefox account generation/management script is attached.
Thanks a lot!
> (except on s... koszko

09:12 AM Feature #90: Make the 0.1 release

OK, the Firefox account generation/management script is attached. Some notes:
* It depends on `librecaptcha`, `reque... jahoti

09:01 AM Feature #90: Make the 0.1 release

> > (it's also possible they just distribute the signed extensions and package the signatures when building from sour... jahoti

08:02 AM Feature #90: Make the 0.1 release

> > Also, disrtos like Debian actually have extensions in their repositories, so there is surely some way to install ... koszko

08:25 AM Feature #92: Replace cookie smuggling with some safer approach

> That said, there are several options. Apart from the obvious approach of `data:` URLs for Chromium and `contentScri... koszko

09/12/2021

11:20 AM Feature #93 (Rejected): Elaborate on ethics in the documentation

The user manual currently contains several references to what we recommend or what the reader should be doing without... jahoti

11:13 AM Feature #90: Make the 0.1 release

I'm working through testing the Mozilla account-generation script now. I've removed the signing functionality rather ... jahoti

03:00 AM Feature #92: Replace cookie smuggling with some safer approach

It turns out Firefox did once support redirection to `data:` URLs (prior to v60, it seems), before it was accidentall... jahoti

09/11/2021

12:53 AM Feature #92: Replace cookie smuggling with some safer approach

> Jahoti, please, remind me. Why aren't we just making a synchronous AJAX call in the content script and redirecting ... jahoti

09:58 PM Feature #92: Replace cookie smuggling with some safer approach

Jahoti, please, remind me. Why aren't we just making a synchronous AJAX call in the content script and redirecting it... koszko

09:55 PM Feature #92 (Closed): Replace cookie smuggling with some safer approach

Yep, we need to find something that works. `registerContentScript()` might do the job on newer browsers (and under Ma... koszko

12:41 AM Feature #90: Make the 0.1 release

> Interesting. The flag that enables unverified installs is supposedly still supported in developer edition of Firefo... jahoti

12:51 PM Feature #90: Make the 0.1 release

Interesting. The flag that enables unverified installs is supposedly still supported in developer edition of Firefox:... koszko

12:35 PM Feature #90: Make the 0.1 release

>> Wait- is it possible to sign XPIs with our own key? If so that would be much better than relying on Mozilla.
>
>... jahoti

12:22 PM Feature #90: Make the 0.1 release

jahoti wrote:
> Wait- is it possible to sign XPIs with our own key? If so that would be much better than relying on ... koszko

11:54 AM Feature #90: Make the 0.1 release

Wait- is it possible to sign XPIs with our own key? If so that would be much better than relying on Mozilla.
In an... jahoti

11:38 AM Feature #90: Make the 0.1 release

jahoti wrote:
> On that note (and your breakthrough with CRX on #13), do we want to sign releases?
Yes. And I'd l... koszko

05:03 AM Feature #90: Make the 0.1 release

> Also, at some point we'll upload prebuilt versions of Hachette here.
On that note (and your breakthrough with CR... jahoti

12:22 PM Support #75: ServiceWorkers

I unfortunately couldn't test this, as I couldn't find any test cases or understand how to set one up. jahoti

11:44 AM Support #75: ServiceWorkers

jahoti wrote:
> Somehow, it seems the biggest technical challenge for this project has become *blocking (unwanted) s... koszko

05:15 AM Support #75: ServiceWorkers

> Ultimately, we should stop using cookies for policy smuggling, even though they initially seemed like a super good ... jahoti

12:17 PM Support #78: Investigate into how browsers handle files that are not HTML

Your most recent push seems to be working well! jahoti

05:08 AM Support #78: Investigate into how browsers handle files that are not HTML

Good point! jahoti

04:52 AM Support #78: Investigate into how browsers handle files that are not HTML

> didn't the CSP-filtering part of StreamFilter get removed anyway?
It did, although the part that remains is stil... koszko

04:38 AM Support #78: Investigate into how browsers handle files that are not HTML

> I pushed something to koszko branch.
Rather than reply to all the commits you've made independently, I'll just n... jahoti

12:14 PM Feature #88: [Roadmap 6][Milestone] Allow payloads to also specify CSP rules that should be used instead of the original ones served by page

I read this thread earlier today and had been meaning to reply, yet couldn't find it again- sorry!
> In the end, I... jahoti

12:02 PM Feature #32: Process HTML files in data: URLs instead of just blocking them

> Btw, I've been unaware of that manifest key. It would be cool to utilize it for something else at some point :) Alt... jahoti

11:40 AM Feature #32: Process HTML files in data: URLs instead of just blocking them

:/
Btw, I've been unaware of that manifest key. It would be cool to utilize it for something else at some point :) A... koszko

04:56 AM Feature #32: Process HTML files in data: URLs instead of just blocking them

> It might be possible to utilize this API:
>
> <https://developer.mozilla.org/en-US/docs/Web/API/Navigator/registe... jahoti

11:48 AM Feature #91 (Rejected): Add an option to block HTTP "refresh"

This concerns both the HTTP header and its respective `<meta>` tag.
https://en.wikipedia.org/wiki/Meta_refresh koszko

11:03 AM Feature #77 (Closed): Check LibreJS is compatible with this extension.

Compatibility is confirmed for IceCat 60, which is sufficient assuming there are no functional differences that would... jahoti

05:05 AM Feature #77: Check LibreJS is compatible with this extension.

Results will be added to the user documentation once obtained. jahoti

05:13 AM Feature #13: find some way not to require each chrome user to modify manifest.json

> Unfortunately, the "Google BSD license" link is dead and I cannot check which of the BSD licenses applied to that s... jahoti

04:44 AM Feature #66: Write tests

> Have you considered using UML (no, not that diagraming language, I mean User Mode Linux) to run tests inside? I'm s... jahoti

09/10/2021

10:07 PM Feature #90: Make the 0.1 release

I started documenting Hachette usage. I uploaded the screenshots I made, so if you happen to come there while I sleep... koszko

05:15 PM Feature #90: Make the 0.1 release

"allow" option, CSP behavior and URL length limits are now on `koszko` branch koszko

08:49 PM Feature #13: find some way not to require each chrome user to modify manifest.json

I found details regarding the CRX file format:
http://www.dre.vanderbilt.edu/~schmidt/android/android-4.0/external/c... koszko

05:47 PM Support #75: ServiceWorkers

I added unregistering code on `koszko` branch. It needs testing koszko

05:34 PM Feature #32: Process HTML files in data: URLs instead of just blocking them

It might be possible to utilize this API:
https://developer.mozilla.org/en-US/docs/Web/API/Navigator/registerProto... koszko

05:07 PM Feature #88: [Roadmap 6][Milestone] Allow payloads to also specify CSP rules that should be used instead of the original ones served by page

As this is somehow related, I'll write an update regarding our recent CSP change (where we are no longer modifying ex... koszko

09/09/2021

05:35 PM Support #75: ServiceWorkers

jahoti wrote:
> perhaps we could present some version of [this information](https://www.ghacks.net/2016/03/02/manage... koszko

01:52 PM Feature #66: Write tests

Have you considered using UML (no, not that diagraming language, I mean User Mode Linux) to run tests inside? I'm sug... koszko

12:51 PM Feature #34 (Closed): improve CSP injection blocking

Can be considered done as part of #78 koszko

12:15 PM Support #78: Investigate into how browsers handle files that are not HTML

> I am going to continue with this tomorrow. Btw, I realized some mistakes (including being unaware of what I just de... koszko

09/07/2021

10:31 PM Support #78: Investigate into how browsers handle files that are not HTML

I now realize what is the problem with all XMLs, including SVGs. Any XML can include elements from other XML namespac... koszko

10:52 AM Support #78: Investigate into how browsers handle files that are not HTML

I suppose it's the same as with SVG, although I need to make sure it's really the case koszko

09/06/2021

12:05 AM Feature #90: Make the 0.1 release

That leaves me with 4, I suppose, which is probably just as well; the current (limited) state of the testing suite is... jahoti

08:51 PM Feature #90: Make the 0.1 release

`3`. is now ready, as noted in #78 koszko

04:54 PM Feature #90: Make the 0.1 release

Instead of implementing 2. as specified in the description, I did something else. Effect is as wanted - build.sh gene... koszko

02:39 PM Feature #90 (Closed): Make the 0.1 release

Right now what we have left to do is:
1. ~~Make it impossible to check "allow" option for page with payload, as sugg... koszko

12:02 AM Support #78: Investigate into how browsers handle files that are not HTML

> I came up with code that should do with blocking for now. On koszko branch. Could do with more testing
Doing thi... jahoti

08:49 PM Support #78: Investigate into how browsers handle files that are not HTML

I came up with code that should do with blocking for now. On `koszko` branch. Could do with more testing koszko

06:55 PM Support #78: Investigate into how browsers handle files that are not HTML

Now we know why NoScript included special code for SVGs and XMLs:
https://developer.mozilla.org/en-US/docs/Web/SVG/E... koszko

02:57 PM Support #78: Investigate into how browsers handle files that are not HTML

> > While server might not be able to make user's browser execute scripts in a non-HTML page, we are. Should we restr... koszko

11:48 AM Support #78: Investigate into how browsers handle files that are not HTML

> While server might not be able to make user's browser execute scripts in a non-HTML page, we are. Should we restrai... jahoti

09:56 AM Support #78: Investigate into how browsers handle files that are not HTML

> > Now it would make sense to make content script not try to inject payload if document.contentType is not of proper... koszko

12:00 AM Feature #13: find some way not to require each chrome user to modify manifest.json

> The "key" manifest property was required by Chromium to be an actual key in PEM format
Thank you for explaining!... jahoti

04:53 PM Feature #13: find some way not to require each chrome user to modify manifest.json

> > Wouldn't that still require each user to build the extension themselves?
>
> It would. It would just be less h... koszko

11:45 PM Feature #28: split options_main.js into several smaller files

> Right now I can quickly make this little change you suggested since I already know that code. And you could instead... jahoti

02:20 PM Feature #28: split options_main.js into several smaller files

Discussion moved from #15
>>>>Since long-term we're not really planning to allow our scripts to run together with ... koszko

11:41 AM Feature #15: make sure page's own csp in <head> doesn't block our scripts

> Keep in mind, however, options_main.js is currntly the most tangled script file in Hachette
Perhaps I'll start ... jahoti

10:24 AM Feature #15: make sure page's own csp in <head> doesn't block our scripts

> > Since long-term we're not really planning to allow our scripts to run together with page's ones (i.e. "allow site... koszko

11:41 AM Feature #7: [Roadmap 34][Milestone] find some convenient way to automatically re-add intrinsic javascript

> You mean re-allowing the actual intrinsics as they appear on the page they came with?
I did, having not really t... jahoti

10:37 AM Feature #7: [Roadmap 34][Milestone] find some convenient way to automatically re-add intrinsic javascript

> A hacky and flawed solution to this might be to simply scan the nodes and rever event-handler attribute blocking.
... koszko

11:29 AM Support #75: ServiceWorkers

> Unfortunately, it seems a page reload is required for this to take effect.
>
> Additionally, is there a way servi... jahoti

10:50 AM Support #75: ServiceWorkers

Unfortunately, it seems a page reload is required for this to take effect.
Additionally, is there a way service w... koszko

09:51 AM Feature #70: [Roadmap 7][Milestone] Add facility to replace sites' original HTML with custom one

Together with this, we could allow scripts to access the original, raw HTML code of the page in question. I am mentio... koszko

09/05/2021

10:50 AM Feature #26 (Closed): besides blocking scripts through csp, also block connections that needlessly fetch those scripts

Tentatively closed; the bug is no longer reproduceable on IceCat, LibreWolf, or Ungoogled Chromium (version to be not... jahoti

04:38 AM Feature #26: besides blocking scripts through csp, also block connections that needlessly fetch those scripts

I'll check if this is even an issue either today or in the next few days (if live scripts are never added to the acti... jahoti

05:12 AM Support #75: ServiceWorkers

The following script will deregister all service workers in a page (courtesy of <https://love2dev.com/blog/how-to-uni... jahoti

04:52 AM Feature #14: test with more browser forks (Abrowser, newest Parabola IceWeasel, LibreWolf)

> I am not so sure. Official mobile releases stopped at 38.6.0.
That complicates things. I'll see if I can find w... jahoti

04:50 AM Feature #7: [Roadmap 34][Milestone] find some convenient way to automatically re-add intrinsic javascript

A hacky and flawed solution to this might be to simply scan the nodes and rever event-handler attribute blocking. jahoti

04:44 AM Feature #16 (Closed): create a repository to host scripts

See project:Hydrilla and the instance at [[https://api-demo.hachette-hydrilla.org]]. jahoti

04:29 AM Feature #66: Write tests

The basic infrastructure to support creating a "virtual network" in now in the `jahoti` branch, and can be used on it... jahoti

02:20 AM Feature #15: make sure page's own csp in <head> doesn't block our scripts

> If any part of Hachette can be considered infrastructure trap, it's surely this CSP stuff. Having already done so m... jahoti

09/04/2021

01:40 AM Support #78: Investigate into how browsers handle files that are not HTML

> Btw, I noticed cookies don't work on non-HTML pages. This doesn't seem to be an issue as long as we assume the conc... jahoti

09:05 PM Bug #89 (Closed): Restore, to the extent necessary, the script sanitizing functionality

Merged to `master` koszko

08:50 PM Bug #89 (Closed): Restore, to the extent necessary, the script sanitizing functionality

Sanitizing of `<script>` tags was recently dropped because it seemed sufficient to rely on CSP rules being injected. ... koszko

07:36 PM Feature #88 (New): [Roadmap 6][Milestone] Allow payloads to also specify CSP rules that should be used instead of the original ones served by page

Note that this concerns CSP rules other than those for scripts. For scripts we always use a nonce
[Roadmap](/proje... koszko

07:33 PM Bug #65 (Closed): When a site fails to load, for example due to its IP address not being found, the injected value with settings remains in the URL

Merged to `master` koszko

12:36 PM Feature #11 (Closed): add some nice styling to settings page

Merged to `master` koszko

12:35 PM Feature #15 (Closed): make sure page's own csp in <head> doesn't block our scripts

Merged to `master` koszko

12:35 PM Feature #23 (Closed): also implement support for whitelisting of non-https urls

Merged to `master` koszko

12:34 PM Feature #31 (Closed): add an option to disable script blocking globally

Merged to `master` koszko

12:34 PM Feature #49 (Closed): add some nice styling to popup

Merged to `master` koszko

09/03/2021

07:19 PM Support #78: Investigate into how browsers handle files that are not HTML

Modified StreamFilter code is now on `koszko-rethinked-meta-sanitizing`. The `policy` object now also contains inform... koszko

12:36 PM Support #78: Investigate into how browsers handle files that are not HTML

No, since under Chromium I've never actually seen our "document_start" content scripts start with DOM partially or fu... koszko

12:19 PM Support #78: Investigate into how browsers handle files that are not HTML

> Perhpas we could instead, in StreamFilter, just try running DOMParser over the first chunk of data and examining th... jahoti

11:17 AM Support #78: Investigate into how browsers handle files that are not HTML

Heuristics. That's bad... For us.
Even mere parsing of response headers is already risky because of some subtletie... koszko

10:21 AM Support #78: Investigate into how browsers handle files that are not HTML

According to <https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types#mime_sniffing>:
> In th... jahoti

12:52 PM Feature #85: Make Haketilo use the same format as Hydrilla for import and export of settings

jahoti wrote:
> Is the Hydrilla format stable? If not, is it worth waiting for that first or should this be easy eno... koszko

12:27 PM Feature #85: Make Haketilo use the same format as Hydrilla for import and export of settings

Is the Hydrilla format stable? If not, is it worth waiting for that first or should this be easy enough to do now? jahoti

12:50 PM Feature #14: test with more browser forks (Abrowser, newest Parabola IceWeasel, LibreWolf)

jahoti wrote:
> I suspect IceCat can be built on FSDG-compliant distros.
I am not so sure. Official mobile releas... koszko

12:25 PM Feature #14: test with more browser forks (Abrowser, newest Parabola IceWeasel, LibreWolf)

I suspect IceCat can be built on FSDG-compliant distros. Ungoogled Chromium might have that option, yet it's pointles... jahoti

12:23 PM Feature #15: make sure page's own csp in <head> doesn't block our scripts

If any part of Hachette can be considered infrastructure trap, it's surely this CSP stuff. Having already done so muc... koszko

11:59 AM Feature #15: make sure page's own csp in <head> doesn't block our scripts

> So we still need workarounds under Mozilla :/
How easy life would be if everything worked reasonably well!
> ... jahoti

10:32 AM Feature #15: make sure page's own csp in <head> doesn't block our scripts

> - On Chromium, nodes injected by content scripts are CSP-exempt, meaning CSP filtering is unnecessary (albeit harml... koszko

09:51 AM Feature #15: make sure page's own csp in <head> doesn't block our scripts

Sorry I didn't see your question! I distracted myself with researching around the topic (in the midst of general busy... jahoti

12:18 PM Feature #83: Also add ability to selectively block other types of content (e.g. fonts)

> I am not entirely sure the actual fetching of resources is also prevented by CSP. What I am sure would work, though... jahoti

11:44 AM Feature #83: Also add ability to selectively block other types of content (e.g. fonts)

I am not entirely sure the actual fetching of resources is also prevented by CSP. What I am sure would work, though, ... koszko

10:16 AM Feature #83: Also add ability to selectively block other types of content (e.g. fonts)

To summarise from the [full list of CSP directives](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content... jahoti

09/02/2021

09:37 PM Feature #15: make sure page's own csp in <head> doesn't block our scripts

I pushed some code for this to new `koszko-rethinked-meta-sanitizing` branch. I am not yet 100% sure this will work. ... koszko

09:05 PM Feature #85 (In Progress): Make Haketilo use the same format as Hydrilla for import and export of settings

I just realized it should be possible to access entire directories:
https://developer.mozilla.org/en-US/docs/Web/API... koszko

06:37 PM Feature #31 (Resolved): add an option to disable script blocking globally

On `koszko-smuggle-policy` branch koszko

09/01/2021

02:18 PM Feature #11: add some nice styling to settings page

Import dialog is now also styled. All that's left is merging to `master` koszko

02:18 PM Feature #49: add some nice styling to popup

Install dialog is now also styled. All that's left is merging to `master` koszko

11:48 AM Feature #49: add some nice styling to popup

This is now also on `koszko-smuggle-policy` branch, except for the install dialog koszko

01:49 PM Feature #15: make sure page's own csp in <head> doesn't block our scripts

Did you have any success? koszko

08/31/2021

01:36 PM Feature #83 (New): Also add ability to selectively block other types of content (e.g. fonts)

Google uses fonts sites load from its servers for snooping. Blocking them causes relatively little issues (compared t... koszko

01:32 PM Feature #11: add some nice styling to settings page

Forgot to mention: this has been ready (except for settings import window) on `koszko-smuggle-policy` branch since ye... koszko

08/30/2021

12:13 PM Feature #14: test with more browser forks (Abrowser, newest Parabola IceWeasel, LibreWolf)

## Mobile version considerations
I don't have any Android device to test on, although it might be possible to use ... koszko

12:02 PM Feature #82 (New): Style extension's pages for mobile usage

koszko

12:00 PM Feature #80 (New): Make Haketilo work with mobile versions of browsers

This is mostly the matter of apropriately styling extension's pages. Unfortunately, a libre mobile browser to test on... koszko

08/28/2021

08:56 AM Support #78: Investigate into how browsers handle files that are not HTML

> As for making sure we only filter relevant data, do any browsers try to guess mime types?
By guessing you mean a... koszko

03:00 AM Support #78: Investigate into how browsers handle files that are not HTML

For the second point at least, I know NoScript operates on XML (and will check uBlock Origin for similar behavior). W... jahoti

08:48 AM Feature #13: find some way not to require each chrome user to modify manifest.json

> Wouldn't that still require each user to build the extension themselves?
It would. It would just be less hacky t... koszko

02:54 AM Feature #13: find some way not to require each chrome user to modify manifest.json

> Using a synchronous AJAX call from the content script might allow us to use a bundled file as a secret
Wouldn't ... jahoti

08/27/2021

10:58 AM Feature #79 (Closed): Improve the build script by using awk

Since writing `build.sh` I realized some things could be done a lot easier using awk koszko

10:56 AM Feature #23 (Resolved): also implement support for whitelisting of non-https urls

koszko

10:55 AM Feature #23: also implement support for whitelisting of non-https urls

`ftp://` is now also ready and pushed to this temporary branch. Changes will be merged together with completed Featur... koszko

10:12 AM Feature #23: also implement support for whitelisting of non-https urls

Support for the `file://` protocol is now on the `koszko-smuggle-policy` branch. I re-used the temporarily-unused app... koszko

10:32 AM Support #78 (Rejected): Investigate into how browsers handle files that are not HTML

Our tampering with HTML pages, including rewriting parts of them using the StreamFilter API, might cause problems whe... koszko

10:26 AM Feature #77: Check LibreJS is compatible with this extension.

# History before copying
koszko wrote:
> I assume by compatibility you mean the ability to run side-by-side with ... koszko

10:26 AM Feature #77 (Closed): Check LibreJS is compatible with this extension.

Many swfreedom supporters prefer LibreJS' blocking mechanism. As there's good reason to expect compatability, it woul... koszko

10:07 AM Feature #13: find some way not to require each chrome user to modify manifest.json

Using a synchronous AJAX call from the content script might allow us to use a bundled file as a secret koszko