Haketilo

Haketilo moving away from being a browser extension seems like it would reduce the number of people who download it. Installing an extension is so easy (when it's signed), but every time I've used mitmproxy or other HTTP proxies, I've found it complicated to set up (though, I haven't used a proxy in a long time). Would it be possible, long term, to bundle the proxy as part of the web browser? I think it would be better to have an extension, even if it had limited functionality, but I don't know the details of how much effort that would take, so maybe it's not worth it.

Also, I found a page that seems to suggest that modifying CSP headers might be allowed in this case: https://discourse.mozilla.org/t/add-ons-policy-changes-2021-q-a/89654/1 (https://archive.ph/rkMEh or document.write(document.getElementsByTagName("noscript")[0].innerHTML); (Strangely view-source:https://discourse.mozilla.org/t/add-ons-policy-changes-2021-q-a/89654/1 seems to load forever.))

Q: Does " Add-ons must not relax web page security headers, such as the Content Security Policy" include addons that:

    Are designed to do specifically that - for example, CORS Everywhere,
    Add-ons that require relaxing security headers to be functional, such as an extension adding/modifying security headers (e.g. Referer) when accessing an API from background page which is normally inaccessible due to server-side CSRF protection or,
    a userscript loading extension using script tag/eval that replaces CSP?

A: Add-ons related to web development that are explicitly built and described to interact with those headers (e.g. CORS Everywhere) continue to be permitted. This is considered an edge case of the rule. Developers of such extensions should make an effort to clearly describe the security impact and ensure that users don’t accidentally have these security headers disabled while not actively using the add-on.
We’ve seen a number of add-ons that relax the CSP or remove certain headers out of convenience to enable certain functionality. This isn’t great because the protections website authors have put into place are being relaxed, which may have unintended consequences.
One example mentioned above I’d like to provide a bit more detail on, if you are setting a Referer header to achieve a specific result when making a singular request, this is not something disallowed by the policy. With this approach you are not relaxing security headers.

Have you asked Mozilla about this Q&A entry (not sure how the signing process works)? I'm not totally sure what is meant by "related to web development", so I guess Mozilla can deny it if they want. It would probably be good to have a mechanism by which the extension can be signed by another entity in addition to Mozilla, that browsers shipped with FSDG distros (Trisquel, etc.) trust in the way they currently trust Mozilla (Then again, maybe a waste of time if the extension is ultimately going away anyway).

Thanks for your feedback, Jacob. I respond inline.

Haketilo moving away from being a browser extension seems like it would reduce the number of people who download it. Installing an extension is so easy (when it's signed), but every time I've used mitmproxy or other HTTP proxies, I've found it complicated to set up (though, I haven't used a proxy in a long time).

I admit being able to install an extension is a great convenience. This is what made me start Haketilo as an extension in the first place. For some time, I was considering developing a proxy at a significantly later point, as a mere alternative to the extension.

Unfortunately, I am only a human and at some point I simply got discouraged by multiple things that didn't work as expected and slowed down the extension development. The AMO policy issue was just "the last straw that broke the camel's back". Perhaps the fact I gave up with the extension speaks badly of myself. Or perhaps it was actually a good decision? The policy issue is a rather significant one, after all, and there are loads of things that are only going to be achievable with a proxy.

If curious, you can look at some old issues like #15 to see what kinds of problems the WebExtensions platform used to cause.

Would it be possible, long term, to bundle the proxy as part of the web browser?

It would be. Although for that to happen, we'd need more people working on the project (the 2 contributors that I had are no longer participating 😢).

Have you asked Mozilla about this Q&A entry (not sure how the signing process works)?

No, I haven't. In fact, I did not see it before. I guess Haketilo would fall into the category of "add-ons that relax the CSP or remove certain headers out of convenience to enable certain functionality", but I might actually ask them about it (just not right now).

I guess Mozilla can deny it if they want

They surely can. This is yet another thing I don't like - that even though the software (well, 99% of it) is nominally free/libre, I might still get trapped by it.

It would probably be good to have a mechanism by which the extension can be signed by another entity in addition to Mozilla, that browsers shipped with FSDG distros (Trisquel, etc.) trust in the way they currently trust Mozilla (Then again, maybe a waste of time if the extension is ultimately going away anyway).

In fact, I am surprised something like this has not yet been done. Ideally, the browser should allow users to specify what entities they want to trust. And this would make sense regardless of Haketilo being there or not :)

I think it would be better to have an extension, even if it had limited functionality, but I don't know the details of how much effort that would take, so maybe it's not worth it.

You might have just pointed out to the main problem of mine: that I am trying to make my product perfect and spend a lot of time. If I instead delivered something that's just "good enough" but did it faster, it could be way better for the actual project.

The amount of effort needed to battle the WebExtension APIs has so far been enormous. You know what? While I cannot make the AMO policy issue go away, I have a vision of how we could at least make the extension development more approachable:

1. We could make it Firefox-only. Chromium releases were not receiving much interest and other browsers would be covered by the proxy anyway. At the same time, Firefox APIs have more to offer (StreamFilter, blocking webRequest under manifest v3).
2. We could restrict the extension to only working with HTTP(s) pages. Support for file:// and ftp:// (for older browsers that still supported it) is of less importance and diverges the development from focusing on what's really important.
3. Once we agree for 1. and 2., we can handle all pages with just StreamFilter and webRequest which simplifies things A LOT.

I am still unsure how best to comply with the other problematic AMO policy point:

Only release versions of third-party libraries and/or frameworks may be included with an add-on. Modifications to these libraries/frameworks are not permitted.

This by itself sounds like an imprisonment of free software. And it's of course incompatible with the current Haketilo build system... We could move away from our custom build system to some established one at the cost of adding some build-time dependencies (up until now Haketilo could be build with just POSIX). Actually, doing this earlier could have saved me some effort (mea culpa).

We'd also need to find a way to make automated tests easier to maintain. So far I've been using pytest and selenium. The first problem is that I had to put a lot of JS code inside Python strings which is inconvenient. I also didn't make the "unit tests" (quotes needed) truly restricted to single functions which led to complex test cases (this is my own fault again). The error messages from selenium are also rather poor and the tests themselves take somewhat long to run 😕

A reasonable solution might be to test the JS code outside of browser, for example with a library like JSDOM (also packaged for Debian ^^).

I haven't yet used the alternative technologies I am considering here. I'd of course need to spend additional time learning them. It might be worth it, though.

So, I gave you the relevant details. You can share your thoughts on them. Just please keep in mind that I'm not likely to refocus on the extension until I actually have some other contributors to help me with it.

Lastly, I am sorry for taking so long to respond (Redmine's email notifications seem to have failed me once again)

every time I've used mitmproxy or other HTTP proxies, I've found it complicated to set up

I might have a good news for you: it should be perfectly possible to package the proxy into a .deb so that it starts automatically and automatically installs its certificate system-wide :)