Allow host files hosted on HTTP.

[mikhoul]

In the past to bypass this restriction from Github with Steven Black Host files that prevent AdAway to know if the files where updated I was using his alternate hosting here: http://sbc.io/hosts/alternates/gambling/hosts but AdAway don't seem to no longer allow non-https files with the latest version 4.2.2 😞

The button stay ghosted with HTTP file.

You should ad an option to enable the use of non-https file.

Regards

[J316]

Making this opt-in should make everyone happy

[PerfectSlayer]

Hi @mikhoul

Sadly, it is not by choice but by design. When you target Android 9 as build version, you have to define which host must be allowed without SSL layer.
And as the host are choosen by user, it can't be hardcoded in the app.

So only https from now. All the details on this post.

@J316 User option is not an option! 😓

[IRainman]

I have root, it's possible to fix the problem?

P.S. I reverted to version 3.3 from F-Droid.

[Tobias-B-Besemer]

... it's possible to fix the problem?

Best fix is to give the hosts (of the files) https. ;-)

With which host do you have issues?

[Ji-eF]

In my case this host :

http://winhelp2002.mvps.org/hosts.txt

[Tobias-B-Besemer]

@Ji-eF, you can request https support at winhelp, or switch e.g. to Steven Black's hosts file, that include the winhelp file...

[J316]

@Tobias-B-Besemer There are many other lists without https support and the fact he wants to use that list doesn't mean he wants to use all the others from a list of lists

[Tobias-B-Besemer]

@J316, as @PerfectSlayer already said in #1112 (comment):

Sadly, it is not by choice but by design. When you target Android 9 as build version, you have to define which host must be allowed without SSL layer.
And as the host are choosen by user, it can't be hardcoded in the app.

Google (Chrome) already started to degrade pages that have no https... So I wonder that pages with "lots of users" still don't have it... Also: Everybody can have a certificate for https for free, nowadays!
And at any point there must be a break to old versions/compatibility... AA supports now Android 9, if you want to have compatibility with Android 1 & http, you should use a old version, because newer versions don't support it anymore...
Dropping http is like dropping support for WinXP - it's old, it's EOF, it's time to drop it!

[J316]

@Tobias-B-Besemer It's not fair to compare Android <9, to Windows XP, Pie represents a very small percentage of Android devices around at this time and Nougat and Oreo are all but old. Why not having an option to allow http only for Android versions below 9?

[PerfectSlayer]

@J316 Did you read the related thread post I linked?

[Tobias-B-Besemer]

@J316, no, I compared http-only-pages to WinXP... ;-)
But have a switch in the program for Android versions <9, is a good idea! :-)
I just don't know if it's possible (valid code for Android 9) and how much work it would be... But lets ask @PerfectSlayer about it...
And why not target the hosts file makers ??? Tell them you need (finally) https-support... It's really time for it! ;-)

Edit: Sorry PS, I didn't refreshed the page before I send out my answer. -> Saw your answer later...

[Tobias-B-Besemer]

@PerfectSlayer:

@J316 Did you read the related thread post I linked?

Even if my goal is to do not require root, I still need it to challenge the system to find breaches when I can settle in.

Btw.: Whats about a Magisk-Addon (or Xposed) for AA ??? Would this help ???

[Tobias-B-Besemer]

@PerfectSlayer:

@J316 Did you read the related thread post I linked?

But have a switch in the program for Android versions <9, is a good idea! :-)
I just don't know if it's possible (valid code for Android 9) and how much work it would be... But lets ask @PerfectSlayer about it...

As far as I understand it: You decided that you don't want http support anymore?

[PerfectSlayer]

@Tobias-B-Besemer There is no way to set an option.
All untrusted or all trusted.
All untrusted is not acceptable.

[Tobias-B-Besemer]

@Ji-eF, @J316, @PerfectSlayer, to make progress here, I wrote now the following mail to winhelp2002:

Subject: https for your site

Hi,

we discuss your project at the moment in this issue #1112 from AdAway. Reason is: AdAway supports from now on only pages with https! So my question: Do you have any plans to add https to your site? An answer per mail, or direct in the issue, would be nice!

Greets, Tobias.

[Tobias-B-Besemer]

I got an answer...

Tobias,
Thanks for your feedback ...

I am not the owner of MVPS.org ... so I have no control over the use of https.

The Adaway you cite seems to pertain to Android ... the majority of my users (+50,000) use my HOSTS file for what it was intended ... on a computer.

Mike Burgess
Consumer Privacy and Security
"There's no place like 127.0.0.1"
http://winhelp2002.mvps.org/hosts.htm

[Tobias-B-Besemer]

My mail back:

Hi Mike,

thank you for you fast answer, I will copy it into the issue - hope that's OK for you - the others are interessted in your answer, too...
What's about a https-mirror for your hosts-file, would this be a solution?

Greets, Tobias.

[Tobias-B-Besemer]

Intermediate result @ALL: Use a page/host file that is willing to support the nowadays "normal" standard https! ^^ (...and have a bit of respect for AdAway and Android users... ^^)

[Tobias-B-Besemer]

Answer:

Sorry ... I have no intention on creating a mirror site ...

Mike Burgess
Consumer Privacy and Security
"There's no place like 127.0.0.1"
http://winhelp2002.mvps.org/hosts.htm

[Tobias-B-Besemer]

So he didn't even understood, that I was talking about having an hosts-file mirror... :-D

As I said: Use a other list! ^^

[Ji-eF]

@Tobias-B-Besemer : thanks for your concern :)
While waiting for the jump to P, I wrote a script that uses adb over TCP to copy my computer's /etc/hosts over to my phone :)

[tonia6970]

Any solution ? Why previous adaway support http & this latest version NOT ? Also via shrtening url, you can convert the http to https (redirecting method) so that to add it in Adaway but Adaway can't update the host file although the redirecting method for host files from the preference is enabled. Recover this option which was on Adaway 3.3.61.

[Kronopt]

#1112 (comment)
https://forum.xda-developers.com/showpost.php?p=77442553&postcount=11698

[tonia6970]

If I have understood well from the comments that this problem is due to Pie due to increased privacy. I don't think that it is the reallity because in Pie with Adaway 3.3.61 I can have http hosts which means that the problem is by Adaway v4+.

[IRainman]

@tonia6970 maybe it's in updating the SDK or something like that. In any case, I simply neglected the updates and stayed on the last 3 version. It's just a program for writing a text file (hosts) from several other text files. Updates do not give new functionality.

P.S. Perhaps this is just the desire of the AdAway developers to be so :) I am not going to add SSL to my home site inside the LAN, because these are unnecessary problems for me and zero benefit.

[tonia6970]

Anyway, I believe that this is bud by adaway & not limitation from the Pie version due to sdk or something else because latest version of Adaway v3 in Pie works properly. I also have staied iin latest version 3 of Adaway (v3.3.61 I think) ubtil to see if some new version fix this problem. Also i don't se any benefit to go in v4, only problems.  Στις Κυριακή, 15 Σεπτεμβρίου 2019, 11:59:06 π.μ. EEST, ο χρήστης IRainman <notifications@github.com> έγραψε: @tonia6970 maybe it's in updating the SDK or something like that. In any case, I simply neglected the updates and stayed on the last 3 version. It's just a program for writing a text file from several other text files. Updates do not give new functionality. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[mikhoul]

@PerfectSlayer Could you do a second version that would allow HTTP for users that don't care about using non secure HTTP domain ?

This version could be hosted here to be installed manually, it could have a big warning about allowing non-secure URL so more advanced users would be happy and versions on Fdroid would be secure.

Regards

[PerfectSlayer]

I don't think that it is the reallity because in Pie with Adaway 3.3.61 I can have http hosts which means that the problem is by Adaway v4+.

@tonia6970 Did you read the Android developer documentation before telling the problem comes from the app? I mean, I spend quite some time to write this post. Do it and you should find the issue.

@IRainman Oh yes, it's totally fine. There is nothing really important in the 4.x versions. You could safely keep it (maybe some systemless root support but it's bundled in Magisk now).

Hello @mikhoul. Sadly, I won't make it because it will take too much time (I do maintain the app on my free time). I already have two variants with XDA version and F-Droid version and two versions with stable 4.x and beta 5.x. So 4 releases. If I add HTTP variant, there will be 8 different builds 😢
As previously explained, HTTPS is now the standard (even browsers mark http resources as insecure). The app as root access, it will soon update itself so it is definitely a good idea.
I would recommend:

• To stay on 3.x version,
• To stay on Android 8,
• To request hosts file hoster to use TLS
• To host your own hosts file in github, gitlab or gist for example.
  Any of this solution will solve your HTTP only issue.

[nnnn20430]

@PerfectSlayer

Sadly, it is not by choice but by design. When you target Android 9 as build version, you have to define which host must be allowed without SSL layer.
And as the host are choosen by user, it can't be hardcoded in the app.

actually not true according to the doc.
i just read https://developer.android.com/training/articles/security-config.html
and it seems there is <base-config cleartextTrafficPermitted="true"> which sets default for every request, instead of <domain-config cleartextTrafficPermitted="true"> which is per domain.

i would personally like to use hosts file from my own local server.

[nnnn20430]

@Tobias-B-Besemer There is no way to set an option.
All untrusted or all trusted.
All untrusted is not acceptable.

again not true, you can set all domains by default to allow cleartext and then make per domain exceptions to be tls only

[PerfectSlayer]

As I explained multiple times before, I won't allow unencrypted connections for all connections.

I already given all the reasons why this settings is not an option. Check all the details on the XDA discussion.

[PerfectSlayer]

@nnnn20430 You still don't control all domains your app request. There is CDN, domain redirection, etc… You can't seriously make a privacy related app by allowing untrusted networks as default settings.

There is numerous way to setup TLS, even for free and a lot of users really want better security for the app.
Even if some of you still thumb down my answers, I won't lower security or privacy just because it is easier to add the few http only hosts that left.

[pchris7489]

Hello,

I came to this topic because I have a massive dynamically updated self hosted file and I would like to use it in AdAway. What's the solution in this case?

[IRainman]

@pchris7489 try to use GitHub for these lists. HTTP isn't available at the level of Android API at all.

[IRainman]

P. S. for me work solution is completely break support for Android, get a pocket router with AdGuard Home and use another instance of AdGuard Home in LAN and my personal VPN too.

[Johnn78]

Hello,

I came to this topic because I have a massive dynamically updated self hosted file and I would like to use it in AdAway. What's the solution in this case?

Can you share this updated host file with us ?

[pchris7489]

Hello,
I came to this topic because I have a massive dynamically updated self hosted file and I would like to use it in AdAway. What's the solution in this case?

Can you share this updated host file with us ?

I don't understand your question. Its a simple host file (as simple as it gets in its structure) with a lot of entries, and this host file is hosted on an HTTP server in my home (Python SimpleHTTPServer, but whatever the web server). This helps me to control dynamically what ads can be displayed or redirected on the phone (and other devices in my home), depending on different factors and situations.

In this case, its obvious that implementing HTTPS is total overkill. Using GitHub also is overkill (why would I need to upload to the Internet a file to redownload shortly it in my own home). I will think about the multiple solutions including this one. It seems Adaway can use local files as sources, I will maybe download the file using some Tasker or whatever (that can still connect to HTTP btw), and place it locally.

[IRainman]

Hello,
I came to this topic because I have a massive dynamically updated self hosted file and I would like to use it in AdAway. What's the solution in this case?

Can you share this updated host file with us ?

My lists are located here https://github.com/IRainman/internet_additional_cleanup_and_fix