seeds: Pull additional nodes from my seeder and update fixed seeds #30008
Conversation
|
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. Code CoverageFor detailed information about the code coverage, see the test coverage report. ReviewsSee the guideline for information on the review process. ConflictsNo conflicts as of last run. |
| @@ -510,518 +510,6 @@ zvchlrjuzqdlx37fhibhnym4y6p56vtlymujjuzhh2cp34yqfrtq.b32.i2p:0 | |||
| zxsd3fqczh6ddgejc24nnmb3ww7nalieq3a7cs2mqiy6tmff3wia.b32.i2p:0 | |||
| zy2ywvyqds5bgdoo4tgbu3bwjp3ygyn3zfuby44jemc6xa6fbwta.b32.i2p:0 | |||
| zzre44vh766jgfordw2ehu2r6p44j23uyovgvm7iwuhp3g5iz4ca.b32.i2p:0 | |||
| ycdw2e4ufgfwhcqna4g3m2qsvaly23ozaexawcj3x4gtgcehgwujjgid.onion:8333 | |||
There was a problem hiding this comment.
It looks like something odd happened to the manual onion and i2p seeds. Only a small range of first letters were present, and seeds run by colleagues and the bitcoin community were no longer present.
There was a problem hiding this comment.
They were updated in #29561, and the addresses were pulled from my node's addrman. Some sorting happened somewhere, and because makeseeds.py doesn't shuffle (this PR adds a commit that does that), when it applied the max node count, it ended up with the tail end of that list.
There seem to be sufficient i2p and onion nodes now that there is no need to specifically include nodes run by known people. We don't do this for IPv4 or IPv6.
Update the user agent regex to match all 3 digits of the version number, not just the first 2 digits. Also updates it to include 24.2, 25.2, 26.1, 27.0, and 27.99.
The crawlers are not guaranteed to output nodes in a random order, so shuffle the ips list after parsing to break any biasing that may be caused by the output order.
achow101
force-pushed
the
my-seeder-fixed-seeds
branch
from
ef78deb
to
f6bc14a
Compare
The seeders now produce onion and i2p seeds, so there is no need to keep these in the manual list. Although should also be produced, there are not enough good ones detected by the seeder, so we keep the manual seeds for them.
Update the fixed seeds for both mainnet and testnet
achow101
force-pushed
the
my-seeder-fixed-seeds
branch
from
f6bc14a
to
94e99a9
Compare
|
My seeder has now found several i2p nodes, so I've gone ahead and removed the manually curated ones for both mainnet and testnet. These are now filled in by the script. The only manual ones remaining are cjdns. However, as my seeder has also found cjdns nodes, they have been added, but are only a couple. |
The DNS seeder that I wrote collects statistics on node reliability in the same way that sipa's seeder does, and also outputs this information in the same file format. Thus it can also be used in our fixed seeds update scripts. My seeder additionally crawls onion v3, i2p, and cjdns, so will now be able to set those fixed seeds automatically rather than curating manual lists.
In doing this update, I've found that
makeseeds.pyis missing newer versions from the regex as well as cjdns support; both of these have been updated.I also noticed that the testnet fixed seeds are all manually curated and sipa's seeder does not appear to publish any testnet data. Since I am also running the seeder for testnet, I've added the commands to generate testnet fixed seeds from my seeder's data too.
Lastly, I've updated all of the fixed seeds. However, since my seeder has not found any cjdns nodes that met the reliability criteria (possibly due to connectivity issues present in those networks), I've left the previous manual seeds for that network.