This adds a bitset module that implements a BitSet<N> class, a variant
of std::bitset with a few additional features that cannot be implemented
in a wrapper without performance loss (specifically, finding first and
last bit set, or iterating over all set bits).

…ypes

This primarily adds the DepGraph class, which encapsulated precomputed
ancestor/descendant information for a given transaction cluster, with a
number of a utility features (inspectors for set feerates, computing
reduced parents/children, adding transactions, adding dependencies), which
will become needed in future commits.

This introduces a bespoke fuzzing-focused serialization format for DepGraphs,
and then tests that this format can represent any graph, roundtrips, and then
uses that to test the correctness of DepGraph itself.

This forms the basis for future fuzz tests that need to work with interesting
graph.

This is a class that encapsulated precomputes ancestor set feerates, and
presents an interface for getting the best remaining ancestor set.

Similar to AncestorCandidateFinder, this encapsulates the state needed for
finding good candidate sets using a search algorithm.

This adds a first version of the overall linearization interface, which given
a DepGraph constructs a good linearization, by incrementally including good
candidate sets (found using AncestorCandidateFinder and SearchCandidateFinder).

Add benchmarks for known bad graphs for the purpose of search (as
an upper bound on work per search iterations) and ancestor sorting
(as an upper bound on linearization work with no search iterations).

Add utility functions to DepGraph for finding connected components.

Before this commit, the worst case for linearization involves clusters which
break apart in several smaller components after the first candidate is
included in the output linearization.

Address this by never considering work items that span multiple components
of what remains of the cluster.

Switch to BFS exploration of the search tree in SearchCandidateFinder
instead of DFS exploration. This appears to behave better for real
world clusters.

As BFS has the downside of needing far larger search queues, switch
back to DFS temporarily when the queue grows too large.

To make search non-deterministic, change the BFS logic from always picking
the first queue item, randomly picking the first or second queue item.

This implements the LIMO algorithm for linearizing by improving an existing
linearization. See
https://delvingbitcoin.org/t/limo-combining-the-best-parts-of-linearization-search-and-merging
for details.

This is a requirement for a future commit, which will rely on quickly iterating
over transaction sets in decreasing individual feerate order.

…ion)

In each work item, keep track of a conservative overestimate of the best
possible feerate that can be reached from it, and then use these to avoid
exploring hopeless work items.

Keep track of which transactions in the graph have an individual
feerate that is better than the best included set so far. Others do not
need to be added to the pot set, as they cannot possibly help beating
best.

Automatically add topologically-valid subsets of the potential set pot
to inc. It can be proven that these must be part of the best reachable
topologically-valid set from that work item.

Emperically, this approach seems to be more efficient in common real-life
clusters, and does not change the worst case.

…ion)

Cache the potential set inside work items, and use it to skip part of
the computation of split-off work items from it.

Rather than evicting the transactions with the lowest descendant feerate,
instead evict transactions that have the lowest chunk feerate.

Once mining is implemented based on choosing transactions with highest chunk
feerate (see next commit), mining and eviction will be opposites, so that we
will evict the transactions that would be mined last.

The addition of a cluster size limit makes the CPFP carveout rule useless,
because carveout cannot be used to bypass the cluster size limit. Remove this
policy rule and update tests to no longer rely on the behavior.

With a total ordering on mempool transactions, we are now able to calculate a
transaction's mining score at all times. Use this to improve the RBF logic:

- we no longer enforce a "no new unconfirmed parents" rule

- we now require that the mempool's feerate diagram must improve in order
  to accept a replacement

TODO: update functional test feature_rbf.py to cover all our new scenarios.

Previously, transaction batches were first sorted by ancestor count and then
feerate, to ensure transactions are announced in a topologically valid order,
while prioritizing higher feerate transactions. Ancestor count is a crude
topological sort criteria, so replace this with linearization order so that the
highest feerate transactions (as would be observed by the mining algorithm) are
relayed before lower feerate ones, in a topologically valid way.

This also fixes a test that only worked due to the ancestor-count-based sort
order.

The mempool clusters and linearization permit sorting the mempool topologically
without making use of ancestor counts.

In preparation for removing ancestor data from CTxMemPoolEntry, recalculate the
ancestor statistics on demand wherever needed.

The cluster limits should be sufficient.

Remove a reference to GetCountWithDescendants() in preparation for removing
this function and the associated cached state from the mempool.

This is in preparation for removing the cached descendant state from the
mempool.

Minimal fix to the test that the RBF carveout doesn't apply in certain package
validation cases. Now that RBF carveout doesn't exist, we can just test that
the cluster count limit is respected (in preparation for removing the
descendant limit altogether).

Cluster size limits should be enough.

With the descendant size limits removed, replace the concept of "max number of
descendants of any ancestor of a given tx" with the cluster count of the cluster
that the transaction belongs to.

The new cluster mempool RBF rules take into account clusters sizes exactly, so
with the removal of descendant count enforcement this idea is obsolete.

…hed state

Also remove extra linearization that was happening and some logging

Update interface_zmq.py for new block connection behavior

Now that ancestor calculation never fails (due to ancestor/descendant limits
being eliminated), we can eliminate the error handling from
CalculateMemPoolAncestors.

interface_zmq test is broken

…l entry

The only place we still use the older interface is in policy/rbf.cpp, where
it's helpful to incrementally calculate descendants to avoid calculating too
many at once (or cluttering the CalculateDescendants interface with a
calculation limit).

TO DO: Rewrite unit tests for PV3C to not lie about mempool parents, so that we
can push down the parent calculation into v3_policy from validation.

Add benchmarks for:

  - mempool update time when blocks are found
  - adding a transaction
  - performing the mempool's RBF calculation
  - calculating mempool ancestors/descendants

Including test coverage for mempool eviction and expiry

This is in preparation for eliminating the block template building happening in
mini_miner, in favor of directly using the linearizations done in the mempool.