Fixes#1614.
Prior to this commit, if data in the persisted cache in the wallet file
were wrong (should be a very extraordinary case), then the joinmarket
code would have to crash with a cache invalid warning. After this
commit, in such an extraordinary case, the option exists to invalidate
or remove the cache on startup, so that it can be rebuilt from scratch.
This is done with a config var wallet_caching_disabled in the POLICY
section.
Add a validate_cache parameter to the five principal caching methods:
- _get_key_from_path
- _get_keypair_from_path
- _get_pubkey_from_path
- get_script_from_path
- get_address_from_path
and to the five convenience methods that wrap the above:
- get_script
- get_addr
- script_to_addr
- get_new_script
- get_new_addr
The value of this new parameter defaults to False in all but the last
two methods, where we are willing to sacrifice speed for the sake of
extra confidence in the correctness of *new* scripts and addresses to
be used for new deposits and new transactions.
Deriving private keys from BIP32 paths, public keys from private keys,
scripts from public keys, and addresses from scripts are some of the
most CPU-intensive tasks the wallet performs. Once the wallet inevitably
accumulates thousands of used paths, startup times become painful due to
needing to re-derive these data items for every used path in the wallet
upon every startup. Introduce a persistent cache to avoid the need to
re-derive these items every time the wallet is opened.
Introduce _get_keypair_from_path and _get_pubkey_from_path methods to
allow cached public keys to be used rather than always deriving them on
the fly.
Change many code paths that were calling CPU-intensive methods of
BTCEngine so that instead they call _get_key_from_path,
_get_keypair_from_path, _get_pubkey_from_path, get_script_from_path,
and/or get_address_from_path, all of which can take advantage of the new
cache.
Hoist _populate_script_map from BIP32Wallet into BaseWallet, rename it
to _populate_maps, and have it populate the new _addr_map in addition to
the existing _script_map. Have the constructor of each concrete wallet
subclass pass to _populate_maps the paths it contributes. Additionally,
do not implement yield_known_paths by iterating over _script_map, but
rather have each wallet subclass contribute its own paths to the
generator returned by yield_known_paths.
Sometimes calling code is only interested in the balance or UTXOs at a
single mixdepth. In these cases, it is wasteful to get the balance or
UTXOs at all mixdepths, only to throw away the returned information
about all but the single mixdepth of interest. Implement new methods in
BaseWallet to get the balance or UTXOs at a single mixdepth.
Also, correct an apparent oversight due to apparently misplaced
indentation: the maxheight parameter of get_balance_by_mixdepth was
ignored unless the include_disabled parameter was passed as False. It
appears that the intention was for include_disabled and maxheight to be
independent filters on the returned information.
* Add support for testing against multiple Bitcoin Core versions, add
0.18.0 (oldest officially support for test suite) and 25.0 (latest
stable release). Previously we tested against hardcoded 0.19.1. IMO
testing against oldest supported and latest makes sense, if tests pass
on both, they should pass on releases inbetween too, unless there is
some behaviour affecting us changed twice.
* Update actions/cache from v2 to v3.
Also changed Core downloads from bitcoin.org to bitcoincore.org.
refactoring to improve readability and testing
added support for taking the fee from multiple owned outputs as needed
added unit tests
removed same mixdepth restriction for inputs
added option for specifying which output to deduct fees from
added additional test cases
refactoring: moved tx_vsize to jmbitcoin package
added a sanity check to ensure that the transaction id supplied belongs to our wallet
fixed a linting error
This allows running tests that do not require, e.g., bitcoind, without having to run these setup fixtures at all.
Signed-off-by: PulpCattel <PulpCattel@users.noreply.github.com>
zebra-lucky
Kristaps Kaupe
Adam Gibson
Matt Whitlock
roshii
Tim Akinbo
PulpCattel