Fixes#1614.
Prior to this commit, if data in the persisted cache in the wallet file
were wrong (should be a very extraordinary case), then the joinmarket
code would have to crash with a cache invalid warning. After this
commit, in such an extraordinary case, the option exists to invalidate
or remove the cache on startup, so that it can be rebuilt from scratch.
This is done with a config var wallet_caching_disabled in the POLICY
section.
Hoist _populate_script_map from BIP32Wallet into BaseWallet, rename it
to _populate_maps, and have it populate the new _addr_map in addition to
the existing _script_map. Have the constructor of each concrete wallet
subclass pass to _populate_maps the paths it contributes. Additionally,
do not implement yield_known_paths by iterating over _script_map, but
rather have each wallet subclass contribute its own paths to the
generator returned by yield_known_paths.
Sometimes calling code is only interested in the balance or UTXOs at a
single mixdepth. In these cases, it is wasteful to get the balance or
UTXOs at all mixdepths, only to throw away the returned information
about all but the single mixdepth of interest. Implement new methods in
BaseWallet to get the balance or UTXOs at a single mixdepth.
Also, correct an apparent oversight due to apparently misplaced
indentation: the maxheight parameter of get_balance_by_mixdepth was
ignored unless the include_disabled parameter was passed as False. It
appears that the intention was for include_disabled and maxheight to be
independent filters on the returned information.
This allows running tests that do not require, e.g., bitcoind, without having to run these setup fixtures at all.
Signed-off-by: PulpCattel <PulpCattel@users.noreply.github.com>
Adds code to validate bech32m encoded taproot addresses
and scriptPubKeys. The underlying functions are in
python-bitcointx upcoming version 1.1.3.
Adds tests at both the basic address validation layer,
and also at the layer of sending transactions to
taproot outputs.
This commit changes how timelocked addresses are created from the seed and bip32 tree.
A good thing to do would be to have each locktime (e.g. 1st jan 2020,
1st feb 2020, 1st march 2020, etc) actually use a different pubkey from
the HD tree (i.e. ("m/84'/1'/0'/2/0" + 1st jan 2020) ("m/84'/1'/0'/2/1" + 1st feb 2020), etc).
This now means that the sync code doesnt need to know what keys have been associated with
a fidelity bond to scan for the next one. Previously when a user funded a single timelocked
address, the wallet will generate _another_ pubkey and import _another_ ~960 addresses, so
funding one address would actually mean watching and generating ~1920 addresses not ~960.
This should help with the problem found by some people that fidelity bond wallets are slower
to sync. Other optimizations are possible but the structure of fidelity bond wallets will
probably be fixed for decades, so this change is worth doing now.
Prior to this commit, several test functions were using
"True" to flag internal and "False" to flag external for
the HD branch for the wallet, but we now use BaseWallet vars
ADDRESS_TYPE_[IN/EX]TERNAL (1/0), so this
is changed to explicitly reference those. There is no change
to the live code (which calls get_[internal/external]_addr).
In addition _index_cache updates in wallet are protected
with a wrapper function to ensure that the branch requested
is valid.
Tests pass both before and after this change.
Note in particular that:
bitcoin.mktx in this PR now does support script
entries in outputs to account for nonstandard
destinations (as is needed for burn).
bitcoin.sign now supports p2wsh (as is needed
for timelocks).
Update no-history-sync code:
This updates the new functionality in jmclient.wallet_utils
in the no-history-sync PR #444 to be compatible
with the python-bitcointx refactoring.
Remove all future/py2 compatibility code remaining:
This is in line with #525 and corrects erroneous
addition of more compatibility code.
Addresses all flake8 complaints (ununsed imports etc)
Addresses review of @dgpv
Addresses review of @kristapsk
Replaces core transaction, address, serialization
and sign functionality for Bitcoin with
python-bitcointx backend.
Removes bech32 and btscript
modules from jmbitcoin. Removes all string,
hex, binary conversion routines. A generic
hex/binary conversion now is added to jmbase.
Removes all transaction serialization and
deserialization routines. Removes the now
irrelevant test modules.
Remaining functions in jmbitcoin remove any parsing of
hex format, requiring callers to use binary only.
One additional test added, testing the remaining
function in secp256k1_transaction.py: the signing
of transactions. Deserialized form is now
bitcointx.CMutableTransaction.
For jmbase, in addition to the above, generic conversions
for utxos to and from strings is added, and a dynamic conversion
for AMP messages to binary-only. Within the code, utxos are
now only in (binarytxid, int) form, except where converted
for communcation.
Tthe largest part of the changes are
the modifications to jmbitcoin calls in jmclient;
as well as different encapsulation with CMutableTransaction,
there is also a removal of some but not all hex parsing;
it remains for rpc calls to Core and for AMP message
parsing. Backwards compatibility must be ensured so some
joinmarket protocol messages still use hex, and it is
also preserved in persistence of PoDLE data.
As part of this, some significant simplification of
certain legacy functions within the wallet has been done.
jmdaemon is entirely unaltered (save for one test which
simulates jmclient code).
Watchonly wallets use pubkeys instead of privkeys, but in a bit of
hack the functions previously called "_get_priv_from_path" would
actually return public keys for watchonly wallets. This could have
pretty terrible consequences one day, so functions like that have
been renamed to use the word "key" instead, which could be either
private or public.
Previously an example of a BIP32 path
would be: m/wallet-type'/mixdepth'/internal/index
The 'internal' name referred to internal and external addresses (also
called change and receive). The renaming to 'address_type' is in
preparation to add more branches for timelocked addresses and burner
outputs.
The variable formally known as 'internal' is now no longer a boolean
but always an integer. This almost-always seemlessly fits because the
values False and Ture correspond to 0 and 1. The function
_get_internal_type therefore has no purpose anymore. Delete it.
Fixes#477. Removes nonstandard script support
from 0.19+ bitcoind configuration for regtest and
changes scripts in test_wallet and test_tx_creation
to make them standard.
Those names as confusing. They could imply that the function obtains
a path or address given a script. To help the code be more
self-documenting I add the verb from.
To facilitate easier management by users and to
follow generally accepted standards, this PR moves
the following all to user home directory, subdir
.joinmarket :
joinmarket.cfg file
wallets/ directory
logs/ directory
cmtdata/ directory
commitmentlist file
User can override location with --datadir option.
An info message is added on startup showing location.
Introduces WalletService object which is in control of
blockchain and wallet access.
The service manages a single transaction monitoring loop,
instead of multiple, and allows updates to the wallet from
external sources to be handled in real time, so that both Qt
and other apps (yg) can respond to deposits or withdrawals
automatically.
The refactoring also controls access to both wallet and
blockchain so that client apps (Taker, Maker) will not need
to be changed for future new versions e.g. client-side filtering.
Also updates and improves Wallet Tab behaviour in Qt (memory
of expansion state).
Additionally, blockchain sync is now --fast by default, with
the former default of detailed sync being renamed --recoversync.
Wallet persists utxo metadata; currently only
contains a field 'disabled' indexed by utxo.
User can switch this on or off (enabled) via
wallet-tool 'freeze' method.
Disabled utxos will not be used in coin
selection in any transaction.
Wallet still displays all utxo balances in
display method (and in GUI).
Add tests of disabling to test_utxomanager
Add Coins tab to Qt, with freeze/unfreeze feature.
Coins tab shows updated utxo info txid:n, amt, address and
enabled/disabled, which can be toggled from right click menu.
* parsing of scripts and addresses of all segwit types.
* ability to verify arbitrary tx inputs for all segwit types.
* simplify mktx syntax (only allow [],[] args)
* add p2wpkh and p2wsh spending test and fixes to sign calls in wallet
* simplify wallet signing calls in cryptoengine
* add p2wpkh engine, add bip84 wallet
Previous to this commit, the jmbitcoin package was accessed via
a file in jmclient `btc.py` which was originally added as an
interface to allow the client to use a non-jmbitcoin package to
provide the implementation; while this idea is useful, the way
it was implemented was not, moreover it is not currently used
and contained duplicated code that was unmanaged. Also, the
original usage of this was only by the electrum plugin, which
has currently been abandoned. This simplifies the code and
avoids spurious error messages. Note that most of the changes
are a result of pulling the logging function directly from the
jmbase package instead of indirectly via this interface (which
was unnecessary and not connected with jmbitcoin).
Adam Gibson
Matt Whitlock
roshii
PulpCattel
Kristaps Kaupe
undeath
chris-belcher
Jules Comte
James Hilliard