This is now tested as compatible with BIP78 as
implemented by BTCPayServer.
An additional config section [PAYJOIN] is added to
manage settings for fee control in payjoin as
described in the BIP. These settings are marked as
advanced usage as they're rather complex for users
to understand and the defaults should be very safe.
Upgrade python-bitcointx to 1.1.0:
Address requirements of python-bitcointx 1.1.0:
Specifically, the witness `utxo` field can no longer be
assumed to be of type CTxOut, so we should access the
CTxOut with the field witness_utxo and also when updating
the `utxo` field we now use `set_utxo()`.
Use PartiallySignedTransaction.get_fee() method.
Use PartiallySignedTransaction.set_utxo.
Additionally some minor typos/comment corrections and removal
of the now defunct `apply_freeze_signature`.
Add custom load location for libsecp where needed;
falls back to system installation if Joinmarket custom
installation is not found.
Decode error msg from server in payjoin
Cleanup test file test_proposals.txt (delete after test)
Human readable function names (names for human readable
conversions are now themselves human readable).
Remove unused get_*_vbyte functions and cleanup
Removes old unused files (electrum*.py).
Fixes core nohistory sync test to use both standard
wallet types, and fixes address import counter.
Fixes that test to use the right chain params so that
native segwit wallets can work in regtest with
nohistory mode.
Removes some now unneeded imports.
Fixes commontest.create_wallet_for_sync to hash all
parameters, including optional ones.
Replaces usage of binascii.hexlify with bintohex.
Note in particular that:
bitcoin.mktx in this PR now does support script
entries in outputs to account for nonstandard
destinations (as is needed for burn).
bitcoin.sign now supports p2wsh (as is needed
for timelocks).
Fix bug in Coins tab, get_utxos_by_mixdepth calls
Fix bug in jmbitcoin.mktx, now respects nVersion choice
Fix bug in tumbler restart wait
Fix bug in makercount for payjoin for fee check, set to 0:
The value of `options.makercount` is set to zero
so that the fee sanity check in the sendpayment script
operates approximately correctly (the receiver will bump
the fee to keep the fee rate the same if necessary).
Also the `bip79` variable is better named `payjoinurl`.
Fix bug in "freeze" context menu function in Qt
See:
https://github.com/bitcoin/bips/blob/master/bip-0078.mediawiki
Adds a new module jmclient.payjoin which implements
the full sender workflow from a BIP21 uri to a
payjoin broadcast, state is managed in JMPayjoinManager,
includes all checks as per documentation of
btcpayserver (and later, BIP78).
Added simple client and server implementations in
test/payjoinclient.py and test/payjoinserver.py
which allow a full end to end test on regtest.
Add TLS support to payjoin tests:
Note: the jmclient.payjoin module already
supports TLS by default (Agent object), but
here we add the ability to test without
certificate verification. Both test/payjoinclient.py
and test/payjoinserver.py now support TLS, but
the server needs a key and certificate in its
directory to run.
Adds BIP78 payjoin option to sendpayment.py
Users can use a bip21 uri with the "pj" field to
send a payment to a remote server.
Removes require_path_templates setting from KeyStore call
in PSBTWalletMixin.sign_psbt
Human readable representation for CTransaction
objects in jmbitcoin.secp256k1_transaction.py and for
PartiallySignedTransaction objects in jmclient.wallet.
PSBTWalletMixin, use of these in maker, taker, direct
send and in tests. Users should note that PSBT human
readable representations can in some cases be really
huge.
Update no-history-sync code:
This updates the new functionality in jmclient.wallet_utils
in the no-history-sync PR #444 to be compatible
with the python-bitcointx refactoring.
Remove all future/py2 compatibility code remaining:
This is in line with #525 and corrects erroneous
addition of more compatibility code.
Addresses all flake8 complaints (ununsed imports etc)
Addresses review of @dgpv
Addresses review of @kristapsk
This commit uses the now created PSBTWalletMixin and additionally
creates a SNICKERWalletMixin, and adds a SNICKERReceiver object
to jmclient. A test of the end to end workflow of create and then
co-sign a SNICKER coinjoin as per the draft BIP is in test_snicker.
Additional changes:
updated python-bitcointx dependency to >=1.0.5
Minor refactoring of callbacks in tests and additional redeem
script checks to PSBTWalletMixin.sign_psbt.
Note that this work replaces #403 .
Subclassed btcointx.CKeyBase to create a privkey type
that uses our signing code.
These are then used to allow signing of inputs in
transactions owned by our wallet.
Created a PSBTWalletMixin to perform the above function,
and added it to our three wallet types.
Wrote a detailed creation, sign and broadcast test
for a psbt using utxos from within and outside the wallet.
Detailed tests across all 3 wallet types for psbt
Tests cover also mixed inputs of different
types and owned/unowned. Direct send now exported to be used
in tests rather than only script usage, also supports returning
a tx object rather than only a txid.
Replaces core transaction, address, serialization
and sign functionality for Bitcoin with
python-bitcointx backend.
Removes bech32 and btscript
modules from jmbitcoin. Removes all string,
hex, binary conversion routines. A generic
hex/binary conversion now is added to jmbase.
Removes all transaction serialization and
deserialization routines. Removes the now
irrelevant test modules.
Remaining functions in jmbitcoin remove any parsing of
hex format, requiring callers to use binary only.
One additional test added, testing the remaining
function in secp256k1_transaction.py: the signing
of transactions. Deserialized form is now
bitcointx.CMutableTransaction.
For jmbase, in addition to the above, generic conversions
for utxos to and from strings is added, and a dynamic conversion
for AMP messages to binary-only. Within the code, utxos are
now only in (binarytxid, int) form, except where converted
for communcation.
Tthe largest part of the changes are
the modifications to jmbitcoin calls in jmclient;
as well as different encapsulation with CMutableTransaction,
there is also a removal of some but not all hex parsing;
it remains for rpc calls to Core and for AMP message
parsing. Backwards compatibility must be ensured so some
joinmarket protocol messages still use hex, and it is
also preserved in persistence of PoDLE data.
As part of this, some significant simplification of
certain legacy functions within the wallet has been done.
jmdaemon is entirely unaltered (save for one test which
simulates jmclient code).
Prior to this fix, doing Wallet->Generate in Qt and
choosing the mnemonic extension option creates an
erroneous error dialog on completion of wallet creation
(even though the wallet is created correctly), due to #565
which correctly decoded the binary mnemonic extension for
the Wallet->Show Seed function, but this was not correct
for the Wallet->Generate case.
The cause for this was the difference between the type of
the mnemonic extension variable when it was created, and
when it is received from storage.
This commit ensures consistency between the two cases by
making a newly created mnemonic extension variable binary.
Reasoning for this change: to ensure that Qt will show
a message and gracefully exit (or quit attempting to
load a wallet) in all 3 cases: on startup it show an
intelligible message if the RPC connection fails (as
before this PR), if the RPC fails while no wallet is
loaded and thus no wallet service is started, it should
show an intelligible error message when you attempt to
load a wallet and it fails, and finally it should show
an intelligible error message before quitting, if the rpc
connection fails during the period when the wallet is
already loaded.
By switching to an Exception instead of sys.exit, it does
mean that starting a yieldgenerator shows a stack trace,
but it also shows an intelligible error message (in red),
and this is command line, so UI requirements are less strong.
We preserve the "good" behaviour of no stack trace, but
only error message, if the rpc connection is lost during
running.
Fixes#442.
First, the CONNREFUSED socket error is handled in jsonrpc.
Second, we respond to this (but *not* to resets) with a reactor
shutdown in BitcoinCoreInterface.rpc(). This also necessitates
early-quitting in the calling function
(WalletService.transaction_monitor) since the reactor stop
will only stop future deferred calls, not the currently running
one. The obvious sys.exit approach is only used in startup,
because the reactor is not currently running at that point.
Also minor change to DummyBlockchainInterface for test.
Previously timelocked UTXOs would be returned by calls like
select_utxo() and get_utxos_by_mixdepth(). This caused annoyances
if trying to burn a single UTXO. It could also cause recently-
unlocked coins to accidently get spent, perhaps co-spent with other
coins.
This commit fixes that by freezing UTXOs with the coin control
feature whenever the wallet is sync'd. When the timelock of a coin
passes the user must explicitly use coin control to spend it.
Watchonly wallets use pubkeys instead of privkeys, but in a bit of
hack the functions previously called "_get_priv_from_path" would
actually return public keys for watchonly wallets. This could have
pretty terrible consequences one day, so functions like that have
been renamed to use the word "key" instead, which could be either
private or public.
The cryptoengine class BTC_Timelocked_P2WSH now implements
sign_transaction() which can be used to spend timelocked UTXOs.
FidelityBondMixin.is_timelocked_path() is now used outside the class
so its leading underscore has been removed.
Watch only wallets can now be created via wallet-tool. The wallets
store a bip32 xpub key from which all the public keys are generated.
Watch only wallets only store and display the zeroth mixdepth, which
is the only one needed for fidelity bonds.
The bip32 xpub key needed to create a watch only wallet is now
specially highlighted in the wallet-tool display, this is to help users
actually find it amongst all the other xpubs.
The field key_ident in the wallet class was previously generated using
private keys, which are not available in watch only wallets. So now
for fidelity bond wallets key_ident is generated using a public key.
Existing non-fidelity-bond wallets are unaffected
Fidelity bond wallets are intended to be used when at a later date
using fidelity bonds to greatly increase joinmarket's resistance to
sybil attacks. This commit adds support for timelocked addresses.
It allows users to optionally create wallet which support such
addresses. The synchronization code is modified to also scan for
timelocked addresses. The keypairs of the timelocked addresses go in
the newly created 2nd address type, where before the zeroth index were
receive addresses and first index was change.
The locktime dates are fixed at the first of each month for the next 30
years. This means users dont need to remember any dates, and so just
their seed phrase and wallet type will still be enough to recover all
funds. Each keypair used for timelocking requires an additional 360
addresses to be scanned for, which isn't a problem for Bitcoin Core.
Fidelity bonds are only stored in the zeroth mixdepth, as they are not
used in repeated coinjoins so theres no point having them in multiple
mixdepths.
Timelocked addresses don't use the get_new_script() family of functions
because they all assume that one index equals one address, and that
therefore it's possible to ask for a "next" address. For timelocked
addresses knowing the index is not enough to know the address, the
timestamp must be known too. Also once one address made of
(index, timestamp) is used you mustn't use that index and pubkey again,
even though all the other timelocks for that index/pubkey are unused.
This is for privacy reasons, as its equivalent to address reuse.
Previously an example of a BIP32 path
would be: m/wallet-type'/mixdepth'/internal/index
The 'internal' name referred to internal and external addresses (also
called change and receive). The renaming to 'address_type' is in
preparation to add more branches for timelocked addresses and burner
outputs.
The variable formally known as 'internal' is now no longer a boolean
but always an integer. This almost-always seemlessly fits because the
values False and Ture correspond to 0 and 1. The function
_get_internal_type therefore has no purpose anymore. Delete it.
Prior to this commit, if a tumbler coinjoin negotiation
failed in Phase 2, then the retry as per the logic in
taker_utils.tumbler_taker_finished_update would always
attempt to retry the transaction with those counterparties
that returned valid !sig responses. However this ignored
the case that all the counterparties responded validly,
but there was a mempool conflict in the created transaction.
After this commit, if it is detected that all counterparties
responded, it is assumed that a mempool conflict or similar
occurred with the transaction, and therefore it is better
to fallback to a schedule tweak and choose randomly again,
not to fix the counterparty set (which is likely to result
in failing again).
Prior to this commit, the function `exit()` was unavailable
due to the removal of the builtins import in 4bf1f50.
This led to NameError exceptions instead of the intended error
messages in Qt startup with wrong configuration (I did not
investigate if it could also happen on command line).
Fix was to make all exit() calls sys.exit().
AgoraIRC has problems at the time of writing. I found hackint
IRC which allows both clearnet and tor connections, and has
many populated channels so will presumably be well-maintained.
Adam Gibson
Kristaps Kaupe
chris-belcher