As per BIP78, new inputs or outputs must be
inserted, without changing the original order;
since we don't add extra outputs, we should leave
them in an unchanged order. Previously they were
shuffled.
Before this commit, if a BIP78 sender rejected the
PSBT sent by the receiver in the negotiation, JM's
receiver was hanging as the fallback broadcast was not
triggered. This commit corrects that error, and also
makes the fallback message visible on the GUI, and makes
it more ambiguous (since it not always clear whether
the transaction was broadcast or not, in this case).
The handle_unbroadcast_transaction fallback in Taker
is useful because Makers may not cooperate in broadcasting
a transaction, but it is only appropriate for the
random-peer option of `tx_broadcast`; for not-self the
user has chosen to avoid using their own IP, so this
commit simply abandons the attempt to broadcast the
transaction in that case.
The comments added to the config emphasize the problematic
aspect of this, which is that the tx may need to be
manually broadcast via another channel.
Prior to this commit, the payjoin receiver code
was signing a PSBT containing the utxo field
for every input, including the ones it did not
own, and transferring this to the sender.
However BIP78 specifies that, for inputs belonging
to the sender, no utxo field should be included.
This is corrected in this commit.
Also, ensure witness_utxo field is populated,
plus minor bugfixes related to presence of
NONWITNESS_UTXO field in provided payment PSBT.
Tested as being functional either with or without
NONWITNESS_UTXO field, for all-segwit inputs.
Also manually fire order creation in coinjoin tests.
This clarification and test change is required due
to the fact that LoopingCalls are designed to fire
immediately by default, before the reactor is
initialized (and therefore in a `running` state),
making it not possible to shutdown the reactor as
a result of events happening in that first call;
so we delay the first call of the maker's orderbook
populating code, so that if a no-coins error
occurs, it will actually shut down the reactor and
hence the whole yield generator program, as intended.
Prior to this commit, in case an RPC failure occurred when
accesing the block height, the program would continue but the
wallet would be in an un-writeable state (for command line
programs, specifically yield generators; for Qt the shutdown
would occur).
This commit slightly cleans up the process of shutting down,
ensuring that duplicate shutdown calls do not result in
stack traces. It also ensures that also for command line
programs, the application will immediately shutdown if the
regular heartbeat call to query the block height fails, as this
risks inconsistencies in the wallet (though the previous
situation luckily did not result in this as the call to
BaseWallet.close() resulted in the wallet being read only).
A future PR should develop a more sophisticated approach to
RPC call failures that may allow the program to wait.
stopservice
Prior to this commit, the cancel button remained
activated when the BIP78 payjoin processing had
completed, either successfully or unsuccessfully
which could be confusing for the user.
After this commit, when the processing is complete,
the JMBIP78ReceiverManager object fires the shutdown
callback, which Qt uses to signal the dialog, which
then updates to disable the Cancel button and show
the Close button.
Additionally, line breaks were added to make tooltips
more readable.
This commit implements a command line script and a GUI
dialog to receive a payment using the BIP78 protocol,
by setting up an ephemeral hidden service.
It also deprecates the pre-existing inter-Joinmarket
protocol for payjoin payments, since we now have
both sending and receiving support for BIP78. Thus,
much code in Maker, Taker and client-daemon protocol
is removed, as is some documentation in docs/PAYJOIN.md.
Also the script `sendpayment.py` is altered to support
only the BIP78 variant.
The test in jmclient/test/test_payjoin now implements
BIP78 over a TCP connection, while the custom tests in
test/payjoinserver.py can support hidden service based
tests, but the latter is not included in the test suite
and may not always work (it is only for manual
investigations).
The following features of BIP78 are supported:
minfeerate
additionalfeeoutputindex - but *only* for single
change output transactions
maxadditionalfeecontribution
The receiver does not have nor request payment
output substitution.
Utxo selection is no longer sophisticated, instead
we only choose a single utxo to keep the size
increase of the transaction minimal. Thus UIH is
not addressed at the moment.
Errors returned are in line with BIP78.
Sequence numbers are checked by receiver, and
kept identical if uniform, otherwise respected.
Receiver uses transaction monitor to shut down
when the payment is seen.
The workflow is almost entirely implemented in
jmclient/payjoin.py and the command line script
is in scripts/receive-payjoin.py. The setup, including
configuration changes for Tor, are documented in
docs/PAYJOIN.md, including a user guide video linked.
Prior to this commit, if non-self broadcast was enabled
but the counterparty chosen did not broadcast, the transaction
would remain unbroadcast.
After this commit, the Taker checks, after the configured
value of TIMEOUT.unconfirm_timeout_sec (default 90s), the
Taker will broadcast the transaction.
Also amended default config comment for this function.
Prior to this commit, users setting the POLICY config
option `tx_broadcast` to anything other than `self` would
cause a crash after the merge of #536 due to a bin/hex
conversion failure (before this merge, the tx would simply
fail to broadcast).
This commit adds a `JMTXBroadcast` AMP command so that makers
can send arbitrary transactions from daemon to client, for
broadcast via the blockchain interface. This allows the
existing code in `taker.push()` to function correctly, after
fixing the bin/hex conversion bug. Hence users can now select
`random-peer` or `not-self` and the transaction will be
broadcast as expected according to the comments, and the
WalletService will react to the broadcast just as it does
currently for self-broadcast.
Note that this change will be ineffective if the counterparties
do not support it; the transaction will simply remain un-broadcast.
Additionally, we add a note to wif_to_privkey()
in jmclient.cryptoenging.BTCEngine because the
'keytype' variable constructed there is not currently
usabe, and we remove a check of that keytype in
the add_utxo script used for PoDLE (it is not relevant).
In this commit, the jmclient.payjoin module now supports
sending payments to BIP21 URIs where the pj= parameter is
to a hidden service address.
Additionally, the test/payjoinclient and test/payjoinserver
modules are edited to support optionally testing payments to
an ephemeral hidden service.
Prior to this commit, the BIP78 payjoin would broadcast
a fallback transaction in case of the server returning an
error, but would not do so in case of server non-response.
After this commit, we add a 60 second timeout after which
the non-payjoin fallback is broadcast client side; see
BIP78 for an explanation of this behaviour.
This is now tested as compatible with BIP78 as
implemented by BTCPayServer.
An additional config section [PAYJOIN] is added to
manage settings for fee control in payjoin as
described in the BIP. These settings are marked as
advanced usage as they're rather complex for users
to understand and the defaults should be very safe.
Upgrade python-bitcointx to 1.1.0:
Address requirements of python-bitcointx 1.1.0:
Specifically, the witness `utxo` field can no longer be
assumed to be of type CTxOut, so we should access the
CTxOut with the field witness_utxo and also when updating
the `utxo` field we now use `set_utxo()`.
Use PartiallySignedTransaction.get_fee() method.
Use PartiallySignedTransaction.set_utxo.
Additionally some minor typos/comment corrections and removal
of the now defunct `apply_freeze_signature`.
Add custom load location for libsecp where needed;
falls back to system installation if Joinmarket custom
installation is not found.
Decode error msg from server in payjoin
Cleanup test file test_proposals.txt (delete after test)
Human readable function names (names for human readable
conversions are now themselves human readable).
Remove unused get_*_vbyte functions and cleanup
Removes old unused files (electrum*.py).
Fixes core nohistory sync test to use both standard
wallet types, and fixes address import counter.
Fixes that test to use the right chain params so that
native segwit wallets can work in regtest with
nohistory mode.
Removes some now unneeded imports.
Fixes commontest.create_wallet_for_sync to hash all
parameters, including optional ones.
Replaces usage of binascii.hexlify with bintohex.
Note in particular that:
bitcoin.mktx in this PR now does support script
entries in outputs to account for nonstandard
destinations (as is needed for burn).
bitcoin.sign now supports p2wsh (as is needed
for timelocks).
Fix bug in Coins tab, get_utxos_by_mixdepth calls
Fix bug in jmbitcoin.mktx, now respects nVersion choice
Fix bug in tumbler restart wait
Fix bug in makercount for payjoin for fee check, set to 0:
The value of `options.makercount` is set to zero
so that the fee sanity check in the sendpayment script
operates approximately correctly (the receiver will bump
the fee to keep the fee rate the same if necessary).
Also the `bip79` variable is better named `payjoinurl`.
Fix bug in "freeze" context menu function in Qt
See:
https://github.com/bitcoin/bips/blob/master/bip-0078.mediawiki
Adds a new module jmclient.payjoin which implements
the full sender workflow from a BIP21 uri to a
payjoin broadcast, state is managed in JMPayjoinManager,
includes all checks as per documentation of
btcpayserver (and later, BIP78).
Added simple client and server implementations in
test/payjoinclient.py and test/payjoinserver.py
which allow a full end to end test on regtest.
Add TLS support to payjoin tests:
Note: the jmclient.payjoin module already
supports TLS by default (Agent object), but
here we add the ability to test without
certificate verification. Both test/payjoinclient.py
and test/payjoinserver.py now support TLS, but
the server needs a key and certificate in its
directory to run.
Adds BIP78 payjoin option to sendpayment.py
Users can use a bip21 uri with the "pj" field to
send a payment to a remote server.
Removes require_path_templates setting from KeyStore call
in PSBTWalletMixin.sign_psbt
Jules Comte
Adam Gibson
nixbitcoin
Kristaps Kaupe
Tim Akinbo
James Hilliard
jules23