This commit implements a command line script and a GUI
dialog to receive a payment using the BIP78 protocol,
by setting up an ephemeral hidden service.
It also deprecates the pre-existing inter-Joinmarket
protocol for payjoin payments, since we now have
both sending and receiving support for BIP78. Thus,
much code in Maker, Taker and client-daemon protocol
is removed, as is some documentation in docs/PAYJOIN.md.
Also the script `sendpayment.py` is altered to support
only the BIP78 variant.
The test in jmclient/test/test_payjoin now implements
BIP78 over a TCP connection, while the custom tests in
test/payjoinserver.py can support hidden service based
tests, but the latter is not included in the test suite
and may not always work (it is only for manual
investigations).
The following features of BIP78 are supported:
minfeerate
additionalfeeoutputindex - but *only* for single
change output transactions
maxadditionalfeecontribution
The receiver does not have nor request payment
output substitution.
Utxo selection is no longer sophisticated, instead
we only choose a single utxo to keep the size
increase of the transaction minimal. Thus UIH is
not addressed at the moment.
Errors returned are in line with BIP78.
Sequence numbers are checked by receiver, and
kept identical if uniform, otherwise respected.
Receiver uses transaction monitor to shut down
when the payment is seen.
The workflow is almost entirely implemented in
jmclient/payjoin.py and the command line script
is in scripts/receive-payjoin.py. The setup, including
configuration changes for Tor, are documented in
docs/PAYJOIN.md, including a user guide video linked.
In this commit, the jmclient.payjoin module now supports
sending payments to BIP21 URIs where the pj= parameter is
to a hidden service address.
Additionally, the test/payjoinclient and test/payjoinserver
modules are edited to support optionally testing payments to
an ephemeral hidden service.
This is now tested as compatible with BIP78 as
implemented by BTCPayServer.
An additional config section [PAYJOIN] is added to
manage settings for fee control in payjoin as
described in the BIP. These settings are marked as
advanced usage as they're rather complex for users
to understand and the defaults should be very safe.
Upgrade python-bitcointx to 1.1.0:
Address requirements of python-bitcointx 1.1.0:
Specifically, the witness `utxo` field can no longer be
assumed to be of type CTxOut, so we should access the
CTxOut with the field witness_utxo and also when updating
the `utxo` field we now use `set_utxo()`.
Use PartiallySignedTransaction.get_fee() method.
Use PartiallySignedTransaction.set_utxo.
Additionally some minor typos/comment corrections and removal
of the now defunct `apply_freeze_signature`.
Add custom load location for libsecp where needed;
falls back to system installation if Joinmarket custom
installation is not found.
Decode error msg from server in payjoin
Cleanup test file test_proposals.txt (delete after test)
Human readable function names (names for human readable
conversions are now themselves human readable).
Remove unused get_*_vbyte functions and cleanup
Removes old unused files (electrum*.py).
Fixes core nohistory sync test to use both standard
wallet types, and fixes address import counter.
Fixes that test to use the right chain params so that
native segwit wallets can work in regtest with
nohistory mode.
Removes some now unneeded imports.
Fixes commontest.create_wallet_for_sync to hash all
parameters, including optional ones.
Replaces usage of binascii.hexlify with bintohex.
See:
https://github.com/bitcoin/bips/blob/master/bip-0078.mediawiki
Adds a new module jmclient.payjoin which implements
the full sender workflow from a BIP21 uri to a
payjoin broadcast, state is managed in JMPayjoinManager,
includes all checks as per documentation of
btcpayserver (and later, BIP78).
Added simple client and server implementations in
test/payjoinclient.py and test/payjoinserver.py
which allow a full end to end test on regtest.
Add TLS support to payjoin tests:
Note: the jmclient.payjoin module already
supports TLS by default (Agent object), but
here we add the ability to test without
certificate verification. Both test/payjoinclient.py
and test/payjoinserver.py now support TLS, but
the server needs a key and certificate in its
directory to run.
Adds BIP78 payjoin option to sendpayment.py
Users can use a bip21 uri with the "pj" field to
send a payment to a remote server.
Removes require_path_templates setting from KeyStore call
in PSBTWalletMixin.sign_psbt
Update no-history-sync code:
This updates the new functionality in jmclient.wallet_utils
in the no-history-sync PR #444 to be compatible
with the python-bitcointx refactoring.
Remove all future/py2 compatibility code remaining:
This is in line with #525 and corrects erroneous
addition of more compatibility code.
Addresses all flake8 complaints (ununsed imports etc)
Addresses review of @dgpv
Addresses review of @kristapsk
Replaces core transaction, address, serialization
and sign functionality for Bitcoin with
python-bitcointx backend.
Removes bech32 and btscript
modules from jmbitcoin. Removes all string,
hex, binary conversion routines. A generic
hex/binary conversion now is added to jmbase.
Removes all transaction serialization and
deserialization routines. Removes the now
irrelevant test modules.
Remaining functions in jmbitcoin remove any parsing of
hex format, requiring callers to use binary only.
One additional test added, testing the remaining
function in secp256k1_transaction.py: the signing
of transactions. Deserialized form is now
bitcointx.CMutableTransaction.
For jmbase, in addition to the above, generic conversions
for utxos to and from strings is added, and a dynamic conversion
for AMP messages to binary-only. Within the code, utxos are
now only in (binarytxid, int) form, except where converted
for communcation.
Tthe largest part of the changes are
the modifications to jmbitcoin calls in jmclient;
as well as different encapsulation with CMutableTransaction,
there is also a removal of some but not all hex parsing;
it remains for rpc calls to Core and for AMP message
parsing. Backwards compatibility must be ensured so some
joinmarket protocol messages still use hex, and it is
also preserved in persistence of PoDLE data.
As part of this, some significant simplification of
certain legacy functions within the wallet has been done.
jmdaemon is entirely unaltered (save for one test which
simulates jmclient code).
To facilitate easier management by users and to
follow generally accepted standards, this PR moves
the following all to user home directory, subdir
.joinmarket :
joinmarket.cfg file
wallets/ directory
logs/ directory
cmtdata/ directory
commitmentlist file
User can override location with --datadir option.
An info message is added on startup showing location.
updated xenial and stretch dockerfiles since they bundle old versions of libsodium
refacatoring to make BlockchainInterface and BitcoinCoreNoHistoryInterface python2 compatible
add the -acceptnonstdtxn flag when invoking bitcoind for versions >= 0.19.0
refactor the use of nonlocal to make it compatible with python2
document the get_bitcoind_version method
Closes#274. Utxos are disabled if they are sent
to a reused address, and are below a threshold
set by the value `max_sats_freeze_reuse` in the
`POLICY` section of the config file. If the value
is -1, such utxos are always frozen irrespective of
the value.
Users are prompted with a warning level logging message
on CLI and a popup on Joinmarket-Qt. Such disabled utxos
can of course be re-enabled by the existing methods.
Also adds test case for address reuse freezing function.
Introduces WalletService object which is in control of
blockchain and wallet access.
The service manages a single transaction monitoring loop,
instead of multiple, and allows updates to the wallet from
external sources to be handled in real time, so that both Qt
and other apps (yg) can respond to deposits or withdrawals
automatically.
The refactoring also controls access to both wallet and
blockchain so that client apps (Taker, Maker) will not need
to be changed for future new versions e.g. client-side filtering.
Also updates and improves Wallet Tab behaviour in Qt (memory
of expansion state).
Additionally, blockchain sync is now --fast by default, with
the former default of detailed sync being renamed --recoversync.
- update mktx() to allow optional locktime setting (and sequence)
- add a mk_shuffled_tx method to the wallet module
- add a P2EPTaker and P2EPMaker class (inherit from Taker, Maker)
- add a -T option to sendpayment script for doing payjoins
- add a receive_payjoin script for receivers.
- add payjoin tests in jmclient/test/test_payjoin.py
- add a custom utxo selection method select_one_utxo to support.py
- support bech32 wallets (SegwitWallet, p2wpkh) with native=true
in config POLICY for PayJoin and direct send (not Joinmarket CJ)
- add a PayJoin.md usage guide in docs/
- include version bytes in pubkey message for forward compat
- taker pays fees but controls size (utxo number and fee/kB)
- add P2WPKH fee estimator
- Enforce INFO level logging in payjoin
- refactors regtest config settings into one place
- bugfix: script_to_address vbyte argument is bytes not integer
* parsing of scripts and addresses of all segwit types.
* ability to verify arbitrary tx inputs for all segwit types.
* simplify mktx syntax (only allow [],[] args)
* add p2wpkh and p2wsh spending test and fixes to sign calls in wallet
* simplify wallet signing calls in cryptoengine
* add p2wpkh engine, add bip84 wallet
Previous to this commit, the jmbitcoin package was accessed via
a file in jmclient `btc.py` which was originally added as an
interface to allow the client to use a non-jmbitcoin package to
provide the implementation; while this idea is useful, the way
it was implemented was not, moreover it is not currently used
and contained duplicated code that was unmanaged. Also, the
original usage of this was only by the electrum plugin, which
has currently been abandoned. This simplifies the code and
avoids spurious error messages. Note that most of the changes
are a result of pulling the logging function directly from the
jmbase package instead of indirectly via this interface (which
was unnecessary and not connected with jmbitcoin).
The test file added here can be run locally to see potential
errors in E2E coinjoin operation, but it is not yet ready to
be included in automated testing, since errors may occur silently
in this setup. This should be fixed, and then the test should
be added to travis, in a future PR.
If N makers are chosen and M fail to respond with sig,
this alteration to taker code allows restart of
entire transaction with that specific subset of makers (N-M)
that originally responded honestly.
As for maker, to use this feature set segwit = false
in the POLICY section of the config file.
This commit does *not* include support in the GUI.
Additionally, minor fixes to utility scripts for non-sw wallets,
and updated add-utxo tool to support non-sw wallets.
Bugfixes to wallet_utils to support non-sw wallets
and signmessage.
Adam Gibson
Kristaps Kaupe
jules23
jules23
undeath
James Hilliard
Tim Akinbo
fivepiece