This completes the task of enabling
network isolation by running the receiver
side using a hidden service in the daemon,
and communicating over AMP, as is already
the case for the sender.
Updates test_payjoin for daemon receiver.
Qt BIP78 receiver update for daemon.
This PR creates a client-daemon protocol for
the BIP78 sender, using the base protocol
`HTTPPassThrough` which provides tor and non-tor
agents with POST and GET request functionality.
As for Joinmarket coinjoins, the use of an in-process
daemon is the default option, but it can be isolated
by changing the `[DAEMON]` section of the config.
The receiver side of BIP78 will be addressed in a
future PR.
Also manually fire order creation in coinjoin tests.
This clarification and test change is required due
to the fact that LoopingCalls are designed to fire
immediately by default, before the reactor is
initialized (and therefore in a `running` state),
making it not possible to shutdown the reactor as
a result of events happening in that first call;
so we delay the first call of the maker's orderbook
populating code, so that if a no-coins error
occurs, it will actually shut down the reactor and
hence the whole yield generator program, as intended.
Prior to this commit, in case an RPC failure occurred when
accesing the block height, the program would continue but the
wallet would be in an un-writeable state (for command line
programs, specifically yield generators; for Qt the shutdown
would occur).
This commit slightly cleans up the process of shutting down,
ensuring that duplicate shutdown calls do not result in
stack traces. It also ensures that also for command line
programs, the application will immediately shutdown if the
regular heartbeat call to query the block height fails, as this
risks inconsistencies in the wallet (though the previous
situation luckily did not result in this as the call to
BaseWallet.close() resulted in the wallet being read only).
A future PR should develop a more sophisticated approach to
RPC call failures that may allow the program to wait.
stopservice
Prior to this commit, users setting the POLICY config
option `tx_broadcast` to anything other than `self` would
cause a crash after the merge of #536 due to a bin/hex
conversion failure (before this merge, the tx would simply
fail to broadcast).
This commit adds a `JMTXBroadcast` AMP command so that makers
can send arbitrary transactions from daemon to client, for
broadcast via the blockchain interface. This allows the
existing code in `taker.push()` to function correctly, after
fixing the bin/hex conversion bug. Hence users can now select
`random-peer` or `not-self` and the transaction will be
broadcast as expected according to the comments, and the
WalletService will react to the broadcast just as it does
currently for self-broadcast.
Note that this change will be ineffective if the counterparties
do not support it; the transaction will simply remain un-broadcast.
See:
https://github.com/bitcoin/bips/blob/master/bip-0078.mediawiki
Adds a new module jmclient.payjoin which implements
the full sender workflow from a BIP21 uri to a
payjoin broadcast, state is managed in JMPayjoinManager,
includes all checks as per documentation of
btcpayserver (and later, BIP78).
Added simple client and server implementations in
test/payjoinclient.py and test/payjoinserver.py
which allow a full end to end test on regtest.
Add TLS support to payjoin tests:
Note: the jmclient.payjoin module already
supports TLS by default (Agent object), but
here we add the ability to test without
certificate verification. Both test/payjoinclient.py
and test/payjoinserver.py now support TLS, but
the server needs a key and certificate in its
directory to run.
Adds BIP78 payjoin option to sendpayment.py
Users can use a bip21 uri with the "pj" field to
send a payment to a remote server.
Removes require_path_templates setting from KeyStore call
in PSBTWalletMixin.sign_psbt
Update no-history-sync code:
This updates the new functionality in jmclient.wallet_utils
in the no-history-sync PR #444 to be compatible
with the python-bitcointx refactoring.
Remove all future/py2 compatibility code remaining:
This is in line with #525 and corrects erroneous
addition of more compatibility code.
Addresses all flake8 complaints (ununsed imports etc)
Addresses review of @dgpv
Addresses review of @kristapsk
Replaces core transaction, address, serialization
and sign functionality for Bitcoin with
python-bitcointx backend.
Removes bech32 and btscript
modules from jmbitcoin. Removes all string,
hex, binary conversion routines. A generic
hex/binary conversion now is added to jmbase.
Removes all transaction serialization and
deserialization routines. Removes the now
irrelevant test modules.
Remaining functions in jmbitcoin remove any parsing of
hex format, requiring callers to use binary only.
One additional test added, testing the remaining
function in secp256k1_transaction.py: the signing
of transactions. Deserialized form is now
bitcointx.CMutableTransaction.
For jmbase, in addition to the above, generic conversions
for utxos to and from strings is added, and a dynamic conversion
for AMP messages to binary-only. Within the code, utxos are
now only in (binarytxid, int) form, except where converted
for communcation.
Tthe largest part of the changes are
the modifications to jmbitcoin calls in jmclient;
as well as different encapsulation with CMutableTransaction,
there is also a removal of some but not all hex parsing;
it remains for rpc calls to Core and for AMP message
parsing. Backwards compatibility must be ensured so some
joinmarket protocol messages still use hex, and it is
also preserved in persistence of PoDLE data.
As part of this, some significant simplification of
certain legacy functions within the wallet has been done.
jmdaemon is entirely unaltered (save for one test which
simulates jmclient code).
To facilitate easier management by users and to
follow generally accepted standards, this PR moves
the following all to user home directory, subdir
.joinmarket :
joinmarket.cfg file
wallets/ directory
logs/ directory
cmtdata/ directory
commitmentlist file
User can override location with --datadir option.
An info message is added on startup showing location.
Introduces WalletService object which is in control of
blockchain and wallet access.
The service manages a single transaction monitoring loop,
instead of multiple, and allows updates to the wallet from
external sources to be handled in real time, so that both Qt
and other apps (yg) can respond to deposits or withdrawals
automatically.
The refactoring also controls access to both wallet and
blockchain so that client apps (Taker, Maker) will not need
to be changed for future new versions e.g. client-side filtering.
Also updates and improves Wallet Tab behaviour in Qt (memory
of expansion state).
Additionally, blockchain sync is now --fast by default, with
the former default of detailed sync being renamed --recoversync.
The keys created in the bigstring.py module in case of message
length greater than 65535 were unicode and this was unsupported
by amp.py, this fixes the key to be ascii.
Additionally the tests in test_commands.py were failing silently
due to an unupgraded command syntax in JM_SETUP, it's now upgraded
and now tests, as originally, the large orderbook case, which now
passes due to the previously mentioned bugfix in bigstring for the
new Py3 compatibilty mode (builtins).
Adam Gibson
Kristaps Kaupe
jules23
James Hilliard
cindiv
dependabot[bot]