Upgrade python-bitcointx to 1.1.0:
Address requirements of python-bitcointx 1.1.0:
Specifically, the witness `utxo` field can no longer be
assumed to be of type CTxOut, so we should access the
CTxOut with the field witness_utxo and also when updating
the `utxo` field we now use `set_utxo()`.
Use PartiallySignedTransaction.get_fee() method.
Use PartiallySignedTransaction.set_utxo.
Additionally some minor typos/comment corrections and removal
of the now defunct `apply_freeze_signature`.
Add custom load location for libsecp where needed;
falls back to system installation if Joinmarket custom
installation is not found.
Decode error msg from server in payjoin
Cleanup test file test_proposals.txt (delete after test)
Human readable function names (names for human readable
conversions are now themselves human readable).
Remove unused get_*_vbyte functions and cleanup
Removes old unused files (electrum*.py).
Fixes core nohistory sync test to use both standard
wallet types, and fixes address import counter.
Fixes that test to use the right chain params so that
native segwit wallets can work in regtest with
nohistory mode.
Removes some now unneeded imports.
Fixes commontest.create_wallet_for_sync to hash all
parameters, including optional ones.
Replaces usage of binascii.hexlify with bintohex.
Update no-history-sync code:
This updates the new functionality in jmclient.wallet_utils
in the no-history-sync PR #444 to be compatible
with the python-bitcointx refactoring.
Remove all future/py2 compatibility code remaining:
This is in line with #525 and corrects erroneous
addition of more compatibility code.
Addresses all flake8 complaints (ununsed imports etc)
Addresses review of @dgpv
Addresses review of @kristapsk
Replaces core transaction, address, serialization
and sign functionality for Bitcoin with
python-bitcointx backend.
Removes bech32 and btscript
modules from jmbitcoin. Removes all string,
hex, binary conversion routines. A generic
hex/binary conversion now is added to jmbase.
Removes all transaction serialization and
deserialization routines. Removes the now
irrelevant test modules.
Remaining functions in jmbitcoin remove any parsing of
hex format, requiring callers to use binary only.
One additional test added, testing the remaining
function in secp256k1_transaction.py: the signing
of transactions. Deserialized form is now
bitcointx.CMutableTransaction.
For jmbase, in addition to the above, generic conversions
for utxos to and from strings is added, and a dynamic conversion
for AMP messages to binary-only. Within the code, utxos are
now only in (binarytxid, int) form, except where converted
for communcation.
Tthe largest part of the changes are
the modifications to jmbitcoin calls in jmclient;
as well as different encapsulation with CMutableTransaction,
there is also a removal of some but not all hex parsing;
it remains for rpc calls to Core and for AMP message
parsing. Backwards compatibility must be ensured so some
joinmarket protocol messages still use hex, and it is
also preserved in persistence of PoDLE data.
As part of this, some significant simplification of
certain legacy functions within the wallet has been done.
jmdaemon is entirely unaltered (save for one test which
simulates jmclient code).
Prior to this commit, the function `exit()` was unavailable
due to the removal of the builtins import in 4bf1f50.
This led to NameError exceptions instead of the intended error
messages in Qt startup with wrong configuration (I did not
investigate if it could also happen on command line).
Fix was to make all exit() calls sys.exit().
AgoraIRC has problems at the time of writing. I found hackint
IRC which allows both clearnet and tor connections, and has
many populated channels so will presumably be well-maintained.
To facilitate easier management by users and to
follow generally accepted standards, this PR moves
the following all to user home directory, subdir
.joinmarket :
joinmarket.cfg file
wallets/ directory
logs/ directory
cmtdata/ directory
commitmentlist file
User can override location with --datadir option.
An info message is added on startup showing location.
This is intended primarily to support running the
ob-watcher.py script from any environment that has no
blockchain access.
It could be used in debug and possibly recovery scenarios,
but this is explicitly unsupported (see comments in config).
Closes#274. Utxos are disabled if they are sent
to a reused address, and are below a threshold
set by the value `max_sats_freeze_reuse` in the
`POLICY` section of the config file. If the value
is -1, such utxos are always frozen irrespective of
the value.
Users are prompted with a warning level logging message
on CLI and a popup on Joinmarket-Qt. Such disabled utxos
can of course be re-enabled by the existing methods.
Also adds test case for address reuse freezing function.
No-history is a method for synchronizing a wallet by scanning the UTXO
set. It can be useful for checking whether seed phrase backups have
money on them before committing the time and effort required to
rescanning the blockchain. No-history sync is compatible with pruning.
The sync method cannot tell which empty addresses have been used, so
cannot guarentee avoidance of address reuse. For this reason no-history
sync disables wallet address generation and can only be used with
wallet-tool and for sending transactions without change addresses.
Since ConfigParser does not sufficiently support comments
in config objects, in order to support dynamic update of
the config from within a run, this PR edits the config
file at the same time as updating the config object, so
as to prevent the earlier problem that all comments in the
config file were lost whenever Qt updated the file with
the new config object.
Currently, only 'self' works until the other options for tx_broadcast are implemented. Update the docs accordingly, because there was someone wondering on IRC today why his transaction failed to broadcast; the reason was he changed this setting, which currently only works for 'self'.
If the other options are implemented (tagging #368 ), this documentation should be changed along with it.
While testing/debugging #105 we found that agora disconnects our taker about 1 minute after joining the pit, just before starting the CJ. Only that IRC server seems to be affected.
- update mktx() to allow optional locktime setting (and sequence)
- add a mk_shuffled_tx method to the wallet module
- add a P2EPTaker and P2EPMaker class (inherit from Taker, Maker)
- add a -T option to sendpayment script for doing payjoins
- add a receive_payjoin script for receivers.
- add payjoin tests in jmclient/test/test_payjoin.py
- add a custom utxo selection method select_one_utxo to support.py
- support bech32 wallets (SegwitWallet, p2wpkh) with native=true
in config POLICY for PayJoin and direct send (not Joinmarket CJ)
- add a PayJoin.md usage guide in docs/
- include version bytes in pubkey message for forward compat
- taker pays fees but controls size (utxo number and fee/kB)
- add P2WPKH fee estimator
- Enforce INFO level logging in payjoin
- refactors regtest config settings into one place
- bugfix: script_to_address vbyte argument is bytes not integer
Noticed (rare) problem on my maker today: it announced a wrong size. Investigating this identified this root cause:
1. a taker was building a big coinjoin with 8+ counterparties and a rather large amount, which meant it had *lots* of inputs
2. Building that TX and receiving all signatures from all makers took longer than the default timeout, that any maker waits for a coinjoin transaction to be broadcast (90 sec). Especially because there's a rate throttling to decrease the load on the IRC servers built in - so the taker intentionally slows down.
3. After around 100 seconds it was indeed broadcast, but the maker was not polling for that TX any more.
4. The TX confirmed. But the maker has no knowledge of this transaction. This means (potentially) he announces amounts that are not correct any more.
This is not a huge problem, but super easy to fix.
Proposed solution: increase timeout. I've checked my logs, I have seen this 'faulty' behavior 3 times in the last 2 months. It always was between 90-130 sec in those cases. Going from 90 sec to 180 should fix this.
Downsides of this change: can't think of any problems with the change. Anyone else?
Previous to this commit, the jmbitcoin package was accessed via
a file in jmclient `btc.py` which was originally added as an
interface to allow the client to use a non-jmbitcoin package to
provide the implementation; while this idea is useful, the way
it was implemented was not, moreover it is not currently used
and contained duplicated code that was unmanaged. Also, the
original usage of this was only by the electrum plugin, which
has currently been abandoned. This simplifies the code and
avoids spurious error messages. Note that most of the changes
are a result of pulling the logging function directly from the
jmbase package instead of indirectly via this interface (which
was unnecessary and not connected with jmbitcoin).
Adam Gibson
chris-belcher
Kristaps Kaupe
James Hilliard
AlexCato
Alex Cato
undeath
mecampbellsoup