198117f14a CI: Remove editable installs (Kristaps Kaupe)
3641f1e1a4 Disable venv caching (Kristaps Kaupe)
Pull request description:
See https://github.com/JoinMarket-Org/joinmarket-clientserver/pull/1479#issuecomment-1536741199. This seems to be at least increasing chances of CI to succeed. I commented out not removed venv cache part as a reminder that this is not what we want in a long term.
`sed -i.bak` not just `sed -i` is to be compatible with both GNU (Linux) and macOS sed.
ACKs for top commit:
AdamISZ:
utACK 198117f14a
Tree-SHA512: 53c2135589be5e24df234e2d168d49a6d841769c4e4ec52227b7fb39c8c043400a9c402dea58fc28c6a1cc394fb580b32cda201f7289b6eb3bd33c8e70846822
4b2cbe9b34 Pin werkzeug dependency to 2.2.0 (Adam Gibson)
Pull request description:
Fixes#1483.
Prior to this commit, running the RPC API server using Klein failed due to a recent change in Werkzeug 2.3+, preventing any endpoint from returning anything but a 405 error. See
https://github.com/twisted/klein/pull/575.
After this commit we pin to 2.2.0 until such time as updated versions of werkzeug/Klein are reviewed by us.
ACKs for top commit:
kristapsk:
re-ACK 4b2cbe9b34
Tree-SHA512: ae6b256aabba2a6317c9cac1b00b164c8339046b891420d03cc11405590bb9dbb6663059ca4feb4c1081074b202c33e1212052ddd0713e995be3f627aa90f795
Fixes#1483.
Prior to this commit, running the RPC API server using Klein failed
due to a recent change in Werkzeug 2.3+, preventing any endpoint from
returning anything but a 405 error. See
https://github.com/twisted/klein/pull/575.
After this commit we pin to 2.2.0 until such time as updated versions of
werkzeug/Klein are reviewed by us.
e31e839c1d Add get_wallet_rescan_status() instead of getwalletinfo() for bci (Kristaps Kaupe)
Pull request description:
Noticed this when tried to rebase #1462 after merging of #1477. #1461 added public `getwalletinfo()` method to `BitcoinCoreInterface`, which was used by code outside of `jmclient/jmclient/blockchaininterface.py`. This is bad approach, as it relies on Bitcoin Core RPC `getwalletinfo` returned `dict`, which contains a lots of different stuff too, could lead to more problems in future introducing other blockchain interface classes. Let's instead have generic method returning just wallet rescan status. Also it now returns `Tuple[bool, Optional[Decimal]]` with rescan status percentage, if rescan is in progress, although that's not used by any other code for now.
ACKs for top commit:
AdamISZ:
utACK e31e839c1d , very much agree with the thinking here.
Tree-SHA512: 2d8c9b8157847e713838099d0f62dfcd5321c9498cf8453a9087407e2cd9c32906739c8e71460fc6ac6426662d2ac501261080fea08388d928933f788bda9a8d
3805c7af11 Remove ElectrumWalletInterface (Kristaps Kaupe)
Pull request description:
Remove broken `ElectrumWalletInterface` (we can re-introduce it later from history if somebody wants to fix it).
Part of splitting #1462 into smaller PRs for easier testing and reviews.
ACKs for top commit:
AdamISZ:
tACK 3805c7af11
Tree-SHA512: 8d11f3488c5e16058be187212b49e2cf11787b5fb9a1bb87f8949f76fa1569358932bea5bcf4fa3d0b0640534d87681d762b6e595f85a8e02584c9e438960c66
abbffefda0 ignore matched txs with invalid sPKs when scanning (openoms)
Pull request description:
Fixes the second edge case in #1471
completes #1473
ACKs for top commit:
AdamISZ:
tACK abbffefda0
kristapsk:
utACK abbffefda0
Tree-SHA512: 2789a1e604820726fc3261b9611e42ac19968f1ba8272512a715ace885672bca7692cc5ccfd59aa1469961660f2f410f437b406ce2dec319f52505750d9b7d7b
34c0c451d5 Add -l/--label-change option to sendpayment.py to automatically label change address (Kristaps Kaupe)
Pull request description:
Also added typehints to `direct_send()`.
ACKs for top commit:
AdamISZ:
tACK 34c0c451d5
Tree-SHA512: 0d17afde97cb436170f65ff7e0fabaa34d90d2bff8e2ecd19170569a62a1dc75c57f80d0dc159df0da277b4a842129fa998fe827177820500d2ff2b4e480575d
224de08437 CI: Bump used GitHub Actions to newer versions (Kristaps Kaupe)
Pull request description:
Updates [`actions/checkout`](https://github.com/actions/checkout), [`actions/setup-python`](https://github.com/actions/setup-python) and [`actions/cache`](https://github.com/actions/cache) to newest versions. Probably could help with random CI failures we are currently having.
ACKs for top commit:
AdamISZ:
ACK 224de08437
Tree-SHA512: b88850b7d2d50d1730d895dd195d8a7e139ec66d6cad7d055ee1495bd85692413f0934af6d697a618c8fee24993ad9b5aaeba6e6175a0639bf0418f4e63e32b9
7f4eaa9846 RPC-API: add ability to recover wallet (Adam Gibson)
Pull request description:
Fixes#1082.
This commit allows recovery of a wallet from a seedphrase with a new endpoint wallet/recover. 4 parameters are passed in, the same three as for wallet/create but also a bip39 seedphrase as the fourth argument.
Top commit has no ACKs.
Tree-SHA512: 7627a4107d37d47e973422fb1fab348d1e75426cbac6ac80008e08fb620e4ad6bb39c55bcd3ffdd9a727ba52797b12371ad9858a3d8ac4e4fa4d47f2a339fbf2
907f1b0603 Signal BIP125 opt-in RBF for non-cj sends by default (Kristaps Kaupe)
Pull request description:
This seems to be direction where everybody is going. Wasabi Wallet 2.0 does this, Electrum does this. I see no downsides here.
Before this you could signal opt-in RBF only from `sendpayment.py`, but not Qt GUI.
Will be more useful after #1019 is merged, but power users already can use other tools to construct replacement transactions.
ACKs for top commit:
AdamISZ:
tACK 907f1b0603
Tree-SHA512: 8909263e983fb17bac8cfea17532b391ad502f1a69e3079b1e1361177659628ede5de2eb85ed8fbbb7395bfd270a8b5f3d8916817033baa0b81561787bffe876
Fixes#1082:
This commit allows recovery of a wallet from a seedphrase
with a new endpoint wallet/recover. 4 parameters are passed in,
the same three as for wallet/create but also a bip39 seedphrase
as the fourth argument.
This commit also adds a rescanblockchain RPC call and status:
This adds a new endpoint /rescanblockchain which is (Core) wallet specific
(due to underlying JM architecture). This action is spawned in a thread,
since the bitcoind RPC call is blocking and can take a very long time.
To monitor the status of the rescan, an extra field `rescanning` is
added to the /session endpoint.
Also adds test of rpc wallet recovery
f6795e439f Add typehints to jmbitcoin/jmbitcoin/amount.py (Kristaps Kaupe)
Pull request description:
Was working on some other stuff and got to this and I think we should use typehints in future for anything we change, so splitted this off, should be easy to review.
ACKs for top commit:
AdamISZ:
utACK f6795e439f
Tree-SHA512: c8af41383212c06dd8b54081a0f2521762573e2b901a35cbc61ddd3f0a6226b4e2b93c217c079b85084dcb9cefa2aab7e11c22ea580cc604d71cce5a68db0a3b
373493fcce Refactor: move output descriptor code out of blockchaininterface (Kristaps Kaupe)
Pull request description:
This code was originally written before output_descriptors module was introduced. Also added tests.
ACKs for top commit:
AdamISZ:
utACK 373493fcce
Tree-SHA512: 72548414b6911f48dafe6e2cbcff3b7a5af03f5e11119106845ee1e434e87f4ef930ffd76c76bdfc775a2d80b370969975dd900c16eb1e3b84cbf54b66d901e3
cb2ad2b845 docs: add restart config to directory node service (openoms)
Pull request description:
Found it while testing that if Tor fails the start-dn.py exits and by default systemd does not restart it.
This small recommended change ensures that the process is restarted both in case of graceful and ungraceful exit and leaves 10 seconds between.
ACKs for top commit:
kristapsk:
utACK cb2ad2b845. Should not hurt in any way. Merging.
Tree-SHA512: d916553e091125a765b27a6c9a5bbb11b9a653d37af8e74e9521285a55ee91bf54f4a8c47bb4136762c6321036983c2921880db9c5e3ce4dbc0704d9a7414e13
f57e9fd42e Bump cryptography for 64-bit ARM and x86 (Kristaps Kaupe)
Pull request description:
Wheels for these are now being published, so there is no more problem with `rustc` dependency. But modern Linux distributions doesn't support old versions of `cryptography` anymore.
Using 37.0.4 not 39.0.1, as tests are failing with newer versions. Will open separate issue about that.
Resolves#1452.
Note that I had ability to test this only on 64-bit ARM (Python 3.10) and x86 (Python 3.9) on Linux, not other platforms.
Top commit has no ACKs.
Tree-SHA512: 6d1febfe65284652728aec2b09370fdde87b66358003cbad1ced5bfa984f466793d6b00f79b22ab3e2eda3826220102607dfa16c133cae6fb79e2314abb317d6
eeb362be6c added bumpfee.py script for bumping fees for rbf transactions (Tim Akinbo)
Pull request description:
Following the merging of JoinMarket-Org/joinmarket-clientserver#921, this PR adds the `bumpfee.py` script to enable fee bumping of replaceable transactions in JoinMarket.
Top commit has no ACKs.
Tree-SHA512: 7e85874997bb9db96326c7dbd2521dcfa364113d83c54d57051584756fa395bd902a7daccc9e6b1f44daaa92a30453f43707bf19e15f073171e4f4b1e16ad24c
refactoring to improve readability and testing
added support for taking the fee from multiple owned outputs as needed
added unit tests
removed same mixdepth restriction for inputs
added option for specifying which output to deduct fees from
added additional test cases
refactoring: moved tx_vsize to jmbitcoin package
added a sanity check to ensure that the transaction id supplied belongs to our wallet
fixed a linting error
Wheels for these are now being published, so there is no more problem
with rustc dependency. But modern Linux distributions doesn't support
old versions of cryptography anymore.
Using 37.0.4 not 39.0.1, as tests are failing with newer versions.
f8af2bf1f6 Fix libsecp256k1 commit ids in docs (Kristaps Kaupe)
Pull request description:
Noticed when reviewing #1444. Checked for all historical libsecp256k1 commits ids used historically with this:
```
$ git grep -h secp256k1_lib_tar= $(git rev-list --all) | sort | uniq
secp256k1_lib_tar="${secp256k1_version}.tar.gz"
secp256k1_lib_tar='0d9540b13ffcd7cd44cc361b8744b93d88aa76ba'
secp256k1_lib_tar='490022745164b56439688b0fc04f9bd43578e5c3'
secp256k1_lib_tar='f2d9aeae6d5a7c7fbbba8bbb38b1849b784beef7'
```
Then searched for old commit ids in whole repo (`grep -R commitid`).
Using wrong libsecp256k1 may cause problems because of ABI changes.
Top commit has no ACKs.
Tree-SHA512: ad7276aa282a31114f707fb541394a892a0ebc0f5d324925a11297ebde2bb6133efc3655f0d274e5f620723ec97e6e9fb01de4ee16807d0f63184945595e25e0
9574cfc42f release notes and version for v0.9.9 (Adam Gibson)
Pull request description:
Top commit has no ACKs.
Tree-SHA512: 924b68d1ec75af2b17fc5139403ecfa0a54fe69f3fa98131c0738e97356447a92a53dd6a5642e577451795b18d5a3fdcd39f89fa1814f20beead5e7e104ed191
1e159df8a9 Update bencoder.pyx to 3.0.1 (Kristaps Kaupe)
980edd6b2c CI: Add Python 3.11 (Kristaps Kaupe)
Pull request description:
All that was needed was upgrade of [bencoder.pyx to 3.0.1](https://pypi.org/project/bencoder.pyx/3.0.1/). It supports Python versions from 3.6 to 3.11.
Note that I haven't more carefully tested JM with Python 3.11, just made sure it installs and CI tests passes.
ACKs for top commit:
AdamISZ:
Based on above tests, and based on other discussoins, ACK 1e159df8a9
Tree-SHA512: b024359c0c55bdf677ea3161440351498fbb75997e771d80110cdfd44e75df5d1b2cce2d80c82d87239623d59807fae0161772d0bca2556f6e8d0bd8fd4f06f9
61b640073e Tests: bind to any first free TCP ports instead of hardcoded ones (Kristaps Kaupe)
Pull request description:
Without this, tests will fail if `joinmarketd` is already running in system, binding to default ports.
ACKs for top commit:
AdamISZ:
ACK 61b640073e
Tree-SHA512: f636abe3c6b14e042dfac4afbe839a4139b3a6d5d963d77691ac769f113b84051e07b232146651f6987338368a9dd3839e43b5d440e021a1b826c905ac810396
9a412d8be0 Connections to dnodes slow down (Adam Gibson)
Pull request description:
(edited from original comment since force-pushed a new set of code, though the intention is exactly the same)
Fixes https://github.com/JoinMarket-Org/joinmarket-clientserver/issues/1435.
Prior to this commit, if connections were immediately closed,
no backoff or jitter were applied to delay the next connection attempt,
resulting in too many connection attempts to directory nodes that the
client could not reach, and spam in the logs.
After this commit, we enforce that we always shut down the ClientService
on any disconnection trigger or connection failure, and only attempt to
reconnect with a manual backoff delay (4 seconds increasing by 50% for
20 attempts, plus a few seconds jitter), for directory nodes, while for
non-directory nodes we always give up on any failure.
Top commit has no ACKs.
Tree-SHA512: 26af339cd83fee2c3c94aef030e658d8cca58ca8bcfda5fa9a166f90d75b3c5c8bdd56c74d5c83222751f1a3561e3315e635a886bf534c9c49351b0712c12fad
See #1435.
Prior to this commit, if connections were immediately closed,
no backoff or jitter were applied to delay the next connection attempt,
resulting in too many connection attempts to directory nodes that the
client could not reach, and spam in the logs.
After this commit, we enforce that we always shut down the ClientService
on any disconnection trigger or connection failure, and only attempt to
reconnect with a manual backoff delay (4 seconds increasing by 50% for
20 attempts, plus a few seconds jitter), for directory nodes, while for
non-directory nodes we always give up on any failure. Max delay is a bit
above 3 hours.
c4d9b9264a tests: make setup fixtures optional. (PulpCattel)
Pull request description:
Currently, in tests, we always setup `bitcoind` and `miniircd` (this happens because of `autouse=True` in fixtures).
A lot of test modules do not require these slow and expensive setups. (`miniircd` in particular is AFAICT only used in 1 test module `jmdaemon/test/test_irc_messaging.py`).
After #1427, we now have indipendent fixtures for each component, so we can tell `Pytest` specifically what we need.
This PR uses [pytest.mark.usefixtures()](https://docs.pytest.org/en/7.2.x/how-to/fixtures.html#use-fixtures-in-classes-and-modules-with-usefixtures) to specify fixtures usage at the test module level.
Useful so that fixtures like `setup_regtest_bitcoind` can run *only* if we are planning to invoke a test that requires them.
For example, it's now possible to run:
```
pytest jmbitcoin/
```
This will skip all setup initialization because no test module in `/jmbitcoin` asks for it.
When running the entire test suite, there should be no difference in behaviour.
To see "behind the scene” of how `Pytest` schedules a test run, without actually running it, there's the very useful option `--setup-plan`.
ACKs for top commit:
AdamISZ:
Nice change, thanks. tACK c4d9b9264a
Tree-SHA512: 307d8a60a13cbbd9648efc489cb6da2ecf8e3d42772df09ec99a14fa822ad86ca53212364fd437c79ab718a8effe0de6b6810f6a8ee1df3ae1ad44181beec7a3
34fa1b6c63 Add link to my public orderbook mirror (Kristaps Kaupe)
Pull request description:
I've started hosting a permanently running JM orderbook on a VPS. Alternative to the one provided by @nixbitcoin, mine is .onion-only.
At least now there will be two public orderbook sites now, so that people can compare results. There are known issues with long running OB. Have scheduled mine to restart once per hour as a workaround.
Top commit has no ACKs.
Tree-SHA512: e5f314e1e4b84aa55cece6d6c7133684a920d24e0d78f65c0c30533e8029fa8ebb9ea4d2821fb95cd579a2add78b19c2993cd81f34e702d8634451eaa17733ef
Kristaps Kaupe
Adam Gibson
openoms
Tim Akinbo
akhavr
PulpCattel