1e159df8a9 Update bencoder.pyx to 3.0.1 (Kristaps Kaupe)
980edd6b2c CI: Add Python 3.11 (Kristaps Kaupe)
Pull request description:
All that was needed was upgrade of [bencoder.pyx to 3.0.1](https://pypi.org/project/bencoder.pyx/3.0.1/). It supports Python versions from 3.6 to 3.11.
Note that I haven't more carefully tested JM with Python 3.11, just made sure it installs and CI tests passes.
ACKs for top commit:
AdamISZ:
Based on above tests, and based on other discussoins, ACK 1e159df8a9
Tree-SHA512: b024359c0c55bdf677ea3161440351498fbb75997e771d80110cdfd44e75df5d1b2cce2d80c82d87239623d59807fae0161772d0bca2556f6e8d0bd8fd4f06f9
61b640073e Tests: bind to any first free TCP ports instead of hardcoded ones (Kristaps Kaupe)
Pull request description:
Without this, tests will fail if `joinmarketd` is already running in system, binding to default ports.
ACKs for top commit:
AdamISZ:
ACK 61b640073e
Tree-SHA512: f636abe3c6b14e042dfac4afbe839a4139b3a6d5d963d77691ac769f113b84051e07b232146651f6987338368a9dd3839e43b5d440e021a1b826c905ac810396
9a412d8be0 Connections to dnodes slow down (Adam Gibson)
Pull request description:
(edited from original comment since force-pushed a new set of code, though the intention is exactly the same)
Fixes https://github.com/JoinMarket-Org/joinmarket-clientserver/issues/1435.
Prior to this commit, if connections were immediately closed,
no backoff or jitter were applied to delay the next connection attempt,
resulting in too many connection attempts to directory nodes that the
client could not reach, and spam in the logs.
After this commit, we enforce that we always shut down the ClientService
on any disconnection trigger or connection failure, and only attempt to
reconnect with a manual backoff delay (4 seconds increasing by 50% for
20 attempts, plus a few seconds jitter), for directory nodes, while for
non-directory nodes we always give up on any failure.
Top commit has no ACKs.
Tree-SHA512: 26af339cd83fee2c3c94aef030e658d8cca58ca8bcfda5fa9a166f90d75b3c5c8bdd56c74d5c83222751f1a3561e3315e635a886bf534c9c49351b0712c12fad
See #1435.
Prior to this commit, if connections were immediately closed,
no backoff or jitter were applied to delay the next connection attempt,
resulting in too many connection attempts to directory nodes that the
client could not reach, and spam in the logs.
After this commit, we enforce that we always shut down the ClientService
on any disconnection trigger or connection failure, and only attempt to
reconnect with a manual backoff delay (4 seconds increasing by 50% for
20 attempts, plus a few seconds jitter), for directory nodes, while for
non-directory nodes we always give up on any failure. Max delay is a bit
above 3 hours.
c4d9b9264a tests: make setup fixtures optional. (PulpCattel)
Pull request description:
Currently, in tests, we always setup `bitcoind` and `miniircd` (this happens because of `autouse=True` in fixtures).
A lot of test modules do not require these slow and expensive setups. (`miniircd` in particular is AFAICT only used in 1 test module `jmdaemon/test/test_irc_messaging.py`).
After #1427, we now have indipendent fixtures for each component, so we can tell `Pytest` specifically what we need.
This PR uses [pytest.mark.usefixtures()](https://docs.pytest.org/en/7.2.x/how-to/fixtures.html#use-fixtures-in-classes-and-modules-with-usefixtures) to specify fixtures usage at the test module level.
Useful so that fixtures like `setup_regtest_bitcoind` can run *only* if we are planning to invoke a test that requires them.
For example, it's now possible to run:
```
pytest jmbitcoin/
```
This will skip all setup initialization because no test module in `/jmbitcoin` asks for it.
When running the entire test suite, there should be no difference in behaviour.
To see "behind the scene” of how `Pytest` schedules a test run, without actually running it, there's the very useful option `--setup-plan`.
ACKs for top commit:
AdamISZ:
Nice change, thanks. tACK c4d9b9264a
Tree-SHA512: 307d8a60a13cbbd9648efc489cb6da2ecf8e3d42772df09ec99a14fa822ad86ca53212364fd437c79ab718a8effe0de6b6810f6a8ee1df3ae1ad44181beec7a3
34fa1b6c63 Add link to my public orderbook mirror (Kristaps Kaupe)
Pull request description:
I've started hosting a permanently running JM orderbook on a VPS. Alternative to the one provided by @nixbitcoin, mine is .onion-only.
At least now there will be two public orderbook sites now, so that people can compare results. There are known issues with long running OB. Have scheduled mine to restart once per hour as a workaround.
Top commit has no ACKs.
Tree-SHA512: e5f314e1e4b84aa55cece6d6c7133684a920d24e0d78f65c0c30533e8029fa8ebb9ea4d2821fb95cd579a2add78b19c2993cd81f34e702d8634451eaa17733ef
1370c12a5a Use different jm_test_datadir for each local user (Kristaps Kaupe)
Pull request description:
Otherwise will have permission denied errors when trying to run tests from a different local users on the same system.
ACKs for top commit:
AdamISZ:
tACK 1370c12a5a
Tree-SHA512: f99ca2b10fc7a35f6001e9191ad5374222fd19072a48bd0a8a0c516c69a2d8b53233b8c9308d90f48a3d4419995c75ab8662dec26b47aba230d5ed0ca838ca0f
This allows running tests that do not require, e.g., bitcoind, without having to run these setup fixtures at all.
Signed-off-by: PulpCattel <PulpCattel@users.noreply.github.com>
63d74bc0f1 Call num_cores() only after deps_install() (Kristaps Kaupe)
Pull request description:
There may be no Python available before that.
Alternative to #1419.
Top commit has no ACKs.
Tree-SHA512: 7a377fec8b819458e678ef2fd869bd6692f404772f4b42c4fd8caad530b0d49c96010841362b9279488781e914d917897589be46f562f8c5c5025a17f36542c4
This may or may not fix#1057 and other test failures. Since the network
delay in receiving the message is unknown (but expected to be very
small, usually), we keep trying to receive notification of the
transaction message in the websocket test 10 times for a total of 2
seconds (on the assumption that if it takes longer than that, something
else is wrong).
7aebed728b Remove jmdirjmioywe2s5jad7ts6kgcqg66rj6wujj6q77n6wbdrgocqwexzid.onion directory node from default config (Kristaps Kaupe)
Pull request description:
It has been discussed on Telegram. This node seems to be permanently down for a long period of time.
ACKs for top commit:
AdamISZ:
ACK 7aebed728b
Tree-SHA512: f0d32c53e3647903cce93476a03078acf318d7f78830edb8fae07408daf121cd9056b513bfa809e6eaf8d4827e4a2520e75094ff9471f0804b3e14ec2b999b9f
Prior to this commit, p2wsh inputs from fidelity bonds resulted in
miscalculation of transaction fees, even in cases where the exact set of
inputs were known (such as a direct send).
In this commit we change the estimation to a model in which the caller
of jmbitcoin.secp256k1_transaction.estimate_tx_size must specify a list
of types, one for each input to the transaction, and the same for
outputs. In some cases, the caller of the function uses the default
script type of the wallet, but in other cases where the caller can know
the exact types of each utxo used as input, and each destination used as
output, they are specified explicitly. In particular, the use of
fidelity bond outputs as input to transactions can be accounted for.
Currently this is only done in direct send payments; coinjoins still
fall back to assuming all inputs the same type (note that it is not
possible to use fidelity bond utxos as inputs to coinjoins).
Note also that the burn destination calculation in direct send is
removed, since it is not used, so the maintenance burden is best
avoided.
22c13b0ab7 Remove unused code instead of commenting out (Kristaps Kaupe)
b928713880 Add ShellCheck linter script (Kristaps Kaupe)
4f0eebc68d Apply all current shellcheck suggestions to rest of the scripts (Kristaps Kaupe)
f0b9872ff9 Apply all current shellcheck suggestions to install.sh (Kristaps Kaupe)
Pull request description:
Based on #1175, but also covers `install.sh` changes since then, all other shell scripts and adds ShellCheck linter script. Not adding this to CI for now. I'm thinking we could split off all the linting to separate job, don't see a point in running them under all OS / Python version combinations.
Top commit has no ACKs.
Tree-SHA512: 936320280f1ec9d8aa315c70be4d33fcb06d324531a17b30987c0be308d5351aa45dfac05cecadca8e9977a8f797b62ccd91d5d5fe92c5d0eacc6ebdf4dcd1c6
bfb0e25861 Account for missing nick fields in OnionPeer (Adam Gibson)
Pull request description:
Prior to this commit, if a message (like a disconnection control message), was to be sent to live peers, regarding another peer that did not have a nick field set, a crash would occur in directory nodes sending this message, because the function
OnionPeer.get_nick_peerlocation_ser() assumed the existence of the nick field, and raised the OnionPeerError exception if it did not exist.
After this commit, the raising of this Exception is caught when calling OnionPeer.get_nick_peerlocation_ser, so that a peer in this state is just not added to the list of peers to be sent to live peers. This condition is caused specifically by the fact that said peer has not completed a handshake (which must always pass the value of the nick), and so the other live peers will not yet be aware of it.
ACKs for top commit:
kristapsk:
cr utACK bfb0e25861
Tree-SHA512: 7bf1fe216bc7088b986a9f33c65fa07e25a5ec09a42cafc7496567b628a1e3b03e91b6554b9d66f6e0603d8ee15a729f6af76fed4a75ecbdf385a78d8cd4100b
3c8f247d1a CI: Run tests on both Linux and macOS (Kristaps Kaupe)
Pull request description:
If devs have problems with accessing Mac's for tests, let GitHub Actions CI do the basic tesing for us.
Top commit has no ACKs.
Tree-SHA512: a36e46c03704ffde8588342d200e4b7c05df72b5bc3d0046027d793a9058e646c4fe3c45860ccf808f87260ade0a8778ea7ac3531eacd2db43402fa0b879e69b
Prior to this commit, if a message (like a disconnection control
message), was to be sent to live peers, regarding another peer that did
not have a nick field set, a crash would occur in directory nodes
sending this message, because the function
OnionPeer.get_nick_peerlocation_ser() assumed the existence of the nick
field, and raised the OnionPeerError exception if it did not exist.
After this commit, the raising of this Exception is caught when calling
OnionPeer.get_nick_peerlocation_ser, so that a peer in this state is
just not added to the list of peers to be sent to live peers. This
condition is caused specifically by the fact that said peer has not
completed a handshake (which must always pass the value of the nick),
and so the other live peers will not yet be aware of it.
11ddec7841 docs: Remove now-confusing mentions of virtualenv (Carl Dong)
6e3b6eca5d Dockerfiles: Use Python venv instead of virtualenv (Carl Dong)
8bbca60641 install: Use python3's venv module when available (Carl Dong)
Pull request description:
Python 3.3 and above provide a standard venv module which provides all the functionality that install.sh needs from virtualenv.
Detect its existence and use it, fallback to virtualenv if not.
Top commit has no ACKs.
Tree-SHA512: 426ff4397e9142f991c3f091f6a17974a9f2fb694b652e40972a2323992d97c5817528d05dc1a17f7928ac7874ed7c96c076ae1818af2cd3def02cbae71ad512
Fixes#510.
Prior to this commit, if the call to
jmclient.BitcoinCoreInterface.get_current_blockheight() failed and
returned None, the function jmclient.WalletService.sync_unspent() would
simply ignore the failure and the wallet syncing process would complete,
without ever resetting and updating the wallet's utxo set, resulting in
a situation where users saw out of date utxo sets in their wallets (and
concomitant spending failures).
After this commit, if that blockheight access call fails 5 times, we
instead abandon the attempt to sync the wallet, and shut down the
application.
8a25e3a401 Pin remaining deps to specific versions, except for Qt and tests (Kristaps Kaupe)
Pull request description:
For tests it's not important, with Qt there is big mess currently, don't want to touch it here.
Top commit has no ACKs.
Tree-SHA512: 4a72a41302aee5dd3ea289db0c21870753be2586a63f7575f6299749fe0c96924f2cc3f169e6cd873c51d926c62fe41389320712521cf0538458a54ad270c774
7d0ca22749 Update local Tor to 0.4.7.12 (Kristaps Kaupe)
Pull request description:
There have been multiple bugfixes between 0.4.7.8 and 0.4.7.12, including major fixes aimed at helping defend against network denial of service. https://gitlab.torproject.org/tpo/core/tor/-/raw/release-0.4.7/ReleaseNotes
ACKs for top commit:
AdamISZ:
ACK 7d0ca22749 , please merge when it is convenient. (I sanity checked the shasum 3b5d969712c467851bd028f314343ef15a97ea457191e93ffa97310b05b9e395 by downloading from https://archive.torproject.org/tor-package-archive/ ).
Tree-SHA512: b5df074411cfe4362a7148a5baa12de4c1dbd7e15f9026cea631fa94139346f3b8bb08cb54477dd7b46a0ca3d8f1e9891782f55e351f324b02db6a1fb72d4110
b59fdcd623 Remove Python 3.6 (EOL), add 3.10 (Kristaps Kaupe)
Pull request description:
Subset of #1218. Python 3.6 is not only EOL for a long time, also no more available on ubuntu-latest for GitHub workflows, see https://github.com/JoinMarket-Org/joinmarket-clientserver/actions/runs/3705429480/jobs/6279325863. So this is kinda urgent, our CI is broken otherwise.
Top commit has no ACKs.
Tree-SHA512: 9438202d142666736548872a6850d8c72edcfca09d30ed34ecc7290dfe26e1f3af15860cdafa60cd321321666ee9d44a7a3be54931c808f076e1fe89cf2ccb31
44aea32a7c Fix typo in `gettimelockaddress` (Luke)
Pull request description:
Fix trivial typo in command line output and relative documentation. Additionally, fix a couple extra typos.
Top commit has no ACKs.
Tree-SHA512: b7e84f4f032109b7ede8406b23c537331e1236f8477d17fcb5c15521cda528f6b2b3be515c1d262b866b6a41e244d5e3b5fa6e83d6d21ff0d3d05232e6ecd0ea
Adam Gibson
Kristaps Kaupe
PulpCattel