63d74bc0f1 Call num_cores() only after deps_install() (Kristaps Kaupe)
Pull request description:
There may be no Python available before that.
Alternative to #1419.
Top commit has no ACKs.
Tree-SHA512: 7a377fec8b819458e678ef2fd869bd6692f404772f4b42c4fd8caad530b0d49c96010841362b9279488781e914d917897589be46f562f8c5c5025a17f36542c4
This may or may not fix#1057 and other test failures. Since the network
delay in receiving the message is unknown (but expected to be very
small, usually), we keep trying to receive notification of the
transaction message in the websocket test 10 times for a total of 2
seconds (on the assumption that if it takes longer than that, something
else is wrong).
7aebed728b Remove jmdirjmioywe2s5jad7ts6kgcqg66rj6wujj6q77n6wbdrgocqwexzid.onion directory node from default config (Kristaps Kaupe)
Pull request description:
It has been discussed on Telegram. This node seems to be permanently down for a long period of time.
ACKs for top commit:
AdamISZ:
ACK 7aebed728b
Tree-SHA512: f0d32c53e3647903cce93476a03078acf318d7f78830edb8fae07408daf121cd9056b513bfa809e6eaf8d4827e4a2520e75094ff9471f0804b3e14ec2b999b9f
Prior to this commit, p2wsh inputs from fidelity bonds resulted in
miscalculation of transaction fees, even in cases where the exact set of
inputs were known (such as a direct send).
In this commit we change the estimation to a model in which the caller
of jmbitcoin.secp256k1_transaction.estimate_tx_size must specify a list
of types, one for each input to the transaction, and the same for
outputs. In some cases, the caller of the function uses the default
script type of the wallet, but in other cases where the caller can know
the exact types of each utxo used as input, and each destination used as
output, they are specified explicitly. In particular, the use of
fidelity bond outputs as input to transactions can be accounted for.
Currently this is only done in direct send payments; coinjoins still
fall back to assuming all inputs the same type (note that it is not
possible to use fidelity bond utxos as inputs to coinjoins).
Note also that the burn destination calculation in direct send is
removed, since it is not used, so the maintenance burden is best
avoided.
22c13b0ab7 Remove unused code instead of commenting out (Kristaps Kaupe)
b928713880 Add ShellCheck linter script (Kristaps Kaupe)
4f0eebc68d Apply all current shellcheck suggestions to rest of the scripts (Kristaps Kaupe)
f0b9872ff9 Apply all current shellcheck suggestions to install.sh (Kristaps Kaupe)
Pull request description:
Based on #1175, but also covers `install.sh` changes since then, all other shell scripts and adds ShellCheck linter script. Not adding this to CI for now. I'm thinking we could split off all the linting to separate job, don't see a point in running them under all OS / Python version combinations.
Top commit has no ACKs.
Tree-SHA512: 936320280f1ec9d8aa315c70be4d33fcb06d324531a17b30987c0be308d5351aa45dfac05cecadca8e9977a8f797b62ccd91d5d5fe92c5d0eacc6ebdf4dcd1c6
bfb0e25861 Account for missing nick fields in OnionPeer (Adam Gibson)
Pull request description:
Prior to this commit, if a message (like a disconnection control message), was to be sent to live peers, regarding another peer that did not have a nick field set, a crash would occur in directory nodes sending this message, because the function
OnionPeer.get_nick_peerlocation_ser() assumed the existence of the nick field, and raised the OnionPeerError exception if it did not exist.
After this commit, the raising of this Exception is caught when calling OnionPeer.get_nick_peerlocation_ser, so that a peer in this state is just not added to the list of peers to be sent to live peers. This condition is caused specifically by the fact that said peer has not completed a handshake (which must always pass the value of the nick), and so the other live peers will not yet be aware of it.
ACKs for top commit:
kristapsk:
cr utACK bfb0e25861
Tree-SHA512: 7bf1fe216bc7088b986a9f33c65fa07e25a5ec09a42cafc7496567b628a1e3b03e91b6554b9d66f6e0603d8ee15a729f6af76fed4a75ecbdf385a78d8cd4100b
3c8f247d1a CI: Run tests on both Linux and macOS (Kristaps Kaupe)
Pull request description:
If devs have problems with accessing Mac's for tests, let GitHub Actions CI do the basic tesing for us.
Top commit has no ACKs.
Tree-SHA512: a36e46c03704ffde8588342d200e4b7c05df72b5bc3d0046027d793a9058e646c4fe3c45860ccf808f87260ade0a8778ea7ac3531eacd2db43402fa0b879e69b
Prior to this commit, if a message (like a disconnection control
message), was to be sent to live peers, regarding another peer that did
not have a nick field set, a crash would occur in directory nodes
sending this message, because the function
OnionPeer.get_nick_peerlocation_ser() assumed the existence of the nick
field, and raised the OnionPeerError exception if it did not exist.
After this commit, the raising of this Exception is caught when calling
OnionPeer.get_nick_peerlocation_ser, so that a peer in this state is
just not added to the list of peers to be sent to live peers. This
condition is caused specifically by the fact that said peer has not
completed a handshake (which must always pass the value of the nick),
and so the other live peers will not yet be aware of it.
11ddec7841 docs: Remove now-confusing mentions of virtualenv (Carl Dong)
6e3b6eca5d Dockerfiles: Use Python venv instead of virtualenv (Carl Dong)
8bbca60641 install: Use python3's venv module when available (Carl Dong)
Pull request description:
Python 3.3 and above provide a standard venv module which provides all the functionality that install.sh needs from virtualenv.
Detect its existence and use it, fallback to virtualenv if not.
Top commit has no ACKs.
Tree-SHA512: 426ff4397e9142f991c3f091f6a17974a9f2fb694b652e40972a2323992d97c5817528d05dc1a17f7928ac7874ed7c96c076ae1818af2cd3def02cbae71ad512
Fixes#510.
Prior to this commit, if the call to
jmclient.BitcoinCoreInterface.get_current_blockheight() failed and
returned None, the function jmclient.WalletService.sync_unspent() would
simply ignore the failure and the wallet syncing process would complete,
without ever resetting and updating the wallet's utxo set, resulting in
a situation where users saw out of date utxo sets in their wallets (and
concomitant spending failures).
After this commit, if that blockheight access call fails 5 times, we
instead abandon the attempt to sync the wallet, and shut down the
application.
8a25e3a401 Pin remaining deps to specific versions, except for Qt and tests (Kristaps Kaupe)
Pull request description:
For tests it's not important, with Qt there is big mess currently, don't want to touch it here.
Top commit has no ACKs.
Tree-SHA512: 4a72a41302aee5dd3ea289db0c21870753be2586a63f7575f6299749fe0c96924f2cc3f169e6cd873c51d926c62fe41389320712521cf0538458a54ad270c774
7d0ca22749 Update local Tor to 0.4.7.12 (Kristaps Kaupe)
Pull request description:
There have been multiple bugfixes between 0.4.7.8 and 0.4.7.12, including major fixes aimed at helping defend against network denial of service. https://gitlab.torproject.org/tpo/core/tor/-/raw/release-0.4.7/ReleaseNotes
ACKs for top commit:
AdamISZ:
ACK 7d0ca22749 , please merge when it is convenient. (I sanity checked the shasum 3b5d969712c467851bd028f314343ef15a97ea457191e93ffa97310b05b9e395 by downloading from https://archive.torproject.org/tor-package-archive/ ).
Tree-SHA512: b5df074411cfe4362a7148a5baa12de4c1dbd7e15f9026cea631fa94139346f3b8bb08cb54477dd7b46a0ca3d8f1e9891782f55e351f324b02db6a1fb72d4110
b59fdcd623 Remove Python 3.6 (EOL), add 3.10 (Kristaps Kaupe)
Pull request description:
Subset of #1218. Python 3.6 is not only EOL for a long time, also no more available on ubuntu-latest for GitHub workflows, see https://github.com/JoinMarket-Org/joinmarket-clientserver/actions/runs/3705429480/jobs/6279325863. So this is kinda urgent, our CI is broken otherwise.
Top commit has no ACKs.
Tree-SHA512: 9438202d142666736548872a6850d8c72edcfca09d30ed34ecc7290dfe26e1f3af15860cdafa60cd321321666ee9d44a7a3be54931c808f076e1fe89cf2ccb31
44aea32a7c Fix typo in `gettimelockaddress` (Luke)
Pull request description:
Fix trivial typo in command line output and relative documentation. Additionally, fix a couple extra typos.
Top commit has no ACKs.
Tree-SHA512: b7e84f4f032109b7ede8406b23c537331e1236f8477d17fcb5c15521cda528f6b2b3be515c1d262b866b6a41e244d5e3b5fa6e83d6d21ff0d3d05232e6ecd0ea
Python 3.3 and above provide a standard venv module which provides all
the functionality that install.sh needs from virtualenv.
As of this commit, JM itself requires Python 3.6+ anyway.
5b8ef40bd6 jmbitcoin: eliminate dependency on urldecode (Matt Whitlock)
Pull request description:
`urllib.parse.unquote_plus()` serves the same purpose as `url_decode.urldecode()`, and moreover it correctly decodes UTF-8 sequences into Unicode characters rather than generating mojibake.
ACKs for top commit:
kristapsk:
ACK 5b8ef40bd6
Tree-SHA512: b8598efdcc48914f56036d32309e3d1ee52aa58c0cbbd41dbc66748588d28cfccd5e62869954f07cb3e25b9e7290bab9e1752dbdd3a662f71a7e59dda2f490b3
urllib.parse.unquote_plus() serves the same purpose as
url_decode.urldecode(), and moreover it correctly decodes UTF-8
sequences into Unicode characters rather than generating mojibake.
Updated to reflect that it's now required to provide a node message argument when starting the directory node.
Update onion-message-channels.md
Changed message argument to be the same in the explanation and service file for consistency.
Prior to this commit, we timed out the attempts to connect to individual
directory nodes after 10s, which proves to be too optimistic for
connections to onions, as failures are often seen. After this commit, we
allow 60s (note that this time includes handshake completion).
3d0f2d6fd7 fix: randomize fallback transaction fee (theborakompanioni)
Pull request description:
Resolves#1363
Before this PR, if no fee estimate could be sourced, the fallback fee was always 10_000 sats/kilo-vbyte:
```
Before:
2022-10-11 05:45:19,429 [DEBUG] rpc: getmempoolinfo None
2022-10-11 05:45:19,432 [DEBUG] rpc: estimatesmartfee [3]
2022-10-11 05:45:19,438 [WARNING] Could not source a fee estimate from Core, falling back to default: 10000 sat/vkB (10.0 sat/vB).
2022-10-11 05:45:19,439 [INFO] Using bitcoin network feerate for 3 block confirmation target (randomized for privacy): 10000 sat/vkB (10.0 sat/vB)
2022-10-11 05:45:19,444 [DEBUG] rpc: getmempoolinfo None
2022-10-11 05:45:19,446 [DEBUG] rpc: estimatesmartfee [3]
2022-10-11 05:45:19,452 [WARNING] Could not source a fee estimate from Core, falling back to default: 10000 sat/vkB (10.0 sat/vB).
2022-10-11 05:45:19,453 [INFO] Using bitcoin network feerate for 3 block confirmation target (randomized for privacy): 10000 sat/vkB (10.0 sat/vB)
2022-10-11 05:45:19,488 [INFO] Using a fee of: 0.00001420 BTC (1420 sat).
```
After this PR, the fallback fee will take into account the value of `tx_fees_factor` to randomize the value (if `tx_fees_factor` is greater than zero):
```
2022-10-11 05:51:00,112 [DEBUG] rpc: getmempoolinfo None
2022-10-11 05:51:00,114 [DEBUG] rpc: estimatesmartfee [3]
2022-10-11 05:51:00,119 [WARNING] Could not source a fee estimate from Core, falling back to default: 10000 sat/vkB (10.0 sat/vB).
2022-10-11 05:51:00,120 [INFO] Using bitcoin network feerate for 3 block confirmation target (randomized for privacy): 9962 sat/vkB (9.9 sat/vB)
2022-10-11 05:51:00,123 [DEBUG] rpc: getmempoolinfo None
2022-10-11 05:51:00,126 [DEBUG] rpc: estimatesmartfee [3]
2022-10-11 05:51:00,133 [WARNING] Could not source a fee estimate from Core, falling back to default: 10000 sat/vkB (10.0 sat/vB).
2022-10-11 05:51:00,134 [INFO] Using bitcoin network feerate for 3 block confirmation target (randomized for privacy): 10898 sat/vkB (10.8 sat/vB)
2022-10-11 05:51:00,156 [INFO] Using a fee of: 0.00001547 BTC (1547 sat).
```
Additionally, `fx_fees_factor` will be used to calculate `mempoolminfee_in_sat_randomized` instead of using a hardcoded max value of `1.2` (20%). If this is not wanted or you think this is a mistake, the second commit can be reverted in order to only apply the behaviour described above.
ACKs for top commit:
kristapsk:
ACK 3d0f2d6fd7
Tree-SHA512: d950532e0abe9558b067b2b764f0c43b8a20f27c7c2223600ee461174124a21ec05c2d304ace2006d461d0ff0ae504ba246676a509240eea6d86fe975898a2f7
also, use tx_fees_factor for randomized max mempoolminfee and
make sure that tx_fees_factor is equal or greater than zero.
Co-authored-by: Kristaps Kaupe <kristaps@blogiem.lv>
PulpCattel
Adam Gibson
Kristaps Kaupe
Luke
Carl Dong
Matt Whitlock
hellodarkness
theborakompanioni