ca33ecafda Bump txtorcon from 23.0.0 to 23.11.0 (Kristaps Kaupe)
Pull request description:
In #1567 we updated to 23.0.0, not 23.5.0, because that was the last version supporting Python 3.7. Now that Python 3.7 support is dropped with #1639, can upgrade to latest.
Changes in [23.5.0](https://github.com/meejah/txtorcon/releases/tag/v23.5.0):
* twisted.web.client.Agent instances now use the same HTTPS policy by default as twisted.web.client.Agent. It is possible to override this policy with the tls_context_factory= argument, the equivalent to Agent's contextFactory= (Thanks to Itamar Turner-Trauring)
* Added support + testing for Python 3.11.
* No more ipaddress dependency
Changes in [23.11.0](https://github.com/meejah/txtorcon/releases/tag/v23.11.0):
* Fix test-failures on Python 3.12
* Particular GETINFO hanging (https://github.com/meejah/txtorcon/issues/389) (ultra-long lines over 16KiB caused problems in the protocol)
* Use built-in `mock` only (https://github.com/jelly)
* Remove `incremental` (https://github.com/gdrosos)
Basically, not much changes, but fixed bug, dropped dependencies and better support for Python 3.11 and 3.12 is good enough reason to merge for me.
Top commit has no ACKs.
Tree-SHA512: 0379dfd7303f3d2f5567fddda75f4d06af1fb56c0499245745842ba82b5623d49324e4c36e7e759b50becb22874a32282c115f6c5a2c5c18979acc4b096f7d18
Changes in version 0.4.8.10 - 2023-12-08
This is a security release fixing a high severity bug (TROVE-2023-007)
affecting Exit relays supporting Conflux. We strongly recommend to update as
soon as possible.
o Major bugfixes (TROVE-2023-007, exit):
- Improper error propagation from a safety check in conflux leg
linking lead to a desynchronization of which legs were part of a
conflux set, ultimately causing a UAF and NULL pointer dereference
crash on Exit relays. Fixes bug 40897; bugfix on 0.4.8.1-alpha.
o Minor features (fallbackdir):
- Regenerate fallback directories generated on December 08, 2023.
o Minor features (geoip data):
- Update the geoip files to match the IPFire Location Database, as
retrieved on 2023/12/08.
o Minor bugfixes (bridges, statistics):
- Correctly report statistics for client count over Pluggable
transport. Fixes bug 40871; bugfix on 0.4.8.4
Changes in version 0.4.8.9 - 2023-11-09
This is another security release fixing a high severity bug affecting onion
services which is tracked by TROVE-2023-006. We are also releasing a guard
major bugfix as well. If you are an onion service operator, we strongly
recommend to update as soon as possible.
o Major bugfixes (guard usage):
- When Tor excluded a guard due to temporary circuit restrictions,
it considered *additional* primary guards for potential usage by
that circuit. This could result in more than the specified number
of guards (currently 2) being used, long-term, by the tor client.
This could happen when a Guard was also selected as an Exit node,
but it was exacerbated by the Conflux guard restrictions. Both
instances have been fixed. Fixes bug 40876; bugfix
on 0.3.0.1-alpha.
o Major bugfixes (onion service, TROVE-2023-006):
- Fix a possible hard assert on a NULL pointer when recording a
failed rendezvous circuit on the service side for the MetricsPort.
Fixes bug 40883; bugfix on 0.4.8.1-alpha
o Minor features (fallbackdir):
- Regenerate fallback directories generated on November 09, 2023.
o Minor features (geoip data):
- Update the geoip files to match the IPFire Location Database, as
retrieved on 2023/11/09.
Changes in version 0.4.8.8 - 2023-11-03
We are releasing today a fix for a high security issue, TROVE-2023-004, that
is affecting relays. Also a few minor bugfixes detailed below. Please upgrade
as soon as posssible.
o Major bugfixes (TROVE-2023-004, relay):
- Mitigate an issue when Tor compiled with OpenSSL can crash during
handshake with a remote relay. Fixes bug 40874; bugfix
on 0.2.7.2-alpha.
o Minor features (fallbackdir):
- Regenerate fallback directories generated on November 03, 2023.
o Minor features (geoip data):
- Update the geoip files to match the IPFire Location Database, as
retrieved on 2023/11/03.
o Minor bugfixes (directory authority):
- Look at the network parameter "maxunmeasuredbw" with the correct
spelling. Fixes bug 40869; bugfix on 0.4.6.1-alpha.
o Minor bugfixes (vanguards addon support):
- Count the conflux linked cell as valid when it is successfully
processed. This will quiet a spurious warn in the vanguards addon.
Fixes bug 40878; bugfix on 0.4.8.1-alpha.
Old wallet format isn't used for years and script is broken since
removal of pyaes dependency. If somebody still needs it, he can use
older JoinMarket version to do conversion.
fe9ec99322 When looking for a free TCP ports, bind only to localhost (Kristaps Kaupe)
Pull request description:
Not a real security issue, as this is only used in tests, but it's simple to fix and right thing to do. Found by [GitHub CodeQL code scanner](https://docs.github.com/en/code-security/code-scanning/introduction-to-code-scanning/about-code-scanning-with-codeql).
Top commit has no ACKs.
Tree-SHA512: cf2b3267bef2bf9132e4727305ffdb8d1734a1c1bf083c344096c930835df9a625dfd86d3c3dd5c24e3d8e28e4cbbac47d9d95aaee5198198a60d376c7f1b08a
204f1171c2 Drop Python 3.7 support (Kristaps Kaupe)
Pull request description:
It's EOL since 27 Jun 2023 and 3.8+ is required for #1637.
Also mentioned in docs that JoinMarket is currently not compatible with 3.12 (#1589).
ACKs for top commit:
AdamISZ:
utACK 204f1171c2
roshii:
utACK 204f1171c2
Tree-SHA512: 39ee96aca5e305b894e97a2a843d748a645becbfca1bc9e35f02f33c4634a4f7fee37fd36115510cc052e1d11497dceee2d0fafa12d006dd005d050e5ea1cc9f
9ab8ad632e Remove mention of donations from JoinMarketQt guide (Kristaps Kaupe)
cb6077498c Qt: Remove donation link from "About" dialog (Kristaps Kaupe)
Pull request description:
Donation address link was removed from README in #1611, as it is not in control of any of the current active developers. This removes it also from JoinMarketQt "About" dialog.
Top commit has no ACKs.
Tree-SHA512: 5ea77093dbc94d08f7757eac75a32cabe46272d080fc931194840886d63bd7d5d337be8564a2755df3a32ea34632641e7a42068d41cd87a525801bd66df35827
ab1481d814 RPC API: Add optional txfee property for single joins (Kristaps Kaupe)
Pull request description:
Resolves#1607.
Top commit has no ACKs.
Tree-SHA512: 2cbb8cd38d8888d84fd6ea438dadaf31d0406c4bdadcf9206ca539e1ae728e6c9cb4504de4c62f626bdae585fc7fb923cb148883a5a60faa1c82cf2e00837efa
8254a67341 Update secp256k1 lib deps for pythonbitcointx1.1.5 (Adam Gibson)
Pull request description:
Prior to this commit, code for scalar multiplication the module secp256k1_main in jmbitcoin relied on direct access to the secp256k1 linked library, but the API for accessing that object has now changed in python-bitcointx with version 1.1.5 (there is now a Secp256k1 data class with lib and ctx entries, see the Readme of the project for details). After this commit, we update our code for that API (but do not make functional changes to Joinmarket itself).
ACKs for top commit:
kristapsk:
ACK 8254a67341. Tested by first just upgrading to python-bitcointx 1.1.5, which caused test failures, then applying all changes and tests passed.
Tree-SHA512: 371182be0a9abfd8bc020268c92f2de806f2d6b553d03f19dbb4768eee94a7fe8ef0bc441bf33ec5fa342529ee46f22971edaf46e8b9e3f96b6157316a69724e
Prior to this commit, code for scalar multiplication the module
secp256k1_main in jmbitcoin relied on direct access to the secp256k1
linked library, but the API for accessing that object has now changed in
python-bitcointx with version 1.1.5 (there is now a Secp256k1 data class
with lib and ctx entries, see the Readme of the project for details).
After this commit, we update our code for that API (but do not make
functional changes to Joinmarket itself).
196a097667 Allow absurd fee override when setting tx fee manually (Kristaps Kaupe)
Pull request description:
When user manually specifies `--txfee` with `sendpayment.py`, he likely knows what he is doing. So allow there fees above `absurd_fee_per_kb` setting. But still ask for user confirmation as he could possible made a mistake, for example, added some extra zeros to the value.
Top commit has no ACKs.
Tree-SHA512: 2155c38a483fd6392153bf42b2ea87cb9395f0da289cc2743602493df9e924a1d621688704a46d179e3e574ce41e82ced26b62331157ac2c3d1481931a334ace
3d568376a9 Remove unnecessary check for Python 3.7+ (Kristaps Kaupe)
Pull request description:
Python 3.6 support in JoinMarket is dropped for some time.
Top commit has no ACKs.
Tree-SHA512: e094c174219d289236742a92d4acb37c4763605984dc9e83a233bed6129358c2b96e59dfe0f91dd3fc502a8d02b52db5fe714a5c1dac41845623b4756131e822
935a734b93 Fix tests for Core v26 when user has no access to ~/.bitcoin/settings.json (Kristaps Kaupe)
Pull request description:
From v26 Bitcoin Core always checks it's datadir `settings.json` file, even if you do stuff like `-help` or `-version`. It's stupid, but it is what it is. This makes problems with multi-user setups, where `bitcoind` runs under one user (`bitcoin`) and JoinMarket under another (`joinmarket`), and `/home/joinmarket/.bitcoin` is just symlink to original Bitcoin datadir, with sole purpose that RPC auth cookie is accessible, but `settings.json`, created by `bitcoind` itself, is not group readable. That is setup used for RaspiBolt, but not only, it's setup that makes sense.
Without this, tests fail on such setups:
```
(jmvenv) joinmarket@odroid:~/git/joinmarket-clientserver$ ./test/run_tests.sh
~/git/joinmarket-clientserver ~/git/joinmarket-clientserver
miniircd.tar.gz: OK
Error: Settings file could not be read:
- /home/joinmarket/.bitcoin/settings.json. Please check permissions.
==================================================================================== test session starts =====================================================================================
platform linux -- Python 3.8.10, pytest-6.2.5, py-1.11.0, pluggy-1.3.0
rootdir: /home/joinmarket/git/joinmarket-clientserver, configfile: pyproject.toml, testpaths: test
plugins: cov-2.5.1
collected 421 items
test/jmbase/test_base_support.py
!!!!!!!!! _pytest.outcomes.Exit: Cannot setup tests, bitcoind failing.
Error: Settings file could not be read:
- /home/joinmarket/.bitcoin/settings.json. Please check permissions. !!!!!!!!!!
(jmvenv) joinmarket@odroid:~/git/joinmarket-clientserver$ ls -al /home/joinmarket/.bitcoin
lrwxrwxrwx 1 joinmarket joinmarket 13 Nov 22 2022 /home/joinmarket/.bitcoin -> /data/bitcoin
(jmvenv) joinmarket@odroid:~/git/joinmarket-clientserver$ ls -al /data/bitcoin
total 115448
drwxrwxr-x 15 bitcoin bitcoin 4096 Dec 30 11:11 .
drwxr-xr-x 18 root root 4096 Oct 21 11:21 ..
-rw-r----- 1 bitcoin bitcoin 75 Dec 7 06:20 .cookie
-rw------- 1 bitcoin bitcoin 0 Sep 27 2021 .lock
drwx------ 3 bitcoin bitcoin 4096 Dec 7 06:20 SatSale
-rw------- 1 bitcoin bitcoin 34 Sep 27 2021 banlist.json
-rw-r----- 1 bitcoin bitcoin 1238 Dec 7 06:20 bitcoin.conf
drwx------ 3 bitcoin bitcoin 237568 Dec 30 06:00 blocks
drwx------ 2 bitcoin bitcoin 167936 Dec 30 11:19 chainstate
-rw------- 1 bitcoin bitcoin 17409758 Dec 30 11:23 debug.log
drwx------ 2 bitcoin bitcoin 4096 Sep 12 2022 descriptors_test
-rw------- 1 bitcoin bitcoin 247985 Dec 30 11:22 fee_estimates.dat
-rw------- 1 bitcoin bitcoin 679 Sep 27 2021 i2p_private_key
drwx------ 4 bitcoin bitcoin 4096 Sep 12 2022 indexes
drwx------ 3 bitcoin bitcoin 4096 Dec 7 06:21 jm_v26_test
drwx------ 3 bitcoin bitcoin 4096 Dec 7 08:30 jm_wallet
drwx------ 3 bitcoin bitcoin 4096 Feb 17 2022 joinmarket
drwx------ 2 bitcoin bitcoin 4096 Nov 27 2021 joinmarket-old
-rw------- 1 bitcoin bitcoin 94860907 Dec 7 06:02 mempool.dat
-rw------- 1 bitcoin bitcoin 99 Dec 7 06:21 onion_v3_private_key
-rw------- 1 bitcoin bitcoin 5189788 Dec 30 11:11 peers.dat
-rw-rw-r-- 1 bitcoin bitcoin 1534 Nov 27 2021 rpcauth.py
-rw------- 1 bitcoin bitcoin 60 Dec 19 13:19 settings.json
drwx------ 3 bitcoin bitcoin 4096 Sep 16 2022 specterd452d801d8c61692
drwx------ 3 bitcoin bitcoin 4096 Sep 16 2022 specterd452d801d8c61692_hotstorage
drwx------ 3 bitcoin bitcoin 4096 Sep 16 2022 specterf42828a49e7c6fd0
drwx------ 3 bitcoin bitcoin 4096 Dec 7 06:20 test
```
Fix is to provide `-conf`/`-datadir` argument to `bitcoind -version` calls.
See also discussion in https://github.com/raspibolt/raspibolt/pull/1387.
Top commit has no ACKs.
Tree-SHA512: ccceb876e216c946173191ade4f065b413cdaeda9a5c750cf2bda85d76381f4c4484da5a552c07a2a4c25cba0c38ce722826a497467a0fee15b23f82971cabda
d9fee292f4 Fix get_tx_info() type hint and doc (Kristaps Kaupe)
Pull request description:
`txid` is passed as binary `bytes` there, not hex string.
Top commit has no ACKs.
Tree-SHA512: 3e8272a5dc2291add827a06a5d0cda03652168e9d0cc3c9ec7532ace0522eef75c7e38c0dce6e665a65bb73d31579b2afda814fc18d0afcc99db311495642034
fab97a003b Use get_deser_from_gettransaction() everywhere (Kristaps Kaupe)
Pull request description:
There's `get_deser_from_gettransaction()` method in `BlockchainInterface` which is wrapper around python-bitcointx `CMutableTransaction.deserialize()` with some error checking added. It is used in most of the code, but in `jmclient/wallet_utils.py` I found two direct calls of `CMutableTransaction.deserialize()`.
Top commit has no ACKs.
Tree-SHA512: c0fc781e37b3bc72e89805e62b594ba879d5c2b84d3417dcefeba4428d21abcf0f44f143c1b9f5cf23a896e03add26b80b9cc514842142b45432c31adf563d7a
8f382d0d5e Add test for dict_factory() (Kristaps Kaupe)
2978b18245 De-duplicate and change dict_factory (Kristaps Kaupe)
Pull request description:
Move to `jmbase` and switch to implementation from [Python docs](https://docs.python.org/3/library/sqlite3.html#sqlite3-howto-row-factory), which seems nicer.
Top commit has no ACKs.
Tree-SHA512: 02ca21fea0eb45cc2f05d47e85b4150c300ad895aaf6102c90c433b38b95f8fb7b1e04292542afc6073fc22b14b07c38c01957f1380bed080585d9b67c0ee4bf
84914319cc cache deserialized form of txs in history method (Adam Gibson)
Pull request description:
Prior to this commit, while the result of the gettransaction rpc call was being cached so as to not have to repeat these rpc calls, the deserialized form of the transaction created by a call to CMutableTransaction.deserialize, was not, and since this call is rather expensive, the history method was running more slowly than needed. After this commit, we cache the deserialized form also, resulting in a speedup to the wallet_utils.wallet_fetch_history method.
ACKs for top commit:
kristapsk:
ACK 84914319cc
Tree-SHA512: 868a92dbd02c4c52f5b609b1d085bc1c56d959fb4399bc1f3166bc8bbe71727a39aa266a7ebd4f77cb96430bfbba1af51dbe6bc995ed54955bea3b54d10624b5
47acf6a057 Update libsecp256k1 to v0.4.1 (Kristaps Kaupe)
Pull request description:
https://github.com/bitcoin-core/secp256k1/releases/tag/v0.4.1
> This release slightly increases the speed of the ECDH operation and significantly enhances the performance of many library functions when using the default configuration on x86_64.
ABI is backwards compatible with v0.4.0, so this should not impact existing python-bitcointx.
ACKs for top commit:
AdamISZ:
ACK 47acf6a057
Tree-SHA512: 98c27ae2dc3c227e24a015bbb2709dcbaddc74ebe239de059c8172693ab2e71fbcff5f026b2d0a60d9b81c59f1c68bb08d0b3f17bfa38cd4456d9d08a3887409
Prior to this commit, while the result of the gettransaction rpc call
was being cached so as to not have to repeat these rpc calls, the
deserialized form of the transaction created by a call to
CMutableTransaction.deserialize, was not, and since this call is rather
expensive, the history method was running more slowly than needed.
After this commit, we cache the deserialized form also, resulting in a
speedup to the wallet_utils.wallet_fetch_history method.
Prior to this commit, the calls to get_new_addr in the functions in
the initial sync algo used for recovery, used the default value of the
argument validate_cache, which is True (because in normal running,
get_new_addr is used to derive addresses as destinations, for which it's
safer to not use the cache, and as one-off calls, are not
performance-sensitive). This caused initial sync to be very slow in
recovery, especially if using large gap limits (which is common).
After this commit, we set the argument validate_cache to False, as is
intended during initial sync. This allows the optimised performance from
caching to be in effect. See earlier PRs 1594 and 1614 for context.
d234731022 Add -allowignoredconf=1 for Bitcoin Core v26+ (Kristaps Kaupe)
8684853117 Support Bitcoin Core v26 for tests (Kristaps Kaupe)
Pull request description:
Bitcoin Core config needs to be hacked with `deprecatedrpc=create_bdb` for v26 until #1571 is solved.
Top commit has no ACKs.
Tree-SHA512: cc30afe05af7df2c73f60436ee43efe6c2cbda50539052a28bd72fa27dcab91eb6d0c0ba1d42cf37d589f9e00bbc87dbf05d9ac34acc559387aa05b38f18f87f
9410b9cbfa Update libsecp256k1 and python-bitcointx (Kristaps Kaupe)
Pull request description:
See https://github.com/Simplexum/python-bitcointx/issues/75.
This updates libsecp256k1 to latest v0.3.2 release and python-bitcointx to current master. Currently for testing purposes only, for merging into master we will need to wait for a new python-bitcointx release.
ACKs for top commit:
AdamISZ:
tACK 9410b9cbfa
roshii:
tACK 9410b9cbfa
Tree-SHA512: 4c87ed7b896445cb200deeb284ee6ef0aa52ce2bb88954d34dd8f13b76e3a0693aa8ee6f77f9172138e40dec44cff503d2168cce5092610b1a7a1d42530ea02b
79e5c3d0a7 Document Core wallet creation for v26 (Kristaps Kaupe)
Pull request description:
Creation of legacy (BDB) wallet have been deprecated with Bitcoin Core 26.0 release. Proper solution is to [implement descriptor wallet support](https://github.com/JoinMarket-Org/joinmarket-clientserver/issues/1571), but for now let's update documentation.
Also we can't update tests to v26 because of this (we could in theory, but it's unnecessary complexity to maintain two different `bitcoin.conf`'s for different Core versions just because of tests).
Top commit has no ACKs.
Tree-SHA512: 90b6479f21ea7131f080a67aab122970b5ed097fa261ad9a5e4d05aa207c77722a2790d0ce317d8c746e395c421e456b992161d81168cbc758951cb57e8bde2f
ef1d76e9db Allow cache purging at wallet initialization (Adam Gibson)
Pull request description:
Fixes#1614.
Prior to this commit, if data in the persisted cache in the wallet file were wrong (should be a very extraordinary case), then the joinmarket code would have to crash with a cache invalid warning. After this commit, in such an extraordinary case, the option exists to invalidate or remove the cache on startup, so that it can be rebuilt from scratch. This is done with a config var wallet_caching_disabled in the POLICY section.
ACKs for top commit:
kristapsk:
utACK ef1d76e9db
Tree-SHA512: dea2256c7246417ab17be0c89d6b64f49243f35204a8d316ab520f94451d149b3131c928be020f7152f99e0eb5f07843341bb48fca5aaef4c6517701a122b355
dependabot[bot]
Kristaps Kaupe
Adam Gibson
d11n