fe9ec99322 When looking for a free TCP ports, bind only to localhost (Kristaps Kaupe)
Pull request description:
Not a real security issue, as this is only used in tests, but it's simple to fix and right thing to do. Found by [GitHub CodeQL code scanner](https://docs.github.com/en/code-security/code-scanning/introduction-to-code-scanning/about-code-scanning-with-codeql).
Top commit has no ACKs.
Tree-SHA512: cf2b3267bef2bf9132e4727305ffdb8d1734a1c1bf083c344096c930835df9a625dfd86d3c3dd5c24e3d8e28e4cbbac47d9d95aaee5198198a60d376c7f1b08a
204f1171c2 Drop Python 3.7 support (Kristaps Kaupe)
Pull request description:
It's EOL since 27 Jun 2023 and 3.8+ is required for #1637.
Also mentioned in docs that JoinMarket is currently not compatible with 3.12 (#1589).
ACKs for top commit:
AdamISZ:
utACK 204f1171c2
roshii:
utACK 204f1171c2
Tree-SHA512: 39ee96aca5e305b894e97a2a843d748a645becbfca1bc9e35f02f33c4634a4f7fee37fd36115510cc052e1d11497dceee2d0fafa12d006dd005d050e5ea1cc9f
9ab8ad632e Remove mention of donations from JoinMarketQt guide (Kristaps Kaupe)
cb6077498c Qt: Remove donation link from "About" dialog (Kristaps Kaupe)
Pull request description:
Donation address link was removed from README in #1611, as it is not in control of any of the current active developers. This removes it also from JoinMarketQt "About" dialog.
Top commit has no ACKs.
Tree-SHA512: 5ea77093dbc94d08f7757eac75a32cabe46272d080fc931194840886d63bd7d5d337be8564a2755df3a32ea34632641e7a42068d41cd87a525801bd66df35827
ab1481d814 RPC API: Add optional txfee property for single joins (Kristaps Kaupe)
Pull request description:
Resolves#1607.
Top commit has no ACKs.
Tree-SHA512: 2cbb8cd38d8888d84fd6ea438dadaf31d0406c4bdadcf9206ca539e1ae728e6c9cb4504de4c62f626bdae585fc7fb923cb148883a5a60faa1c82cf2e00837efa
8254a67341 Update secp256k1 lib deps for pythonbitcointx1.1.5 (Adam Gibson)
Pull request description:
Prior to this commit, code for scalar multiplication the module secp256k1_main in jmbitcoin relied on direct access to the secp256k1 linked library, but the API for accessing that object has now changed in python-bitcointx with version 1.1.5 (there is now a Secp256k1 data class with lib and ctx entries, see the Readme of the project for details). After this commit, we update our code for that API (but do not make functional changes to Joinmarket itself).
ACKs for top commit:
kristapsk:
ACK 8254a67341. Tested by first just upgrading to python-bitcointx 1.1.5, which caused test failures, then applying all changes and tests passed.
Tree-SHA512: 371182be0a9abfd8bc020268c92f2de806f2d6b553d03f19dbb4768eee94a7fe8ef0bc441bf33ec5fa342529ee46f22971edaf46e8b9e3f96b6157316a69724e
Prior to this commit, code for scalar multiplication the module
secp256k1_main in jmbitcoin relied on direct access to the secp256k1
linked library, but the API for accessing that object has now changed in
python-bitcointx with version 1.1.5 (there is now a Secp256k1 data class
with lib and ctx entries, see the Readme of the project for details).
After this commit, we update our code for that API (but do not make
functional changes to Joinmarket itself).
196a097667 Allow absurd fee override when setting tx fee manually (Kristaps Kaupe)
Pull request description:
When user manually specifies `--txfee` with `sendpayment.py`, he likely knows what he is doing. So allow there fees above `absurd_fee_per_kb` setting. But still ask for user confirmation as he could possible made a mistake, for example, added some extra zeros to the value.
Top commit has no ACKs.
Tree-SHA512: 2155c38a483fd6392153bf42b2ea87cb9395f0da289cc2743602493df9e924a1d621688704a46d179e3e574ce41e82ced26b62331157ac2c3d1481931a334ace
3d568376a9 Remove unnecessary check for Python 3.7+ (Kristaps Kaupe)
Pull request description:
Python 3.6 support in JoinMarket is dropped for some time.
Top commit has no ACKs.
Tree-SHA512: e094c174219d289236742a92d4acb37c4763605984dc9e83a233bed6129358c2b96e59dfe0f91dd3fc502a8d02b52db5fe714a5c1dac41845623b4756131e822
935a734b93 Fix tests for Core v26 when user has no access to ~/.bitcoin/settings.json (Kristaps Kaupe)
Pull request description:
From v26 Bitcoin Core always checks it's datadir `settings.json` file, even if you do stuff like `-help` or `-version`. It's stupid, but it is what it is. This makes problems with multi-user setups, where `bitcoind` runs under one user (`bitcoin`) and JoinMarket under another (`joinmarket`), and `/home/joinmarket/.bitcoin` is just symlink to original Bitcoin datadir, with sole purpose that RPC auth cookie is accessible, but `settings.json`, created by `bitcoind` itself, is not group readable. That is setup used for RaspiBolt, but not only, it's setup that makes sense.
Without this, tests fail on such setups:
```
(jmvenv) joinmarket@odroid:~/git/joinmarket-clientserver$ ./test/run_tests.sh
~/git/joinmarket-clientserver ~/git/joinmarket-clientserver
miniircd.tar.gz: OK
Error: Settings file could not be read:
- /home/joinmarket/.bitcoin/settings.json. Please check permissions.
==================================================================================== test session starts =====================================================================================
platform linux -- Python 3.8.10, pytest-6.2.5, py-1.11.0, pluggy-1.3.0
rootdir: /home/joinmarket/git/joinmarket-clientserver, configfile: pyproject.toml, testpaths: test
plugins: cov-2.5.1
collected 421 items
test/jmbase/test_base_support.py
!!!!!!!!! _pytest.outcomes.Exit: Cannot setup tests, bitcoind failing.
Error: Settings file could not be read:
- /home/joinmarket/.bitcoin/settings.json. Please check permissions. !!!!!!!!!!
(jmvenv) joinmarket@odroid:~/git/joinmarket-clientserver$ ls -al /home/joinmarket/.bitcoin
lrwxrwxrwx 1 joinmarket joinmarket 13 Nov 22 2022 /home/joinmarket/.bitcoin -> /data/bitcoin
(jmvenv) joinmarket@odroid:~/git/joinmarket-clientserver$ ls -al /data/bitcoin
total 115448
drwxrwxr-x 15 bitcoin bitcoin 4096 Dec 30 11:11 .
drwxr-xr-x 18 root root 4096 Oct 21 11:21 ..
-rw-r----- 1 bitcoin bitcoin 75 Dec 7 06:20 .cookie
-rw------- 1 bitcoin bitcoin 0 Sep 27 2021 .lock
drwx------ 3 bitcoin bitcoin 4096 Dec 7 06:20 SatSale
-rw------- 1 bitcoin bitcoin 34 Sep 27 2021 banlist.json
-rw-r----- 1 bitcoin bitcoin 1238 Dec 7 06:20 bitcoin.conf
drwx------ 3 bitcoin bitcoin 237568 Dec 30 06:00 blocks
drwx------ 2 bitcoin bitcoin 167936 Dec 30 11:19 chainstate
-rw------- 1 bitcoin bitcoin 17409758 Dec 30 11:23 debug.log
drwx------ 2 bitcoin bitcoin 4096 Sep 12 2022 descriptors_test
-rw------- 1 bitcoin bitcoin 247985 Dec 30 11:22 fee_estimates.dat
-rw------- 1 bitcoin bitcoin 679 Sep 27 2021 i2p_private_key
drwx------ 4 bitcoin bitcoin 4096 Sep 12 2022 indexes
drwx------ 3 bitcoin bitcoin 4096 Dec 7 06:21 jm_v26_test
drwx------ 3 bitcoin bitcoin 4096 Dec 7 08:30 jm_wallet
drwx------ 3 bitcoin bitcoin 4096 Feb 17 2022 joinmarket
drwx------ 2 bitcoin bitcoin 4096 Nov 27 2021 joinmarket-old
-rw------- 1 bitcoin bitcoin 94860907 Dec 7 06:02 mempool.dat
-rw------- 1 bitcoin bitcoin 99 Dec 7 06:21 onion_v3_private_key
-rw------- 1 bitcoin bitcoin 5189788 Dec 30 11:11 peers.dat
-rw-rw-r-- 1 bitcoin bitcoin 1534 Nov 27 2021 rpcauth.py
-rw------- 1 bitcoin bitcoin 60 Dec 19 13:19 settings.json
drwx------ 3 bitcoin bitcoin 4096 Sep 16 2022 specterd452d801d8c61692
drwx------ 3 bitcoin bitcoin 4096 Sep 16 2022 specterd452d801d8c61692_hotstorage
drwx------ 3 bitcoin bitcoin 4096 Sep 16 2022 specterf42828a49e7c6fd0
drwx------ 3 bitcoin bitcoin 4096 Dec 7 06:20 test
```
Fix is to provide `-conf`/`-datadir` argument to `bitcoind -version` calls.
See also discussion in https://github.com/raspibolt/raspibolt/pull/1387.
Top commit has no ACKs.
Tree-SHA512: ccceb876e216c946173191ade4f065b413cdaeda9a5c750cf2bda85d76381f4c4484da5a552c07a2a4c25cba0c38ce722826a497467a0fee15b23f82971cabda
d9fee292f4 Fix get_tx_info() type hint and doc (Kristaps Kaupe)
Pull request description:
`txid` is passed as binary `bytes` there, not hex string.
Top commit has no ACKs.
Tree-SHA512: 3e8272a5dc2291add827a06a5d0cda03652168e9d0cc3c9ec7532ace0522eef75c7e38c0dce6e665a65bb73d31579b2afda814fc18d0afcc99db311495642034
fab97a003b Use get_deser_from_gettransaction() everywhere (Kristaps Kaupe)
Pull request description:
There's `get_deser_from_gettransaction()` method in `BlockchainInterface` which is wrapper around python-bitcointx `CMutableTransaction.deserialize()` with some error checking added. It is used in most of the code, but in `jmclient/wallet_utils.py` I found two direct calls of `CMutableTransaction.deserialize()`.
Top commit has no ACKs.
Tree-SHA512: c0fc781e37b3bc72e89805e62b594ba879d5c2b84d3417dcefeba4428d21abcf0f44f143c1b9f5cf23a896e03add26b80b9cc514842142b45432c31adf563d7a
8f382d0d5e Add test for dict_factory() (Kristaps Kaupe)
2978b18245 De-duplicate and change dict_factory (Kristaps Kaupe)
Pull request description:
Move to `jmbase` and switch to implementation from [Python docs](https://docs.python.org/3/library/sqlite3.html#sqlite3-howto-row-factory), which seems nicer.
Top commit has no ACKs.
Tree-SHA512: 02ca21fea0eb45cc2f05d47e85b4150c300ad895aaf6102c90c433b38b95f8fb7b1e04292542afc6073fc22b14b07c38c01957f1380bed080585d9b67c0ee4bf
84914319cc cache deserialized form of txs in history method (Adam Gibson)
Pull request description:
Prior to this commit, while the result of the gettransaction rpc call was being cached so as to not have to repeat these rpc calls, the deserialized form of the transaction created by a call to CMutableTransaction.deserialize, was not, and since this call is rather expensive, the history method was running more slowly than needed. After this commit, we cache the deserialized form also, resulting in a speedup to the wallet_utils.wallet_fetch_history method.
ACKs for top commit:
kristapsk:
ACK 84914319cc
Tree-SHA512: 868a92dbd02c4c52f5b609b1d085bc1c56d959fb4399bc1f3166bc8bbe71727a39aa266a7ebd4f77cb96430bfbba1af51dbe6bc995ed54955bea3b54d10624b5
47acf6a057 Update libsecp256k1 to v0.4.1 (Kristaps Kaupe)
Pull request description:
https://github.com/bitcoin-core/secp256k1/releases/tag/v0.4.1
> This release slightly increases the speed of the ECDH operation and significantly enhances the performance of many library functions when using the default configuration on x86_64.
ABI is backwards compatible with v0.4.0, so this should not impact existing python-bitcointx.
ACKs for top commit:
AdamISZ:
ACK 47acf6a057
Tree-SHA512: 98c27ae2dc3c227e24a015bbb2709dcbaddc74ebe239de059c8172693ab2e71fbcff5f026b2d0a60d9b81c59f1c68bb08d0b3f17bfa38cd4456d9d08a3887409
Prior to this commit, while the result of the gettransaction rpc call
was being cached so as to not have to repeat these rpc calls, the
deserialized form of the transaction created by a call to
CMutableTransaction.deserialize, was not, and since this call is rather
expensive, the history method was running more slowly than needed.
After this commit, we cache the deserialized form also, resulting in a
speedup to the wallet_utils.wallet_fetch_history method.
d234731022 Add -allowignoredconf=1 for Bitcoin Core v26+ (Kristaps Kaupe)
8684853117 Support Bitcoin Core v26 for tests (Kristaps Kaupe)
Pull request description:
Bitcoin Core config needs to be hacked with `deprecatedrpc=create_bdb` for v26 until #1571 is solved.
Top commit has no ACKs.
Tree-SHA512: cc30afe05af7df2c73f60436ee43efe6c2cbda50539052a28bd72fa27dcab91eb6d0c0ba1d42cf37d589f9e00bbc87dbf05d9ac34acc559387aa05b38f18f87f
9410b9cbfa Update libsecp256k1 and python-bitcointx (Kristaps Kaupe)
Pull request description:
See https://github.com/Simplexum/python-bitcointx/issues/75.
This updates libsecp256k1 to latest v0.3.2 release and python-bitcointx to current master. Currently for testing purposes only, for merging into master we will need to wait for a new python-bitcointx release.
ACKs for top commit:
AdamISZ:
tACK 9410b9cbfa
roshii:
tACK 9410b9cbfa
Tree-SHA512: 4c87ed7b896445cb200deeb284ee6ef0aa52ce2bb88954d34dd8f13b76e3a0693aa8ee6f77f9172138e40dec44cff503d2168cce5092610b1a7a1d42530ea02b
79e5c3d0a7 Document Core wallet creation for v26 (Kristaps Kaupe)
Pull request description:
Creation of legacy (BDB) wallet have been deprecated with Bitcoin Core 26.0 release. Proper solution is to [implement descriptor wallet support](https://github.com/JoinMarket-Org/joinmarket-clientserver/issues/1571), but for now let's update documentation.
Also we can't update tests to v26 because of this (we could in theory, but it's unnecessary complexity to maintain two different `bitcoin.conf`'s for different Core versions just because of tests).
Top commit has no ACKs.
Tree-SHA512: 90b6479f21ea7131f080a67aab122970b5ed097fa261ad9a5e4d05aa207c77722a2790d0ce317d8c746e395c421e456b992161d81168cbc758951cb57e8bde2f
ef1d76e9db Allow cache purging at wallet initialization (Adam Gibson)
Pull request description:
Fixes#1614.
Prior to this commit, if data in the persisted cache in the wallet file were wrong (should be a very extraordinary case), then the joinmarket code would have to crash with a cache invalid warning. After this commit, in such an extraordinary case, the option exists to invalidate or remove the cache on startup, so that it can be rebuilt from scratch. This is done with a config var wallet_caching_disabled in the POLICY section.
ACKs for top commit:
kristapsk:
utACK ef1d76e9db
Tree-SHA512: dea2256c7246417ab17be0c89d6b64f49243f35204a8d316ab520f94451d149b3131c928be020f7152f99e0eb5f07843341bb48fca5aaef4c6517701a122b355
Fixes#1614.
Prior to this commit, if data in the persisted cache in the wallet file
were wrong (should be a very extraordinary case), then the joinmarket
code would have to crash with a cache invalid warning. After this
commit, in such an extraordinary case, the option exists to invalidate
or remove the cache on startup, so that it can be rebuilt from scratch.
This is done with a config var wallet_caching_disabled in the POLICY
section.
7ebbacf1f4 Add type hints (Kristaps Kaupe)
Pull request description:
It's sometimes especially time consuming with this part of JM codebase for me to figure out types for function arguments and return values, usually I end up looking at `python-bitcointx` code.
Changed return type of `get_equal_outs()` from `Union[bool, List[CTxOut]]` to `Optional[List[CTxOut]]`, only caller (`scripts/snicker/snicker-recovery.py`) already handled this.
ACKs for top commit:
roshii:
utACK 7ebbacf1f4
Tree-SHA512: 675167ef80036c68fbb7f8c6c350dfa17d140e95a4e522c44108a97ca5765ea0fb684d5fdc549e41398487325b31b067388706956d934565f977d50b39a05d72
8555a8b4a6 Removed donation link (Adam Gibson)
Pull request description:
The site linked to here is not under the control of any of the current active developers and any donation going here will not be support for development (this has been the status for a long while, now).
ACKs for top commit:
kristapsk:
ACK 8555a8b4a6
Tree-SHA512: 57bf8813a635a7d99d86ee31a4365fa4fb64a9a6a857c1a82784191f5f8993cc6f308fdf28092ae3d3ab06444321e0637e911f165e3d7c969816fc302e5deff7
The site linked to here is not under the control of any of the current
active developers and any donation going here will not be support for
development (this has been the status for a long while, now).
Kristaps Kaupe
Adam Gibson
d11n