d234731022 Add -allowignoredconf=1 for Bitcoin Core v26+ (Kristaps Kaupe)
8684853117 Support Bitcoin Core v26 for tests (Kristaps Kaupe)
Pull request description:
Bitcoin Core config needs to be hacked with `deprecatedrpc=create_bdb` for v26 until #1571 is solved.
Top commit has no ACKs.
Tree-SHA512: cc30afe05af7df2c73f60436ee43efe6c2cbda50539052a28bd72fa27dcab91eb6d0c0ba1d42cf37d589f9e00bbc87dbf05d9ac34acc559387aa05b38f18f87f
9410b9cbfa Update libsecp256k1 and python-bitcointx (Kristaps Kaupe)
Pull request description:
See https://github.com/Simplexum/python-bitcointx/issues/75.
This updates libsecp256k1 to latest v0.3.2 release and python-bitcointx to current master. Currently for testing purposes only, for merging into master we will need to wait for a new python-bitcointx release.
ACKs for top commit:
AdamISZ:
tACK 9410b9cbfa
roshii:
tACK 9410b9cbfa
Tree-SHA512: 4c87ed7b896445cb200deeb284ee6ef0aa52ce2bb88954d34dd8f13b76e3a0693aa8ee6f77f9172138e40dec44cff503d2168cce5092610b1a7a1d42530ea02b
79e5c3d0a7 Document Core wallet creation for v26 (Kristaps Kaupe)
Pull request description:
Creation of legacy (BDB) wallet have been deprecated with Bitcoin Core 26.0 release. Proper solution is to [implement descriptor wallet support](https://github.com/JoinMarket-Org/joinmarket-clientserver/issues/1571), but for now let's update documentation.
Also we can't update tests to v26 because of this (we could in theory, but it's unnecessary complexity to maintain two different `bitcoin.conf`'s for different Core versions just because of tests).
Top commit has no ACKs.
Tree-SHA512: 90b6479f21ea7131f080a67aab122970b5ed097fa261ad9a5e4d05aa207c77722a2790d0ce317d8c746e395c421e456b992161d81168cbc758951cb57e8bde2f
ef1d76e9db Allow cache purging at wallet initialization (Adam Gibson)
Pull request description:
Fixes#1614.
Prior to this commit, if data in the persisted cache in the wallet file were wrong (should be a very extraordinary case), then the joinmarket code would have to crash with a cache invalid warning. After this commit, in such an extraordinary case, the option exists to invalidate or remove the cache on startup, so that it can be rebuilt from scratch. This is done with a config var wallet_caching_disabled in the POLICY section.
ACKs for top commit:
kristapsk:
utACK ef1d76e9db
Tree-SHA512: dea2256c7246417ab17be0c89d6b64f49243f35204a8d316ab520f94451d149b3131c928be020f7152f99e0eb5f07843341bb48fca5aaef4c6517701a122b355
Fixes#1614.
Prior to this commit, if data in the persisted cache in the wallet file
were wrong (should be a very extraordinary case), then the joinmarket
code would have to crash with a cache invalid warning. After this
commit, in such an extraordinary case, the option exists to invalidate
or remove the cache on startup, so that it can be rebuilt from scratch.
This is done with a config var wallet_caching_disabled in the POLICY
section.
7ebbacf1f4 Add type hints (Kristaps Kaupe)
Pull request description:
It's sometimes especially time consuming with this part of JM codebase for me to figure out types for function arguments and return values, usually I end up looking at `python-bitcointx` code.
Changed return type of `get_equal_outs()` from `Union[bool, List[CTxOut]]` to `Optional[List[CTxOut]]`, only caller (`scripts/snicker/snicker-recovery.py`) already handled this.
ACKs for top commit:
roshii:
utACK 7ebbacf1f4
Tree-SHA512: 675167ef80036c68fbb7f8c6c350dfa17d140e95a4e522c44108a97ca5765ea0fb684d5fdc549e41398487325b31b067388706956d934565f977d50b39a05d72
8555a8b4a6 Removed donation link (Adam Gibson)
Pull request description:
The site linked to here is not under the control of any of the current active developers and any donation going here will not be support for development (this has been the status for a long while, now).
ACKs for top commit:
kristapsk:
ACK 8555a8b4a6
Tree-SHA512: 57bf8813a635a7d99d86ee31a4365fa4fb64a9a6a857c1a82784191f5f8993cc6f308fdf28092ae3d3ab06444321e0637e911f165e3d7c969816fc302e5deff7
The site linked to here is not under the control of any of the current
active developers and any donation going here will not be support for
development (this has been the status for a long while, now).
c3c10f1615 wallet: implement optional cache validation (Matt Whitlock)
5bc7eb4b8e wallet: add persistent cache, mapping path->(priv, pub, script, addr) (Matt Whitlock)
01ec2a4181 wallet: add _addr_map, paralleling _script_map (Matt Whitlock)
64f18bce18 get_imported_privkey_branch: use O(m+n) algorithm instead of O(m*n) (Matt Whitlock)
77f0194a37 wallet_utils: use new get_utxos_at_mixdepth method (Matt Whitlock)
184d76f7f7 wallet: add get_{balance,utxos}_at_mixdepth methods (Matt Whitlock)
fc1e00058b wallet_showutxos: use O(1) check for frozen instead of O(n) (Matt Whitlock)
b58ac679cb wallet: drop _get_addr_int_ext; replace with calls to get_new_addr (Matt Whitlock)
2c38a813fc wallet: delete redundant get_script and get_addr methods (Matt Whitlock)
574c29e899 wallet: hoist get_script_from_path default impl into BaseWallet (Matt Whitlock)
8245271d7f wallet: avoid IndexError in _is_my_bip32_path (Matt Whitlock)
48aec83d76 wallet: remove a dead store in get_index_cache_and_increment (Matt Whitlock)
Pull request description:
**Note:** Reviewing each commit individually will make more sense than trying to review the combined diff.
This PR implements several performance enhancements that take the CPU time to run `wallet-tool.py display` on my wallet down from ~44 minutes to ~11 seconds.
The most significant gains come from replacing an **O**(_m_*_n_) algorithm in `get_imported_privkey_branch` with a semantically equivalent **O**(_m_+_n_) algorithm and from adding a persistent cache for computed private keys, public keys, scripts, and addresses.
Below are some actual benchmarks on my wallet, which has 5 mixdepths, each having path indices reaching into the 4000s, and almost 700 imported private keys.
* 673fbfb9a5 `origin/master` (baseline)
```
user 44m3.618s
sys 0m6.375s
```
* 48aec83d76 `wallet`: remove a dead store in `get_index_cache_and_increment`
* fbb681a207be465fb53b43ac18a2b52c8a4a6323 `wallet`: add `get_{balance,utxos}_at_mixdepth` methods
* 75a970378579bb04f189e8d9eca22e5e2aadb0b4 `wallet_utils`: use new `get_utxos_at_mixdepth` method
```
user 42m14.464s
sys 0m3.355s
```
* 84966e628d510ddf0cadba170346ea926dc06000 `wallet_showutxos`: use **O**(1) check for frozen instead of **O**(_n_)
* 75c5a75468a6de88e64c4af7a8226c633d358fd5 `get_imported_privkey_branch`: use **O**(_m_+_n_) algorithm instead of **O**(_m_*_n_)
```
user 5m0.045s
sys 0m0.453s
```
* da8daf048369081d882fb591d50583559a2284f0 `wallet`: add `_addr_map`, paralleling `_script_map`
```
user 4m56.175s
sys 0m0.423s
```
* d8aa1afe6f0ec596bb133f594ae88cc2fffb6ad2 `wallet`: add persistent cache, mapping path->(priv, pub, script, addr)
```
user 1m42.272s
sys 0m0.471s
```
* After running another command to modify the wallet file so as to persist the cache, `wallet-tool.py display` now runs in:
```
user 0m11.141s
sys 0m0.225s
```
ACKs for top commit:
AdamISZ:
tACK c3c10f1615
Tree-SHA512: fdd20d436d8f16a1e4270011ad1ba4bf6393f876eb7413da30f75d5830249134911d5d93cab8051c0bf107c213d4cd46ba9614ae23eef4566f867ff1b912fc9b
b2c5603b4e Bump twisted from 22.4.0 to 23.8.0 (dependabot[bot])
Pull request description:
Bumps [twisted](https://github.com/twisted/twisted) from 22.4.0 to 23.8.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/twisted/twisted/releases">twisted's releases</a>.</em></p>
<blockquote>
<h1>Twisted 23.8.0. (2023-08-28)</h1>
<p>This is the last release with support for Python 3.7.</p>
<p>No changes since 23.8.0.rc1.</p>
<h2>Features</h2>
<ul>
<li>reactor.spawnProcess() now uses posix_spawnp when possible, making it much more efficient (<a href="https://redirect.github.com/twisted/twisted/issues/5710">#5710</a>)</li>
<li>Twisted now officially supports Python 3.11. (<a href="https://redirect.github.com/twisted/twisted/issues/10343">#10343</a>)</li>
<li>twisted.internet.defer.Deferred.fromFuture now has a more precise type annotation. (<a href="https://redirect.github.com/twisted/twisted/issues/11753">#11753</a>)</li>
<li>twisted.internet.defer._ConcurrencyPrimitive.<strong>aexit</strong> now has a more precise type annotation. (<a href="https://redirect.github.com/twisted/twisted/issues/11795">#11795</a>)</li>
<li><code>twisted.internet.defer.race</code> has been added as a way to get the first available result from a list of Deferreds. (<a href="https://redirect.github.com/twisted/twisted/issues/11817">#11817</a>)</li>
<li>The CI suite was updated to execute the tests using a Python 3.12 pre-release (<a href="https://redirect.github.com/twisted/twisted/issues/11857">#11857</a>)</li>
</ul>
<h2>Bugfixes</h2>
<ul>
<li>twisted.conch.scripts.ckeygen now substitutes a default of "~/.ssh/id_rsa" if no keyfile is specified. (<a href="https://redirect.github.com/twisted/twisted/issues/6607">#6607</a>)</li>
<li>Correct type hints for <code>IHostnameResolver.resolveHostName</code> and <code>IResolverSimple.getHostByName</code>. (<a href="https://redirect.github.com/twisted/twisted/issues/10276">#10276</a>)</li>
<li><code>twist conch --auth=sshkey</code> can now authenticate users without a traceback again, thanks to twisted.conch.unix.UnixConchUser no longer being incorrectly instantiated with <code>bytes</code>. In the course of this fix, some type hinting has also been applied to <code>twisted.cred.portal</code>. (<a href="https://redirect.github.com/twisted/twisted/issues/11626">#11626</a>)</li>
<li>twisted.internet.gireactor now works with Gtk4, and is tested and supported in CI again. (<a href="https://redirect.github.com/twisted/twisted/issues/11705">#11705</a>)</li>
<li>When interrupted with control-C, <code>trial -j</code> no longer obscures tracebacks for
any errors caused by that interruption with an <code>UnboundLocalError</code> due to a bug
in its own implementation. Note that there are still several internal
tracebacks that will be emitted upon exiting, because tearing down the test
runner mid-suite is still not an entirely clean operation, but it should at
least be possible to see errors reported from, for example, a test that is
hanging more clearly. (<a href="https://redirect.github.com/twisted/twisted/issues/11707">#11707</a>)</li>
<li>PortableGIReactor and PortableGtkReactor are no longer necessary and are now aliases of GIReactor and Gtk2Reactor respectively, improving the performance of any applications using them. (<a href="https://redirect.github.com/twisted/twisted/issues/11738">#11738</a>)</li>
<li>The Twisted package dependencies were updated to minimum versions that
will work with latest Twisted codebase. (<a href="https://redirect.github.com/twisted/twisted/issues/11740">#11740</a>)</li>
<li>Deferred's type annotations have been made more comprehensive, precise, correct, and strict. You may notice new type errors in your applications; be sure to check on those because they may represent real type errors! (<a href="https://redirect.github.com/twisted/twisted/issues/11772">#11772</a>)</li>
<li>To prevent parsing errors and ensure validity when serializing HTML comments, twisted.web.template.flattenString has been updated to escape the --> sequence within comments. (<a href="https://redirect.github.com/twisted/twisted/issues/11804">#11804</a>)</li>
<li>BadZipfile (with a small f) has been deprecated since Python 3.2,
use BadZipFile (big F) instead, added in 3.2. (<a href="https://redirect.github.com/twisted/twisted/issues/11821">#11821</a>)</li>
<li><code>twisted.web.template</code> now avoids unnecessary copying and is faster, particularly for templates with deep nesting. (<a href="https://redirect.github.com/twisted/twisted/issues/11834">#11834</a>)</li>
<li><code>twisted.web.template</code> now avoids some unecessary evaluation of type annotations and is faster. (<a href="https://redirect.github.com/twisted/twisted/issues/11835">#11835</a>)</li>
<li>utcfromtimestamp has been deprecated since Python 3.12,
use fromtimestamp(x, timezone.utc).replace(tzinfo=None) instead. (<a href="https://redirect.github.com/twisted/twisted/issues/11908">#11908</a>)</li>
</ul>
<h2>Deprecations and Removals</h2>
<ul>
<li>Optional dependency "extras" names like <code>conch_nacl</code> now use hyphens rather than underscores to comply with PEP 685. The old names will be supported until the end of 2023. (<a href="https://redirect.github.com/twisted/twisted/issues/11655">#11655</a>)</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/twisted/twisted/blob/trunk/NEWS.rst">twisted's changelog</a>.</em></p>
<blockquote>
<p>This file contains the release notes for Twisted.</p>
<p>It only contains high-level changes that are of interest to Twisted library users.
Users of Twisted should check the notes before planning an upgrade.</p>
<p>Ticket numbers in this file can be looked up by visiting
<a href="https://twisted.org/trac/ticket/">https://twisted.org/trac/ticket/</a><!-- raw HTML omitted --></p>
<p>.. towncrier release notes start</p>
<h1>Twisted 23.8.0. (2023-08-28)</h1>
<p>This is the last release with support for Python 3.7.</p>
<p>No changes since 23.8.0.rc1.</p>
<h2>Features</h2>
<ul>
<li>reactor.spawnProcess() now uses posix_spawnp when possible, making it much more efficient (<a href="https://redirect.github.com/twisted/twisted/issues/5710">#5710</a>)</li>
<li>Twisted now officially supports Python 3.11. (<a href="https://redirect.github.com/twisted/twisted/issues/10343">#10343</a>)</li>
<li>twisted.internet.defer.Deferred.fromFuture now has a more precise type annotation. (<a href="https://redirect.github.com/twisted/twisted/issues/11753">#11753</a>)</li>
<li>twisted.internet.defer._ConcurrencyPrimitive.<strong>aexit</strong> now has a more precise type annotation. (<a href="https://redirect.github.com/twisted/twisted/issues/11795">#11795</a>)</li>
<li><code>twisted.internet.defer.race</code> has been added as a way to get the first available result from a list of Deferreds. (<a href="https://redirect.github.com/twisted/twisted/issues/11817">#11817</a>)</li>
<li>The CI suite was updated to execute the tests using a Python 3.12 pre-release (<a href="https://redirect.github.com/twisted/twisted/issues/11857">#11857</a>)</li>
</ul>
<h2>Bugfixes</h2>
<ul>
<li>twisted.conch.scripts.ckeygen now substitutes a default of "~/.ssh/id_rsa" if no keyfile is specified. (<a href="https://redirect.github.com/twisted/twisted/issues/6607">#6607</a>)</li>
<li>Correct type hints for <code>IHostnameResolver.resolveHostName</code> and <code>IResolverSimple.getHostByName</code>. (<a href="https://redirect.github.com/twisted/twisted/issues/10276">#10276</a>)</li>
<li><code>twist conch --auth=sshkey</code> can now authenticate users without a traceback again, thanks to twisted.conch.unix.UnixConchUser no longer being incorrectly instantiated with <code>bytes</code>. In the course of this fix, some type hinting has also been applied to <code>twisted.cred.portal</code>. (<a href="https://redirect.github.com/twisted/twisted/issues/11626">#11626</a>)</li>
<li>twisted.internet.gireactor now works with Gtk4, and is tested and supported in CI again. (<a href="https://redirect.github.com/twisted/twisted/issues/11705">#11705</a>)</li>
<li>When interrupted with control-C, <code>trial -j</code> no longer obscures tracebacks for
any errors caused by that interruption with an <code>UnboundLocalError</code> due to a bug
in its own implementation. Note that there are still several internal
tracebacks that will be emitted upon exiting, because tearing down the test
runner mid-suite is still not an entirely clean operation, but it should at
least be possible to see errors reported from, for example, a test that is
hanging more clearly. (<a href="https://redirect.github.com/twisted/twisted/issues/11707">#11707</a>)</li>
<li>PortableGIReactor and PortableGtkReactor are no longer necessary and are now aliases of GIReactor and Gtk2Reactor respectively, improving the performance of any applications using them. (<a href="https://redirect.github.com/twisted/twisted/issues/11738">#11738</a>)</li>
<li>The Twisted package dependencies were updated to minimum versions that
will work with latest Twisted codebase. (<a href="https://redirect.github.com/twisted/twisted/issues/11740">#11740</a>)</li>
<li>Deferred's type annotations have been made more comprehensive, precise, correct, and strict. You may notice new type errors in your applications; be sure to check on those because they may represent real type errors! (<a href="https://redirect.github.com/twisted/twisted/issues/11772">#11772</a>)</li>
<li>To prevent parsing errors and ensure validity when serializing HTML comments, twisted.web.template.flattenString has been updated to escape the --> sequence within comments. (<a href="https://redirect.github.com/twisted/twisted/issues/11804">#11804</a>)</li>
<li>BadZipfile (with a small f) has been deprecated since Python 3.2,</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="f3cf54306e"><code>f3cf543</code></a> Revert "No need to get the trunk branch, as this is done by actions/checkout."</li>
<li><a href="b3fbaecd13"><code>b3fbaec</code></a> No need to get the trunk branch, as this is done by actions/checkout.</li>
<li><a href="da5a16a3ec"><code>da5a16a</code></a> Run tests on push for release branches.</li>
<li><a href="a1019a3978"><code>a1019a3</code></a> Update final release version.</li>
<li><a href="bd63ecd1f4"><code>bd63ecd</code></a> Get full git history.</li>
<li><a href="2431593992"><code>2431593</code></a> use towncrier checks instead of custom code.</li>
<li><a href="6e9ea35f1c"><code>6e9ea35</code></a> Add info about python 3.7 support.</li>
<li><a href="7f7f423f4d"><code>7f7f423</code></a> tox -e towncrie</li>
<li><a href="30f014a58b"><code>30f014a</code></a> python -m incremental.update Twisted --rc</li>
<li><a href="9a2ea002e7"><code>9a2ea00</code></a> <a href="https://redirect.github.com/twisted/twisted/issues/11857">#11857</a> test on py 3.12rc (<a href="https://redirect.github.com/twisted/twisted/issues/11910">#11910</a>)</li>
<li>Additional commits viewable in <a href="https://github.com/twisted/twisted/compare/twisted-22.4.0...twisted-23.8.0">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
You can trigger a rebase of this PR by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/JoinMarket-Org/joinmarket-clientserver/network/alerts).
</details>
> **Note**
> Automatic rebases have been disabled on this pull request as it has been open for over 30 days.
Top commit has no ACKs.
Tree-SHA512: f5399ae841fd0c51bc86585df32bde7d11d87319358d88db621bd037aa841f2aba04adb31ee634eecc375f46239906d99472f56f9483f03640efd037604ba0da
f40bd649ce Bump cryptography from 41.0.4 to 41.0.6 (dependabot[bot])
Pull request description:
Bumps [cryptography](https://github.com/pyca/cryptography) from 41.0.4 to 41.0.6.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst">cryptography's changelog</a>.</em></p>
<blockquote>
<p>41.0.6 - 2023-11-27</p>
<pre><code>
* Fixed a null-pointer-dereference and segfault that could occur when loading
certificates from a PKCS#7 bundle. Credit to **pkuzco** for reporting the
issue. **CVE-2023-49083**
<p>.. _v41-0-5:</p>
<p>41.0.5 - 2023-10-24
</code></pre></p>
<ul>
<li>Updated Windows, macOS, and Linux wheels to be compiled with OpenSSL 3.1.4.</li>
<li>Added a function to support an upcoming <code>pyOpenSSL</code> release.</li>
</ul>
<p>.. _v41-0-4:</p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="f09c261ca1"><code>f09c261</code></a> 41.0.6 release (<a href="https://redirect.github.com/pyca/cryptography/issues/9927">#9927</a>)</li>
<li><a href="5012bedaef"><code>5012bed</code></a> bump for 41.0.5 release (<a href="https://redirect.github.com/pyca/cryptography/issues/9766">#9766</a>)</li>
<li><a href="563b119399"><code>563b119</code></a> Added binding needed for pyOpenSSL (<a href="https://redirect.github.com/pyca/cryptography/issues/9739">#9739</a>) (<a href="https://redirect.github.com/pyca/cryptography/issues/9740">#9740</a>)</li>
<li>See full diff in <a href="https://github.com/pyca/cryptography/compare/41.0.4...41.0.6">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/JoinMarket-Org/joinmarket-clientserver/network/alerts).
</details>
ACKs for top commit:
AdamISZ:
> ACK [f40bd64](f40bd649ce). This isn't affecting us in any way (we use it only for AES), but I don't see downsides in upgrading.
kristapsk:
ACK f40bd649ce. This isn't affecting us in any way (we use it only for AES), but I don't see downsides in upgrading.
Tree-SHA512: ab291fbca629f6657aa0cdf0594ec8d235a75e5246110dcdc3d1fb5c299ab182e08173b343fe8c5c45324621104ccba08a12d300c4cfee8f8e1fd75252ad98d2
bfc618a99e Fix OrderbookWatch.on_order_seen() exception debug messages (Kristaps Kaupe)
Pull request description:
You cannot concatenate `str` with `int`. Found while working on #1602.
ACKs for top commit:
AdamISZ:
utACK bfc618a99e
Tree-SHA512: 34e2181bd91c856348fee88233f76d25a51d1626d1ad3523e861caa3ba84c248371de874841038381839efa4816d58d2d0933628f79bcf7d64295483c7959dfc
c990a4da6b Allow fee bump tx not signalling BIP125 if mempoolfullrbf is enabled (Kristaps Kaupe)
Pull request description:
We should allow this if full-RBF is enabled in Bitcoin node.
ACKs for top commit:
AdamISZ:
utACK c990a4da6b
Tree-SHA512: a2bb2374e790891d437ee71647c15ad91eb855ae5bd99afa8e116b91d15bdd4a9f3694469a867f24a37612c38e4215bfb1d45beda8fc921b3dc4363d5d015fd0
d8f1fc42d4 Add optional txfee property for direct-send wallet RPC (Kristaps Kaupe)
Pull request description:
Resolves#1360. Jam wants it for https://github.com/joinmarket-webui/jam/pull/678.
ACKs for top commit:
AdamISZ:
tACK d8f1fc42d4
Tree-SHA512: aa5afc17c0a39d65094c69d847841295c101ed74518be25610378aa7eda95ee3e609f7ae49be75c3e9d148dd8f7787ac1ccc17aa8ee624d1cef3508fa70af114
Add a validate_cache parameter to the five principal caching methods:
- _get_key_from_path
- _get_keypair_from_path
- _get_pubkey_from_path
- get_script_from_path
- get_address_from_path
and to the five convenience methods that wrap the above:
- get_script
- get_addr
- script_to_addr
- get_new_script
- get_new_addr
The value of this new parameter defaults to False in all but the last
two methods, where we are willing to sacrifice speed for the sake of
extra confidence in the correctness of *new* scripts and addresses to
be used for new deposits and new transactions.
4486b10798 Transaction virtual size must be rounded upwards (Kristaps Kaupe)
Pull request description:
Transaction virtual size must be rounded upwards.
Top commit has no ACKs.
Tree-SHA512: 9541ec8754d48aacdc10dc0cfb4323b60c4514f8f61a2d827ee7354d58cf46db530b4b68850f450a58e2053a9f69504fd80ca9f4d3614b7df24bc8d38f09a7fe
Deriving private keys from BIP32 paths, public keys from private keys,
scripts from public keys, and addresses from scripts are some of the
most CPU-intensive tasks the wallet performs. Once the wallet inevitably
accumulates thousands of used paths, startup times become painful due to
needing to re-derive these data items for every used path in the wallet
upon every startup. Introduce a persistent cache to avoid the need to
re-derive these items every time the wallet is opened.
Introduce _get_keypair_from_path and _get_pubkey_from_path methods to
allow cached public keys to be used rather than always deriving them on
the fly.
Change many code paths that were calling CPU-intensive methods of
BTCEngine so that instead they call _get_key_from_path,
_get_keypair_from_path, _get_pubkey_from_path, get_script_from_path,
and/or get_address_from_path, all of which can take advantage of the new
cache.
Hoist _populate_script_map from BIP32Wallet into BaseWallet, rename it
to _populate_maps, and have it populate the new _addr_map in addition to
the existing _script_map. Have the constructor of each concrete wallet
subclass pass to _populate_maps the paths it contributes. Additionally,
do not implement yield_known_paths by iterating over _script_map, but
rather have each wallet subclass contribute its own paths to the
generator returned by yield_known_paths.
The algorithm in get_imported_privkey_branch was O(m*n): for each
imported path, it was iterating over the entire set of UTXOs. Rewrite
the algorithm to make one pass over the set of UTXOs up front to compute
the balance of each script (O(m)) and then, separately, one pass over
the set of imported paths to pluck out the balance for each path (O(n)).
Rather than evaluating wallet_service.get_utxos_by_mixdepth()[md],
instead evaluate wallet_service.get_utxos_at_mixdepth(md). This way
we're not computing a bunch of data that we'll immediately discard.
Sometimes calling code is only interested in the balance or UTXOs at a
single mixdepth. In these cases, it is wasteful to get the balance or
UTXOs at all mixdepths, only to throw away the returned information
about all but the single mixdepth of interest. Implement new methods in
BaseWallet to get the balance or UTXOs at a single mixdepth.
Also, correct an apparent oversight due to apparently misplaced
indentation: the maxheight parameter of get_balance_by_mixdepth was
ignored unless the include_disabled parameter was passed as False. It
appears that the intention was for include_disabled and maxheight to be
independent filters on the returned information.
utxo_d = []
for k, v in disabled.items():
utxo_d.append(k)
{'frozen': True if u in utxo_d else False}
The above was inefficient. Replace with:
{'frozen': u in disabled}
Checking for existence of a key in a dict takes time proportional to
O(1), whereas checking for existence of an element in a list takes time
proportional to O(n).
1cb20d5ae6 Do not reinstall on test (roshii)
Pull request description:
Removing `pip install .[test]` step from test script. It does not make sense to reinstall everything when test are ran especially with editable install.
ACKs for top commit:
kristapsk:
re-ACK 1cb20d5ae6
Tree-SHA512: 7a35f1f7f2160b84e97819f0f91655c551e8343921f734a0092d3fdaad76d46afcf2f053ad55964120240314dced4557d55c44d299e4622a50b55851b4b5df73
4f4945e5c5 Test min and latest Python version only (roshii)
Pull request description:
Taking the same approach as `bitcoind` version testing, setting only minimum required and latest Python version in CI. Testing so many versions does not bring much added value IMHO.
ACKs for top commit:
kristapsk:
ACK 4f4945e5c5
Tree-SHA512: 7678783323828dad9e0899c557077344ba968cc716b6aeccc1f50a4a5eaa23d6b2bf89eb0a607c2e1c31a395e03b26b65187d8ae5386674fc512ae9790cc9560
638200d346 feat(rpc): add block height to session response (theborakompanioni)
Pull request description:
Adds the current block height to the `/session` response.
This can be useful for api clients to display a more sophisticated message on how long to wait for a given UTXO when it has less than 5 confirmations (for [sourcing commitments](https://github.com/JoinMarket-Org/joinmarket-clientserver/blob/master/docs/SOURCING-COMMITMENTS.md#wait-for-at-least-5-confirmations)). `/session` is usually polled, so when the block height changes, additional information can be reloaded (instead of polling the `/wallet/{walletname}/utxos` endpoint).
The value is only included for authenticated requests when a wallet is loaded.
Additionally, block height changes could be pushed via websocket. However, this is not included in this PR.
ACKs for top commit:
kristapsk:
utACK 638200d346
Tree-SHA512: f5275a469a69678709cd88683463da155f9319dbc8cd64d159a942bcdb644f6e6f41b3f47a5bcd48eba66f17d27cd9fbf3126378aa3135ed5f9bc6fdeb5e7215
Kristaps Kaupe
d11n
Adam Gibson
dependabot[bot]
Matt Whitlock
roshii