d9fee292f4 Fix get_tx_info() type hint and doc (Kristaps Kaupe)
Pull request description:
`txid` is passed as binary `bytes` there, not hex string.
Top commit has no ACKs.
Tree-SHA512: 3e8272a5dc2291add827a06a5d0cda03652168e9d0cc3c9ec7532ace0522eef75c7e38c0dce6e665a65bb73d31579b2afda814fc18d0afcc99db311495642034
fab97a003b Use get_deser_from_gettransaction() everywhere (Kristaps Kaupe)
Pull request description:
There's `get_deser_from_gettransaction()` method in `BlockchainInterface` which is wrapper around python-bitcointx `CMutableTransaction.deserialize()` with some error checking added. It is used in most of the code, but in `jmclient/wallet_utils.py` I found two direct calls of `CMutableTransaction.deserialize()`.
Top commit has no ACKs.
Tree-SHA512: c0fc781e37b3bc72e89805e62b594ba879d5c2b84d3417dcefeba4428d21abcf0f44f143c1b9f5cf23a896e03add26b80b9cc514842142b45432c31adf563d7a
8f382d0d5e Add test for dict_factory() (Kristaps Kaupe)
2978b18245 De-duplicate and change dict_factory (Kristaps Kaupe)
Pull request description:
Move to `jmbase` and switch to implementation from [Python docs](https://docs.python.org/3/library/sqlite3.html#sqlite3-howto-row-factory), which seems nicer.
Top commit has no ACKs.
Tree-SHA512: 02ca21fea0eb45cc2f05d47e85b4150c300ad895aaf6102c90c433b38b95f8fb7b1e04292542afc6073fc22b14b07c38c01957f1380bed080585d9b67c0ee4bf
84914319cc cache deserialized form of txs in history method (Adam Gibson)
Pull request description:
Prior to this commit, while the result of the gettransaction rpc call was being cached so as to not have to repeat these rpc calls, the deserialized form of the transaction created by a call to CMutableTransaction.deserialize, was not, and since this call is rather expensive, the history method was running more slowly than needed. After this commit, we cache the deserialized form also, resulting in a speedup to the wallet_utils.wallet_fetch_history method.
ACKs for top commit:
kristapsk:
ACK 84914319cc
Tree-SHA512: 868a92dbd02c4c52f5b609b1d085bc1c56d959fb4399bc1f3166bc8bbe71727a39aa266a7ebd4f77cb96430bfbba1af51dbe6bc995ed54955bea3b54d10624b5
47acf6a057 Update libsecp256k1 to v0.4.1 (Kristaps Kaupe)
Pull request description:
https://github.com/bitcoin-core/secp256k1/releases/tag/v0.4.1
> This release slightly increases the speed of the ECDH operation and significantly enhances the performance of many library functions when using the default configuration on x86_64.
ABI is backwards compatible with v0.4.0, so this should not impact existing python-bitcointx.
ACKs for top commit:
AdamISZ:
ACK 47acf6a057
Tree-SHA512: 98c27ae2dc3c227e24a015bbb2709dcbaddc74ebe239de059c8172693ab2e71fbcff5f026b2d0a60d9b81c59f1c68bb08d0b3f17bfa38cd4456d9d08a3887409
Prior to this commit, while the result of the gettransaction rpc call
was being cached so as to not have to repeat these rpc calls, the
deserialized form of the transaction created by a call to
CMutableTransaction.deserialize, was not, and since this call is rather
expensive, the history method was running more slowly than needed.
After this commit, we cache the deserialized form also, resulting in a
speedup to the wallet_utils.wallet_fetch_history method.
Prior to this commit, the calls to get_new_addr in the functions in
the initial sync algo used for recovery, used the default value of the
argument validate_cache, which is True (because in normal running,
get_new_addr is used to derive addresses as destinations, for which it's
safer to not use the cache, and as one-off calls, are not
performance-sensitive). This caused initial sync to be very slow in
recovery, especially if using large gap limits (which is common).
After this commit, we set the argument validate_cache to False, as is
intended during initial sync. This allows the optimised performance from
caching to be in effect. See earlier PRs 1594 and 1614 for context.
d234731022 Add -allowignoredconf=1 for Bitcoin Core v26+ (Kristaps Kaupe)
8684853117 Support Bitcoin Core v26 for tests (Kristaps Kaupe)
Pull request description:
Bitcoin Core config needs to be hacked with `deprecatedrpc=create_bdb` for v26 until #1571 is solved.
Top commit has no ACKs.
Tree-SHA512: cc30afe05af7df2c73f60436ee43efe6c2cbda50539052a28bd72fa27dcab91eb6d0c0ba1d42cf37d589f9e00bbc87dbf05d9ac34acc559387aa05b38f18f87f
9410b9cbfa Update libsecp256k1 and python-bitcointx (Kristaps Kaupe)
Pull request description:
See https://github.com/Simplexum/python-bitcointx/issues/75.
This updates libsecp256k1 to latest v0.3.2 release and python-bitcointx to current master. Currently for testing purposes only, for merging into master we will need to wait for a new python-bitcointx release.
ACKs for top commit:
AdamISZ:
tACK 9410b9cbfa
roshii:
tACK 9410b9cbfa
Tree-SHA512: 4c87ed7b896445cb200deeb284ee6ef0aa52ce2bb88954d34dd8f13b76e3a0693aa8ee6f77f9172138e40dec44cff503d2168cce5092610b1a7a1d42530ea02b
79e5c3d0a7 Document Core wallet creation for v26 (Kristaps Kaupe)
Pull request description:
Creation of legacy (BDB) wallet have been deprecated with Bitcoin Core 26.0 release. Proper solution is to [implement descriptor wallet support](https://github.com/JoinMarket-Org/joinmarket-clientserver/issues/1571), but for now let's update documentation.
Also we can't update tests to v26 because of this (we could in theory, but it's unnecessary complexity to maintain two different `bitcoin.conf`'s for different Core versions just because of tests).
Top commit has no ACKs.
Tree-SHA512: 90b6479f21ea7131f080a67aab122970b5ed097fa261ad9a5e4d05aa207c77722a2790d0ce317d8c746e395c421e456b992161d81168cbc758951cb57e8bde2f
ef1d76e9db Allow cache purging at wallet initialization (Adam Gibson)
Pull request description:
Fixes#1614.
Prior to this commit, if data in the persisted cache in the wallet file were wrong (should be a very extraordinary case), then the joinmarket code would have to crash with a cache invalid warning. After this commit, in such an extraordinary case, the option exists to invalidate or remove the cache on startup, so that it can be rebuilt from scratch. This is done with a config var wallet_caching_disabled in the POLICY section.
ACKs for top commit:
kristapsk:
utACK ef1d76e9db
Tree-SHA512: dea2256c7246417ab17be0c89d6b64f49243f35204a8d316ab520f94451d149b3131c928be020f7152f99e0eb5f07843341bb48fca5aaef4c6517701a122b355
Fixes#1614.
Prior to this commit, if data in the persisted cache in the wallet file
were wrong (should be a very extraordinary case), then the joinmarket
code would have to crash with a cache invalid warning. After this
commit, in such an extraordinary case, the option exists to invalidate
or remove the cache on startup, so that it can be rebuilt from scratch.
This is done with a config var wallet_caching_disabled in the POLICY
section.
7ebbacf1f4 Add type hints (Kristaps Kaupe)
Pull request description:
It's sometimes especially time consuming with this part of JM codebase for me to figure out types for function arguments and return values, usually I end up looking at `python-bitcointx` code.
Changed return type of `get_equal_outs()` from `Union[bool, List[CTxOut]]` to `Optional[List[CTxOut]]`, only caller (`scripts/snicker/snicker-recovery.py`) already handled this.
ACKs for top commit:
roshii:
utACK 7ebbacf1f4
Tree-SHA512: 675167ef80036c68fbb7f8c6c350dfa17d140e95a4e522c44108a97ca5765ea0fb684d5fdc549e41398487325b31b067388706956d934565f977d50b39a05d72
8555a8b4a6 Removed donation link (Adam Gibson)
Pull request description:
The site linked to here is not under the control of any of the current active developers and any donation going here will not be support for development (this has been the status for a long while, now).
ACKs for top commit:
kristapsk:
ACK 8555a8b4a6
Tree-SHA512: 57bf8813a635a7d99d86ee31a4365fa4fb64a9a6a857c1a82784191f5f8993cc6f308fdf28092ae3d3ab06444321e0637e911f165e3d7c969816fc302e5deff7
The site linked to here is not under the control of any of the current
active developers and any donation going here will not be support for
development (this has been the status for a long while, now).
c3c10f1615 wallet: implement optional cache validation (Matt Whitlock)
5bc7eb4b8e wallet: add persistent cache, mapping path->(priv, pub, script, addr) (Matt Whitlock)
01ec2a4181 wallet: add _addr_map, paralleling _script_map (Matt Whitlock)
64f18bce18 get_imported_privkey_branch: use O(m+n) algorithm instead of O(m*n) (Matt Whitlock)
77f0194a37 wallet_utils: use new get_utxos_at_mixdepth method (Matt Whitlock)
184d76f7f7 wallet: add get_{balance,utxos}_at_mixdepth methods (Matt Whitlock)
fc1e00058b wallet_showutxos: use O(1) check for frozen instead of O(n) (Matt Whitlock)
b58ac679cb wallet: drop _get_addr_int_ext; replace with calls to get_new_addr (Matt Whitlock)
2c38a813fc wallet: delete redundant get_script and get_addr methods (Matt Whitlock)
574c29e899 wallet: hoist get_script_from_path default impl into BaseWallet (Matt Whitlock)
8245271d7f wallet: avoid IndexError in _is_my_bip32_path (Matt Whitlock)
48aec83d76 wallet: remove a dead store in get_index_cache_and_increment (Matt Whitlock)
Pull request description:
**Note:** Reviewing each commit individually will make more sense than trying to review the combined diff.
This PR implements several performance enhancements that take the CPU time to run `wallet-tool.py display` on my wallet down from ~44 minutes to ~11 seconds.
The most significant gains come from replacing an **O**(_m_*_n_) algorithm in `get_imported_privkey_branch` with a semantically equivalent **O**(_m_+_n_) algorithm and from adding a persistent cache for computed private keys, public keys, scripts, and addresses.
Below are some actual benchmarks on my wallet, which has 5 mixdepths, each having path indices reaching into the 4000s, and almost 700 imported private keys.
* 673fbfb9a5 `origin/master` (baseline)
```
user 44m3.618s
sys 0m6.375s
```
* 48aec83d76 `wallet`: remove a dead store in `get_index_cache_and_increment`
* fbb681a207be465fb53b43ac18a2b52c8a4a6323 `wallet`: add `get_{balance,utxos}_at_mixdepth` methods
* 75a970378579bb04f189e8d9eca22e5e2aadb0b4 `wallet_utils`: use new `get_utxos_at_mixdepth` method
```
user 42m14.464s
sys 0m3.355s
```
* 84966e628d510ddf0cadba170346ea926dc06000 `wallet_showutxos`: use **O**(1) check for frozen instead of **O**(_n_)
* 75c5a75468a6de88e64c4af7a8226c633d358fd5 `get_imported_privkey_branch`: use **O**(_m_+_n_) algorithm instead of **O**(_m_*_n_)
```
user 5m0.045s
sys 0m0.453s
```
* da8daf048369081d882fb591d50583559a2284f0 `wallet`: add `_addr_map`, paralleling `_script_map`
```
user 4m56.175s
sys 0m0.423s
```
* d8aa1afe6f0ec596bb133f594ae88cc2fffb6ad2 `wallet`: add persistent cache, mapping path->(priv, pub, script, addr)
```
user 1m42.272s
sys 0m0.471s
```
* After running another command to modify the wallet file so as to persist the cache, `wallet-tool.py display` now runs in:
```
user 0m11.141s
sys 0m0.225s
```
ACKs for top commit:
AdamISZ:
tACK c3c10f1615
Tree-SHA512: fdd20d436d8f16a1e4270011ad1ba4bf6393f876eb7413da30f75d5830249134911d5d93cab8051c0bf107c213d4cd46ba9614ae23eef4566f867ff1b912fc9b
b2c5603b4e Bump twisted from 22.4.0 to 23.8.0 (dependabot[bot])
Pull request description:
Bumps [twisted](https://github.com/twisted/twisted) from 22.4.0 to 23.8.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/twisted/twisted/releases">twisted's releases</a>.</em></p>
<blockquote>
<h1>Twisted 23.8.0. (2023-08-28)</h1>
<p>This is the last release with support for Python 3.7.</p>
<p>No changes since 23.8.0.rc1.</p>
<h2>Features</h2>
<ul>
<li>reactor.spawnProcess() now uses posix_spawnp when possible, making it much more efficient (<a href="https://redirect.github.com/twisted/twisted/issues/5710">#5710</a>)</li>
<li>Twisted now officially supports Python 3.11. (<a href="https://redirect.github.com/twisted/twisted/issues/10343">#10343</a>)</li>
<li>twisted.internet.defer.Deferred.fromFuture now has a more precise type annotation. (<a href="https://redirect.github.com/twisted/twisted/issues/11753">#11753</a>)</li>
<li>twisted.internet.defer._ConcurrencyPrimitive.<strong>aexit</strong> now has a more precise type annotation. (<a href="https://redirect.github.com/twisted/twisted/issues/11795">#11795</a>)</li>
<li><code>twisted.internet.defer.race</code> has been added as a way to get the first available result from a list of Deferreds. (<a href="https://redirect.github.com/twisted/twisted/issues/11817">#11817</a>)</li>
<li>The CI suite was updated to execute the tests using a Python 3.12 pre-release (<a href="https://redirect.github.com/twisted/twisted/issues/11857">#11857</a>)</li>
</ul>
<h2>Bugfixes</h2>
<ul>
<li>twisted.conch.scripts.ckeygen now substitutes a default of "~/.ssh/id_rsa" if no keyfile is specified. (<a href="https://redirect.github.com/twisted/twisted/issues/6607">#6607</a>)</li>
<li>Correct type hints for <code>IHostnameResolver.resolveHostName</code> and <code>IResolverSimple.getHostByName</code>. (<a href="https://redirect.github.com/twisted/twisted/issues/10276">#10276</a>)</li>
<li><code>twist conch --auth=sshkey</code> can now authenticate users without a traceback again, thanks to twisted.conch.unix.UnixConchUser no longer being incorrectly instantiated with <code>bytes</code>. In the course of this fix, some type hinting has also been applied to <code>twisted.cred.portal</code>. (<a href="https://redirect.github.com/twisted/twisted/issues/11626">#11626</a>)</li>
<li>twisted.internet.gireactor now works with Gtk4, and is tested and supported in CI again. (<a href="https://redirect.github.com/twisted/twisted/issues/11705">#11705</a>)</li>
<li>When interrupted with control-C, <code>trial -j</code> no longer obscures tracebacks for
any errors caused by that interruption with an <code>UnboundLocalError</code> due to a bug
in its own implementation. Note that there are still several internal
tracebacks that will be emitted upon exiting, because tearing down the test
runner mid-suite is still not an entirely clean operation, but it should at
least be possible to see errors reported from, for example, a test that is
hanging more clearly. (<a href="https://redirect.github.com/twisted/twisted/issues/11707">#11707</a>)</li>
<li>PortableGIReactor and PortableGtkReactor are no longer necessary and are now aliases of GIReactor and Gtk2Reactor respectively, improving the performance of any applications using them. (<a href="https://redirect.github.com/twisted/twisted/issues/11738">#11738</a>)</li>
<li>The Twisted package dependencies were updated to minimum versions that
will work with latest Twisted codebase. (<a href="https://redirect.github.com/twisted/twisted/issues/11740">#11740</a>)</li>
<li>Deferred's type annotations have been made more comprehensive, precise, correct, and strict. You may notice new type errors in your applications; be sure to check on those because they may represent real type errors! (<a href="https://redirect.github.com/twisted/twisted/issues/11772">#11772</a>)</li>
<li>To prevent parsing errors and ensure validity when serializing HTML comments, twisted.web.template.flattenString has been updated to escape the --> sequence within comments. (<a href="https://redirect.github.com/twisted/twisted/issues/11804">#11804</a>)</li>
<li>BadZipfile (with a small f) has been deprecated since Python 3.2,
use BadZipFile (big F) instead, added in 3.2. (<a href="https://redirect.github.com/twisted/twisted/issues/11821">#11821</a>)</li>
<li><code>twisted.web.template</code> now avoids unnecessary copying and is faster, particularly for templates with deep nesting. (<a href="https://redirect.github.com/twisted/twisted/issues/11834">#11834</a>)</li>
<li><code>twisted.web.template</code> now avoids some unecessary evaluation of type annotations and is faster. (<a href="https://redirect.github.com/twisted/twisted/issues/11835">#11835</a>)</li>
<li>utcfromtimestamp has been deprecated since Python 3.12,
use fromtimestamp(x, timezone.utc).replace(tzinfo=None) instead. (<a href="https://redirect.github.com/twisted/twisted/issues/11908">#11908</a>)</li>
</ul>
<h2>Deprecations and Removals</h2>
<ul>
<li>Optional dependency "extras" names like <code>conch_nacl</code> now use hyphens rather than underscores to comply with PEP 685. The old names will be supported until the end of 2023. (<a href="https://redirect.github.com/twisted/twisted/issues/11655">#11655</a>)</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/twisted/twisted/blob/trunk/NEWS.rst">twisted's changelog</a>.</em></p>
<blockquote>
<p>This file contains the release notes for Twisted.</p>
<p>It only contains high-level changes that are of interest to Twisted library users.
Users of Twisted should check the notes before planning an upgrade.</p>
<p>Ticket numbers in this file can be looked up by visiting
<a href="https://twisted.org/trac/ticket/">https://twisted.org/trac/ticket/</a><!-- raw HTML omitted --></p>
<p>.. towncrier release notes start</p>
<h1>Twisted 23.8.0. (2023-08-28)</h1>
<p>This is the last release with support for Python 3.7.</p>
<p>No changes since 23.8.0.rc1.</p>
<h2>Features</h2>
<ul>
<li>reactor.spawnProcess() now uses posix_spawnp when possible, making it much more efficient (<a href="https://redirect.github.com/twisted/twisted/issues/5710">#5710</a>)</li>
<li>Twisted now officially supports Python 3.11. (<a href="https://redirect.github.com/twisted/twisted/issues/10343">#10343</a>)</li>
<li>twisted.internet.defer.Deferred.fromFuture now has a more precise type annotation. (<a href="https://redirect.github.com/twisted/twisted/issues/11753">#11753</a>)</li>
<li>twisted.internet.defer._ConcurrencyPrimitive.<strong>aexit</strong> now has a more precise type annotation. (<a href="https://redirect.github.com/twisted/twisted/issues/11795">#11795</a>)</li>
<li><code>twisted.internet.defer.race</code> has been added as a way to get the first available result from a list of Deferreds. (<a href="https://redirect.github.com/twisted/twisted/issues/11817">#11817</a>)</li>
<li>The CI suite was updated to execute the tests using a Python 3.12 pre-release (<a href="https://redirect.github.com/twisted/twisted/issues/11857">#11857</a>)</li>
</ul>
<h2>Bugfixes</h2>
<ul>
<li>twisted.conch.scripts.ckeygen now substitutes a default of "~/.ssh/id_rsa" if no keyfile is specified. (<a href="https://redirect.github.com/twisted/twisted/issues/6607">#6607</a>)</li>
<li>Correct type hints for <code>IHostnameResolver.resolveHostName</code> and <code>IResolverSimple.getHostByName</code>. (<a href="https://redirect.github.com/twisted/twisted/issues/10276">#10276</a>)</li>
<li><code>twist conch --auth=sshkey</code> can now authenticate users without a traceback again, thanks to twisted.conch.unix.UnixConchUser no longer being incorrectly instantiated with <code>bytes</code>. In the course of this fix, some type hinting has also been applied to <code>twisted.cred.portal</code>. (<a href="https://redirect.github.com/twisted/twisted/issues/11626">#11626</a>)</li>
<li>twisted.internet.gireactor now works with Gtk4, and is tested and supported in CI again. (<a href="https://redirect.github.com/twisted/twisted/issues/11705">#11705</a>)</li>
<li>When interrupted with control-C, <code>trial -j</code> no longer obscures tracebacks for
any errors caused by that interruption with an <code>UnboundLocalError</code> due to a bug
in its own implementation. Note that there are still several internal
tracebacks that will be emitted upon exiting, because tearing down the test
runner mid-suite is still not an entirely clean operation, but it should at
least be possible to see errors reported from, for example, a test that is
hanging more clearly. (<a href="https://redirect.github.com/twisted/twisted/issues/11707">#11707</a>)</li>
<li>PortableGIReactor and PortableGtkReactor are no longer necessary and are now aliases of GIReactor and Gtk2Reactor respectively, improving the performance of any applications using them. (<a href="https://redirect.github.com/twisted/twisted/issues/11738">#11738</a>)</li>
<li>The Twisted package dependencies were updated to minimum versions that
will work with latest Twisted codebase. (<a href="https://redirect.github.com/twisted/twisted/issues/11740">#11740</a>)</li>
<li>Deferred's type annotations have been made more comprehensive, precise, correct, and strict. You may notice new type errors in your applications; be sure to check on those because they may represent real type errors! (<a href="https://redirect.github.com/twisted/twisted/issues/11772">#11772</a>)</li>
<li>To prevent parsing errors and ensure validity when serializing HTML comments, twisted.web.template.flattenString has been updated to escape the --> sequence within comments. (<a href="https://redirect.github.com/twisted/twisted/issues/11804">#11804</a>)</li>
<li>BadZipfile (with a small f) has been deprecated since Python 3.2,</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="f3cf54306e"><code>f3cf543</code></a> Revert "No need to get the trunk branch, as this is done by actions/checkout."</li>
<li><a href="b3fbaecd13"><code>b3fbaec</code></a> No need to get the trunk branch, as this is done by actions/checkout.</li>
<li><a href="da5a16a3ec"><code>da5a16a</code></a> Run tests on push for release branches.</li>
<li><a href="a1019a3978"><code>a1019a3</code></a> Update final release version.</li>
<li><a href="bd63ecd1f4"><code>bd63ecd</code></a> Get full git history.</li>
<li><a href="2431593992"><code>2431593</code></a> use towncrier checks instead of custom code.</li>
<li><a href="6e9ea35f1c"><code>6e9ea35</code></a> Add info about python 3.7 support.</li>
<li><a href="7f7f423f4d"><code>7f7f423</code></a> tox -e towncrie</li>
<li><a href="30f014a58b"><code>30f014a</code></a> python -m incremental.update Twisted --rc</li>
<li><a href="9a2ea002e7"><code>9a2ea00</code></a> <a href="https://redirect.github.com/twisted/twisted/issues/11857">#11857</a> test on py 3.12rc (<a href="https://redirect.github.com/twisted/twisted/issues/11910">#11910</a>)</li>
<li>Additional commits viewable in <a href="https://github.com/twisted/twisted/compare/twisted-22.4.0...twisted-23.8.0">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
You can trigger a rebase of this PR by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/JoinMarket-Org/joinmarket-clientserver/network/alerts).
</details>
> **Note**
> Automatic rebases have been disabled on this pull request as it has been open for over 30 days.
Top commit has no ACKs.
Tree-SHA512: f5399ae841fd0c51bc86585df32bde7d11d87319358d88db621bd037aa841f2aba04adb31ee634eecc375f46239906d99472f56f9483f03640efd037604ba0da
f40bd649ce Bump cryptography from 41.0.4 to 41.0.6 (dependabot[bot])
Pull request description:
Bumps [cryptography](https://github.com/pyca/cryptography) from 41.0.4 to 41.0.6.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst">cryptography's changelog</a>.</em></p>
<blockquote>
<p>41.0.6 - 2023-11-27</p>
<pre><code>
* Fixed a null-pointer-dereference and segfault that could occur when loading
certificates from a PKCS#7 bundle. Credit to **pkuzco** for reporting the
issue. **CVE-2023-49083**
<p>.. _v41-0-5:</p>
<p>41.0.5 - 2023-10-24
</code></pre></p>
<ul>
<li>Updated Windows, macOS, and Linux wheels to be compiled with OpenSSL 3.1.4.</li>
<li>Added a function to support an upcoming <code>pyOpenSSL</code> release.</li>
</ul>
<p>.. _v41-0-4:</p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="f09c261ca1"><code>f09c261</code></a> 41.0.6 release (<a href="https://redirect.github.com/pyca/cryptography/issues/9927">#9927</a>)</li>
<li><a href="5012bedaef"><code>5012bed</code></a> bump for 41.0.5 release (<a href="https://redirect.github.com/pyca/cryptography/issues/9766">#9766</a>)</li>
<li><a href="563b119399"><code>563b119</code></a> Added binding needed for pyOpenSSL (<a href="https://redirect.github.com/pyca/cryptography/issues/9739">#9739</a>) (<a href="https://redirect.github.com/pyca/cryptography/issues/9740">#9740</a>)</li>
<li>See full diff in <a href="https://github.com/pyca/cryptography/compare/41.0.4...41.0.6">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/JoinMarket-Org/joinmarket-clientserver/network/alerts).
</details>
ACKs for top commit:
AdamISZ:
> ACK [f40bd64](f40bd649ce). This isn't affecting us in any way (we use it only for AES), but I don't see downsides in upgrading.
kristapsk:
ACK f40bd649ce. This isn't affecting us in any way (we use it only for AES), but I don't see downsides in upgrading.
Tree-SHA512: ab291fbca629f6657aa0cdf0594ec8d235a75e5246110dcdc3d1fb5c299ab182e08173b343fe8c5c45324621104ccba08a12d300c4cfee8f8e1fd75252ad98d2
bfc618a99e Fix OrderbookWatch.on_order_seen() exception debug messages (Kristaps Kaupe)
Pull request description:
You cannot concatenate `str` with `int`. Found while working on #1602.
ACKs for top commit:
AdamISZ:
utACK bfc618a99e
Tree-SHA512: 34e2181bd91c856348fee88233f76d25a51d1626d1ad3523e861caa3ba84c248371de874841038381839efa4816d58d2d0933628f79bcf7d64295483c7959dfc
c990a4da6b Allow fee bump tx not signalling BIP125 if mempoolfullrbf is enabled (Kristaps Kaupe)
Pull request description:
We should allow this if full-RBF is enabled in Bitcoin node.
ACKs for top commit:
AdamISZ:
utACK c990a4da6b
Tree-SHA512: a2bb2374e790891d437ee71647c15ad91eb855ae5bd99afa8e116b91d15bdd4a9f3694469a867f24a37612c38e4215bfb1d45beda8fc921b3dc4363d5d015fd0
d8f1fc42d4 Add optional txfee property for direct-send wallet RPC (Kristaps Kaupe)
Pull request description:
Resolves#1360. Jam wants it for https://github.com/joinmarket-webui/jam/pull/678.
ACKs for top commit:
AdamISZ:
tACK d8f1fc42d4
Tree-SHA512: aa5afc17c0a39d65094c69d847841295c101ed74518be25610378aa7eda95ee3e609f7ae49be75c3e9d148dd8f7787ac1ccc17aa8ee624d1cef3508fa70af114
Add a validate_cache parameter to the five principal caching methods:
- _get_key_from_path
- _get_keypair_from_path
- _get_pubkey_from_path
- get_script_from_path
- get_address_from_path
and to the five convenience methods that wrap the above:
- get_script
- get_addr
- script_to_addr
- get_new_script
- get_new_addr
The value of this new parameter defaults to False in all but the last
two methods, where we are willing to sacrifice speed for the sake of
extra confidence in the correctness of *new* scripts and addresses to
be used for new deposits and new transactions.
4486b10798 Transaction virtual size must be rounded upwards (Kristaps Kaupe)
Pull request description:
Transaction virtual size must be rounded upwards.
Top commit has no ACKs.
Tree-SHA512: 9541ec8754d48aacdc10dc0cfb4323b60c4514f8f61a2d827ee7354d58cf46db530b4b68850f450a58e2053a9f69504fd80ca9f4d3614b7df24bc8d38f09a7fe
Deriving private keys from BIP32 paths, public keys from private keys,
scripts from public keys, and addresses from scripts are some of the
most CPU-intensive tasks the wallet performs. Once the wallet inevitably
accumulates thousands of used paths, startup times become painful due to
needing to re-derive these data items for every used path in the wallet
upon every startup. Introduce a persistent cache to avoid the need to
re-derive these items every time the wallet is opened.
Introduce _get_keypair_from_path and _get_pubkey_from_path methods to
allow cached public keys to be used rather than always deriving them on
the fly.
Change many code paths that were calling CPU-intensive methods of
BTCEngine so that instead they call _get_key_from_path,
_get_keypair_from_path, _get_pubkey_from_path, get_script_from_path,
and/or get_address_from_path, all of which can take advantage of the new
cache.
Hoist _populate_script_map from BIP32Wallet into BaseWallet, rename it
to _populate_maps, and have it populate the new _addr_map in addition to
the existing _script_map. Have the constructor of each concrete wallet
subclass pass to _populate_maps the paths it contributes. Additionally,
do not implement yield_known_paths by iterating over _script_map, but
rather have each wallet subclass contribute its own paths to the
generator returned by yield_known_paths.
The algorithm in get_imported_privkey_branch was O(m*n): for each
imported path, it was iterating over the entire set of UTXOs. Rewrite
the algorithm to make one pass over the set of UTXOs up front to compute
the balance of each script (O(m)) and then, separately, one pass over
the set of imported paths to pluck out the balance for each path (O(n)).
Rather than evaluating wallet_service.get_utxos_by_mixdepth()[md],
instead evaluate wallet_service.get_utxos_at_mixdepth(md). This way
we're not computing a bunch of data that we'll immediately discard.
Sometimes calling code is only interested in the balance or UTXOs at a
single mixdepth. In these cases, it is wasteful to get the balance or
UTXOs at all mixdepths, only to throw away the returned information
about all but the single mixdepth of interest. Implement new methods in
BaseWallet to get the balance or UTXOs at a single mixdepth.
Also, correct an apparent oversight due to apparently misplaced
indentation: the maxheight parameter of get_balance_by_mixdepth was
ignored unless the include_disabled parameter was passed as False. It
appears that the intention was for include_disabled and maxheight to be
independent filters on the returned information.
Kristaps Kaupe
Adam Gibson
d11n
dependabot[bot]
Matt Whitlock