Instead of some functions operating with hex strings,
and others using bytes, this consolidates most things to use bytes.
This mainly focuses on bitcoin.py and transaction.py,
and then adapts the API usages in other files.
Notably,
- scripts,
- pubkeys,
- signatures
should be bytes in almost all places now.
> These kind of checks look incorrect and confused/confusing for me:
> 43a5d03426/electrum/gui/qml/qeinvoice.py (L388)
> 43a5d03426/electrum/gui/qt/main_window.py (L1801)
> For example, consider creating a local tx that spends all UTXOs in the wallet, and consolidates them to another ismine address.
> Such a tx basically does not change what wallet.get_balance() returns.
> but to the checks listed above, the confirmed balance of the wallet I guess should be 0?
I am only making this change as it makes it simpler to do manual testing of CPFP-ing *local* txs.
That is, by changing a few easy-to-find lines of code, I can make the GUI allow CPFP-ing a local tx, but
without this change it errors out with "unknown fee for parent transaction".
If one has an incoming local tx saved in the wallet, adb.get_tx_fee(txid) returns None
(if the tx gets into the mempool, it will use the server-reported value instead).
In contrast, wallet.get_tx_info(tx).fee calls both wallet.get_wallet_delta(tx) and adb.get_tx_fee(txid),
making it more expensive but more complete.
A new config API is introduced, and ~all of the codebase is adapted to it.
The old API is kept but mainly only for dynamic usage where its extra flexibility is needed.
Using examples, the old config API looked this:
```
>>> config.get("request_expiry", 86400)
604800
>>> config.set_key("request_expiry", 86400)
>>>
```
The new config API instead:
```
>>> config.WALLET_PAYREQ_EXPIRY_SECONDS
604800
>>> config.WALLET_PAYREQ_EXPIRY_SECONDS = 86400
>>>
```
The old API operated on arbitrary string keys, the new one uses
a static ~enum-like list of variables.
With the new API:
- there is a single centralised list of config variables, as opposed to
these being scattered all over
- no more duplication of default values (in the getters)
- there is now some (minimal for now) type-validation/conversion for
the config values
closes https://github.com/spesmilo/electrum/pull/5640
closes https://github.com/spesmilo/electrum/pull/5649
Note: there is yet a third API added here, for certain niche/abstract use-cases,
where we need a reference to the config variable itself.
It should only be used when needed:
```
>>> var = config.cv.WALLET_PAYREQ_EXPIRY_SECONDS
>>> var
<ConfigVarWithConfig key='request_expiry'>
>>> var.get()
604800
>>> var.set(3600)
>>> var.get_default_value()
86400
>>> var.is_set()
True
>>> var.is_modifiable()
True
```
Previously if a local tx got broadcast, it was still displayed as local
in the history until it got mined (or some other action forced a full refresh).
- they are now distinguished from local by the status text "in %d blocks"
- this status text needs updating occasionally: on new blocks,
and some lnwatcher interactions, hence new event: "adb_set_future_tx"
- wallet.add_input_info() previously had a fallback to download parent
prev txs from the network (after a lookup in wallet.db failed).
wallet.add_input_info() is not async, so the network request cannot
be done cleanly there and was really just a hack.
- tx.add_info_from_wallet() calls wallet.add_input_info() on each txin,
in which case these network requests were done sequentially, not concurrently
- the network part of wallet.add_input_info() is now split out into new method:
txin.add_info_from_network()
- in addition to tx.add_info_from_wallet(), there is now also tx.add_info_from_network()
- callers of old tx.add_info_from_wallet() should now called either
- tx.add_info_from_wallet(), then tx.add_info_from_network(), preferably in that order
- tx.add_info_from_wallet() alone is sufficient if the tx is complete,
or typically when not in a signing context
- callers of wallet.bump_fee and wallet.dscancel are now expected to have already
called tx.add_info_from_network(), as it cannot be done in a non-async context
(but for the common case of all-inputs-are-ismine, bump_fee/dscancel should work regardless)
- PartialTxInput.utxo was moved to the baseclass, TxInput.utxo
fixes https://github.com/spesmilo/electrum/issues/8240#8240 was triggering an AssertionError in wallet.get_invoice_status,
as code there was assuming conf >= 0. To trigger, force-close
a LN channel, and while the sweep is waiting on the CSV, try to
make a payment in the Send tab to the ismine change address used
for the sweep in the future_tx. (order of events can also be reversed)
- add a new event, 'adb_removed_tx'
- new wallet method: get_tx_parents
- number of parents is shown in coins tab
- detailed list of parents is shown in dialog
ShortIDs were originally designed for lightning channels, and are now
understood by some block explorers.
This allows to remove one column in the UTXO tab (height is redundant).
In the transaction dialog, the space saving ensures that all inputs fit
into one line (it was not the case previously with p2wsh addresses).
For clarity and consistency, the ShortID is displayed for both inputs
and outputs in the transaction dialog.
- AddressSynchronizer no longer has its own state re up_to_date,
it defers to Synchronizer/Verifier instead
- Synchronizer is now tracking address sync states throughout their lifecycle:
and Synchronizer.is_up_to_date() checks all states
- Synchronizer.add_queue (internal) is removed as it was redundant
- should fix wallet.is_up_to_date flickering during sync due to race
related:
dc6c4814061c20a29a22
This reverts commit dc6c481406 as it introduced its own issue:
while add_address was running on one thread, synchronizer._reset could be running on another,
and by the time the "enqueue" coro would run, it would use a new add_queue and
addr would not be in requested_addrs anymore...
```
I/w | wallet.Standard_Wallet.[test_segwit_2] | starting taskgroup.
I | lnworker.LNWallet.[test_segwit_2] | starting taskgroup.
E/i | interface.[testnet.qtornado.com:51002] | Exception in run: KeyError('tb1q3wmgf8n5eettnj50pzgnfrrpdpjmwn37x7nzsc5780kk4je9v4hspym8mu')
Traceback (most recent call last):
File ".../electrum/electrum/util.py", line 1243, in wrapper
return await func(*args, **kwargs)
File ".../electrum/electrum/interface.py", line 506, in wrapper_func
return await func(self, *args, **kwargs)
File ".../electrum/electrum/interface.py", line 529, in run
await self.open_session(ssl_context)
File ".../electrum/electrum/interface.py", line 679, in open_session
async with self.taskgroup as group:
File ".../aiorpcX/aiorpcx/curio.py", line 304, in __aexit__
await self.join()
File ".../electrum/electrum/util.py", line 1339, in join
task.result()
File ".../electrum/electrum/synchronizer.py", line 80, in _run_tasks
async with taskgroup as group:
File ".../aiorpcX/aiorpcx/curio.py", line 304, in __aexit__
await self.join()
File ".../electrum/electrum/util.py", line 1339, in join
task.result()
File ".../electrum/electrum/synchronizer.py", line 127, in subscribe_to_address
self.requested_addrs.remove(addr)
KeyError: 'tb1q3wmgf8n5eettnj50pzgnfrrpdpjmwn37x7nzsc5780kk4je9v4hspym8mu'
```
AddressSynchronizer.add_address called synchronizer.add, which would only
schedule adding the addr to the Synchronizer in the next event loop iter.
If during that time, the synchronizer called adb.set_up_to_date(True),
the wallet would falsely believe and advertise itself as up_to_date
(as the wallet would see wallet.synchronize not creating new addresses,
and adb (via synchronizer) telling it is up_to_date).
Moments later, the synchronizer._add_address is finally executed and
up_to_date=False propagates out synchronizer->adb->wallet.
- revert the logic of do_breach_remedy to what it was
before 0ca3d66d15,
but now calling self.maybe_redeem unconditionally.
- replace mempool transactions only if the fee increases
- do not notify the GUI if a local tx is replaced
- delete labels when replacing
The wallet needs its own up_to_date logic:
- the adb being up_to_date means all its addresses are synced
- but an HD wallet might decide to roll the gap limit and generate new addresses
- the adb does not know about this...
- the wallet should be considered *not* up_to_date
- relatedly, it is now the wallet that decides to reset the network request counters
- note that wallet.main() was never cleaned up previously.
- now wallet gets its own taskgroup, which is cleaned up in wallet.stop.
Follow-up to adb refactor: 121d8732f1
- separate AddressSynchronizer from Wallet and LNWatcher
- the AddressSynchronizer class is referred to as 'adb' (address database)
- Use callbacks to replace overloaded methods
It does not make sense to count change outputs in our unconfirmed balance,
because our balance will not be negatively affected if the transaction does
not get confirmed.
It is also incorrect to add signed values of get_addr_balance in order to compute
the balance over a domain. For example, this leads to incoming and outgoing
transactions cancelling out in our total unconfirmed balance.
This commit looks at the coins that are spent by a transaction. If those
coins belong to us and are confirmed, we do not count the transaction outputs
in our unconfirmed balance.
As a result, get_balance always returns positive values for unconfirmed balance.
see https://github.com/spesmilo/electrum/issues/5376
crash report for electrum 4.2.1:
```
Traceback (most recent call last):
File "/home/user/wspace/electrum/.buildozer/android/platform/build-arm64-v8a/build/python-installs/Electrum/kivy/base.py", line 347, in mainloop
File "/home/user/wspace/electrum/.buildozer/android/platform/build-arm64-v8a/build/python-installs/Electrum/kivy/base.py", line 391, in idle
File "/home/user/wspace/electrum/.buildozer/android/platform/build-arm64-v8a/build/python-installs/Electrum/kivy/base.py", line 342, in dispatch_input
File "/home/user/wspace/electrum/.buildozer/android/platform/build-arm64-v8a/build/python-installs/Electrum/kivy/base.py", line 308, in post_dispatch_input
File "kivy/_event.pyx", line 724, in kivy._event.EventDispatcher.dispatch
File "/home/user/wspace/electrum/.buildozer/android/platform/build-arm64-v8a/build/python-installs/Electrum/kivy/uix/behaviors/button.py", line 179, in on_touch_up
File "kivy/_event.pyx", line 720, in kivy._event.EventDispatcher.dispatch
File "kivy/_event.pyx", line 1263, in kivy._event.EventObservers.dispatch
File "kivy/_event.pyx", line 1147, in kivy._event.EventObservers._dispatch
File "/home/user/wspace/electrum/.buildozer/android/platform/build-arm64-v8a/build/python-installs/Electrum/kivy/lang/builder.py", line 57, in custom_callback
File "<string>", line 42, in <module>
File "/home/user/wspace/electrum/.buildozer/android/app/electrum/gui/kivy/uix/dialogs/wallets.py", line 70, in cb
File "/home/user/wspace/electrum/.buildozer/android/app/electrum/gui/kivy/main_window.py", line 710, in load_wallet_by_name
File "/home/user/wspace/electrum/.buildozer/android/app/electrum/gui/kivy/main_window.py", line 728, in on_open_wallet
File "/home/user/wspace/electrum/.buildozer/android/app/electrum/gui/kivy/main_window.py", line 691, in on_wizard_success
File "/home/user/wspace/electrum/.buildozer/android/app/electrum/wallet.py", line 3267, in __new__
File "/home/user/wspace/electrum/.buildozer/android/app/electrum/wallet.py", line 3111, in __init__
File "/home/user/wspace/electrum/.buildozer/android/app/electrum/wallet.py", line 2904, in __init__
File "/home/user/wspace/electrum/.buildozer/android/app/electrum/wallet.py", line 288, in __init__
File "/home/user/wspace/electrum/.buildozer/android/app/electrum/address_synchronizer.py", line 97, in __init__
File "/home/user/wspace/electrum/.buildozer/android/app/electrum/wallet.py", line 402, in load_and_cleanup
File "/home/user/wspace/electrum/.buildozer/android/app/electrum/address_synchronizer.py", line 106, in load_and_cleanup
File "/home/user/wspace/electrum/.buildozer/android/app/electrum/util.py", line 439, in <lambda>
File "/home/user/wspace/electrum/.buildozer/android/app/electrum/util.py", line 435, in do_profile
File "/home/user/wspace/electrum/.buildozer/android/app/electrum/address_synchronizer.py", line 432, in load_local_history
File "/home/user/wspace/electrum/.buildozer/android/app/electrum/address_synchronizer.py", line 533, in _add_tx_to_local_history
File "/home/user/wspace/electrum/.buildozer/android/app/electrum/address_synchronizer.py", line 548, in _mark_address_history_changed
File "/home/user/wspace/electrum/.buildozer/android/platform/build-arm64-v8a/build/other_builds/python3/arm64-v8a__ndk_target_21/python3/Lib/asyncio/locks.py", line 260, in __init__
File "/home/user/wspace/electrum/.buildozer/android/platform/build-arm64-v8a/build/other_builds/python3/arm64-v8a__ndk_target_21/python3/Lib/asyncio/events.py", line 639, in get_event_loop
RuntimeError: There is no current event loop in thread 'GUI'.
```
SomberNight
ThomasV
Sander van Grieken