- Wait until HTLCs are irrevocably removed before cleaning up their
data structures (MPP and forwarding)
- keep methods maybe_cleanup_mpp and maybe_cleanup_forwarding separate
- perform cleanup in htlc_switch, so that process_unfulfilled_htlc
has less side effects
- In htlc_switch, we blank the onion_packet_hex field to signal that
an HTLC has been processed. An item of chan.unfulfilled_htlcs may
go through 4 stages:
- 1. not forwarded yet: (None, onion_packet_hex)
- 2. forwarded: (forwarding_key, onion_packet_hex)
- 3. processed: (forwarding_key, None), not irrevocably removed yet
- 4. done: (forwarding_key, None), irrevocably removed
- in test_lnpeer, an extra iteration of htlc_switch has been added to
trampoline forwarding tests
follow-up 6fb9207a44
> technically the fee estimate of a given bitcoind only changes on new blocks, but because of how we are asking for fee estimates and how we are taking the median of many interfaces, it can change at any time for us
note: running this in the peer taskgroup, for proper exception-handling and logging
If we accept a MPP and we forward the payment (trampoline or swap),
we need to persist the payment accepted status, or we might wrongly
release htlcs on the next restart.
lnworker.received_mpp_htlcs used to be cleaned up in maybe_cleanup_forwarding,
which only applies to forwarded payments. However, since we now
persist this dict, we need to clean it up also in the case of
payments received by us. This part of maybe_cleanup_forwarding has
been migrated to lnworker.maybe_cleanup_mpp
e.g. if you consolidate own utxos, the "amount sent" in the dialog is zero,
but for high fee warnings that's not the amount we should be comparing against
This will be useful if we decide to ship lntransport as a separate
package. It is also a conceptual cleanup.
Notes:
- lntransport still requires crypto.py
- parsing node id from a bolt11 invoice is not supported.
If you closed a main_window soon after opening it, and the UpdateCheckThread network request was slow
and still in progress, the gui would freeze until the network request finished.
```
=============================== warnings summary ===============================
tests/test_wizard.py::ServerConnectWizardTestCase::test_no_advanced
tests/test_wizard.py::ServerConnectWizardTestCase::test_proxy
tests/test_wizard.py::ServerConnectWizardTestCase::test_proxy_and_server
tests/test_wizard.py::ServerConnectWizardTestCase::test_server
tests/test_wizard.py::WalletWizardTestCase::test_2fa
tests/test_wizard.py::WalletWizardTestCase::test_create_standard_wallet_haveseed_bip39
tests/test_wizard.py::WalletWizardTestCase::test_create_standard_wallet_haveseed_electrum
tests/test_wizard.py::WalletWizardTestCase::test_create_standard_wallet_newseed
/tmp/cirrus-ci-build/.tox/py3/lib/python3.10/site-packages/_pytest/threadexception.py:82: PytestUnhandledThreadExceptionWarning: Exception in thread Plugins
Traceback (most recent call last):
File "/usr/local/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
self.run()
File "/tmp/cirrus-ci-build/electrum/plugin.py", line 360, in run
self.on_stop()
File "/tmp/cirrus-ci-build/electrum/util.py", line 430, in on_stop
loop = get_asyncio_loop()
File "/tmp/cirrus-ci-build/electrum/util.py", line 1578, in get_asyncio_loop
raise Exception("event loop not created yet")
Exception: event loop not created yet
warnings.warn(pytest.PytestUnhandledThreadExceptionWarning(msg))
```
The "ssl.VERIFY_X509_STRICT" flag for openssl verification has been enabled by default in python 3.13+ (it was disabled before that).
see https://github.com/python/cpython/issues/107361
and https://discuss.python.org/t/ssl-changing-the-default-sslcontext-verify-flags/30230/16
We explicitly disable it for self-signed certs, thereby restoring the pre-3.13 defaults,
as it seems to break lots of servers.
For example, using python 3.13 (or setting `sslc.verify_flags |= ssl.VERIFY_X509_STRICT`),
- I can connect to `btc.electroncash.dk:60002:s`
- but not to `electrum.emzy.de:50002:s`
despite both using self-signed certs.
We should investigate more why exactly "strict" verification fails for some self-signed certs and not for others,
and make sure that at least newly generated certs adhere to the stricter requirements (e.g. update guide in e-x?).
RunCoroutineDialog has a run() method that blocks the thread
without blocking the GUI (using exec), and a Cancel button
that cancels the coroutine.
main_window.run_coroutine_dialog() is a wrapper that returns
the coroutine result and may raise exceptions.
BlockingWaitingDialog was removed is transaction_dialog,
where it was not particularly useful.
I just had a channel force-closed over a fee-estimate-disagreement :(
Scenario:
1. started electrum, opened wallet
2. waited around 10 seconds, opened the lightning channels overview, saw that channels are open/ready
3. after around 10 more seconds, scanned bolt11 invoice and tried to pay
4. channel got force-closed
Before this commit, we only call maybe_update_fee via lnwatcher callbacks.
These callbacks trigger on events such as "adb_set_up_to_date", "blockchain_updated", "network_updated", "fee".
In my case there was a race that all these events triggered *before* the channel got reestablished
(in fact before the peer handshake finished). And also, by chance there were none of these events after
the reestablish but before I sent the HTLC.
When I sent the HTLC, the channel counterparty (eclair) sent back an "error" msg that the feerates are
too different, which led us to do a local-force-close.
I have other channels in this wallet (with other peers), which reestablished faster and got lucky with
timing: the lnwatcher callbacks came just in time to trigger update_fee for them.
```
20241017T222847.598163Z | INFO | lnpeer.Peer.[LNWallet, 03ecef675b-98960573] | handshake done for 03ecef675be448b615e6176424070673ef8284e0fd19d8be062a6cb5b130a0a0d1@lightning.electrum.org:9740
20241017T222847.602594Z | DEBUG | lnpeer.Peer.[LNWallet, 03ecef675b-98960573] | Sending INIT
20241017T222847.641383Z | DEBUG | lnpeer.Peer.[LNWallet, 03ecef675b-98960573] | Received INIT
20241017T222847.655041Z | DEBUG | lnpeer.Peer.[LNWallet, 03ecef675b-98960573] | Sending CHANNEL_REESTABLISH
20241017T222847.658355Z | INFO | lnpeer.Peer.[LNWallet, 03ecef675b-98960573] | channel_reestablish (<redacted_shortchanid>): sent channel_reestablish with (next_local_ctn=157, oldest_unrevoked_remote_ctn=156)
20241017T222847.659524Z | INFO | lnpeer.Peer.[LNWallet, 03ecef675b-98960573] | reestablish_channel was called but channel <redacted_shortchanid> already in peer_state <PeerState.REESTABLISHING: 1>
20241017T222847.660491Z | INFO | lnpeer.Peer.[LNWallet, 03ecef675b-98960573] | reestablish_channel was called but channel <redacted_shortchanid> already in peer_state <PeerState.REESTABLISHING: 1>
20241017T222847.661442Z | INFO | lnpeer.Peer.[LNWallet, 03ecef675b-98960573] | reestablish_channel was called but channel <redacted_shortchanid> already in peer_state <PeerState.REESTABLISHING: 1>
20241017T222847.662768Z | INFO | lnpeer.Peer.[LNWallet, 03ecef675b-98960573] | reestablish_channel was called but channel <redacted_shortchanid> already in peer_state <PeerState.REESTABLISHING: 1>
20241017T222847.669875Z | DEBUG | lnpeer.Peer.[LNWallet, 03ecef675b-98960573] | Received QUERY_CHANNEL_RANGE
20241017T222847.690318Z | DEBUG | lnpeer.Peer.[LNWallet, 03ecef675b-98960573] | Received GOSSIP_TIMESTAMP_FILTER
20241017T222847.705782Z | DEBUG | lnpeer.Peer.[LNWallet, 03ecef675b-98960573] | Received CHANNEL_REESTABLISH
20241017T222847.707932Z | INFO | lnpeer.Peer.[LNWallet, 03ecef675b-98960573] | channel_reestablish (<redacted_shortchanid>): received channel_reestablish with (their_next_local_ctn=157, their_oldest_unrevoked_remote_ctn=156)
20241017T222847.712504Z | INFO | lnpeer.Peer.[LNWallet, 03ecef675b-98960573] | channel_reestablish (<redacted_shortchanid>): replayed 0 unacked messages. []
20241017T222847.716096Z | DEBUG | lnpeer.Peer.[LNWallet, 03ecef675b-98960573] | Sending CHANNEL_READY
20241017T222847.738709Z | INFO | lnpeer.Peer.[LNWallet, 03ecef675b-98960573] | saved remote channel_update gossip msg for chan <redacted_shortchanid>
20241017T222907.627447Z | INFO | lnpeer.Peer.[LNWallet, 03ecef675b-98960573] | lnpeer.pay len(route)=1
20241017T222907.628927Z | INFO | lnpeer.Peer.[LNWallet, 03ecef675b-98960573] | 0: edge=<redacted_shortchanid> hop_data=<OnionHopsDataSingle. payload={<redacted>}. hmac=None>
20241017T222907.629184Z | INFO | lnpeer.Peer.[LNWallet, 03ecef675b-98960573] | adding trampoline onion to final payload
20241017T222907.629405Z | INFO | lnpeer.Peer.[LNWallet, 03ecef675b-98960573] | lnpeer.pay len(t_route)=2
20241017T222907.629653Z | INFO | lnpeer.Peer.[LNWallet, 03ecef675b-98960573] | 0: t_node=03ecef675be448b615e6176424070673ef8284e0fd19d8be062a6cb5b130a0a0d1 hop_data=<OnionHopsDataSingle. payload={<redacted>}. hmac=<redacted>>
20241017T222907.629894Z | INFO | lnpeer.Peer.[LNWallet, 03ecef675b-98960573] | 1: t_node=<redacted> hop_data=<OnionHopsDataSingle. payload={<redacted>}. hmac=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'>
20241017T222907.631495Z | INFO | lnpeer.Peer.[LNWallet, 03ecef675b-98960573] | starting payment. len(route)=1.
20241017T222907.633075Z | INFO | lnpeer.Peer.[LNWallet, 03ecef675b-98960573] | starting payment. htlc: UpdateAddHtlc(amount_msat=<redacted>, payment_hash=<redacted>, cltv_abs=<redacted>, timestamp=1729204147, htlc_id=66)
20241017T222907.633385Z | DEBUG | lnpeer.Peer.[LNWallet, 03ecef675b-98960573] | Sending UPDATE_ADD_HTLC
20241017T222907.635118Z | INFO | lnpeer.Peer.[LNWallet, 03ecef675b-98960573] | send_commitment. chan <redacted_shortchanid>. ctn: 157.
20241017T222907.643229Z | DEBUG | lnpeer.Peer.[LNWallet, 03ecef675b-98960573] | Sending COMMITMENT_SIGNED
20241017T222907.721929Z | DEBUG | lnpeer.Peer.[LNWallet, 03ecef675b-98960573] | Received ERROR
20241017T222907.722621Z | INFO | lnpeer.Peer.[LNWallet, 03ecef675b-98960573] | remote peer sent error [DO NOT TRUST THIS MESSAGE]: 'local/remote feerates are too different: remoteFeeratePerKw=<redacted_low_val> localFeeratePerKw=50125'. chan_id=<redacted>. is_known_chan_id=True
20241017T222907.734272Z | DEBUG | lnchannel.Channel.[<redacted_shortchanid>] | Setting channel state: OPEN -> FORCE_CLOSING
20241017T222907.825540Z | INFO | lnpeer.Peer.[LNWallet, 03ecef675b-98960573] | Disconnecting: GracefulDisconnect()
```
accumulator
ThomasV
SomberNight