We must not process incoming updates for a given channel until we ~finished reestablishing it.
Consider both parties have some unacked updates they want to replay during reestablish.
If Bob reacts to Alice's replayed stuff before he himself replays his stuff, madness ensues.
I think this should fix the remaining part of https://github.com/spesmilo/electrum/pull/8778
(timing issues when running the unit tests with py3.12)
Assume Alice and Bob have a channel, and Alice is on an old state,
but neither of them knows the latter yet.
Timing scenarios:
1. Alice sends reest first, and Bob receives it before sending reest himself
- old code: Bob realises Alice is behind, Bob will force-close,
Bob won't send reest to Alice, so Alice does not learn she is behind
- new code: Bob realises Alice is behind, Bob will force-close,
Bob will still send reest to Alice, so Alice learns she is behind.
2. Bob sends reest first, and Alice receives it before sending reest herself
- old code: Alice learns she is behind. Alice won't send reest to Bob,
so Bob does not learn he is ahead, so Bob won't force-close.
- new code: Alice learns she is behind. Alice will still send reest to Bob
though with ctn=0 instead of actual. Bob learns he is ahead, so
Bob will force close.
3. Alice and Bob both send reest, and then they both receive what the other sent
- no change: Alice and Bob both learn Alice is behind. Bob will force-close.
- all forwarding types use the same flow
- forwarding callback returns a htlc_key or None
- forwarding info is persisted in lnworker:
- ongoing_forwardings
- downstream to upstream htlc_key
- htlc_key -> error_bytes
- introduce PaymentFeeBudget, which contains limits for fee budget and cltv budget
- when splitting a payment,
- the fee budget is linearly distributed between the parts
- this resolves a FIXME in lnrouter ("FIXME in case of MPP")
- the cltv budget is simply copied
- we could also add other kinds of budgets later, e.g. for the num in-flight htlcs
- resolves TODO in lnworker ("todo: compare to the fee of the actual route we found")
- Refactor _run_mpp so that it takes only one set of arguments.
Success and failure conditions are now tested by calling
_run_mpp multiple times.
- In test_payment_multipart with_timeout, check that the received
failure message actually is MPP_TIMEOUT, and not a generic
failure. Since the onion is obfuscated by the forwarding node,
this tests that obfuscate_onion_packet and decode_onion_packet
work as expected.
- see comment in lnaddr.py
- Previously we used feature bit 50/51 for trampoline.
The spec subsequently defined fbit 50/51 as option_zeroconf, which
requires fbit 46/47 (option_scid_alias) to also be set.
We moved the non-standard trampoline fbit to a different int.
However, old wallets might have old invoices saved that set fbit 50/51
for trampoline, and those would not have the dependent bit set.
Invoices are parsed at wallet-open, so if the parser ran these checks,
those wallets could not be opened.
- note: we could potentially also run lnaddr.validate_and_compare_features
when saving new invoices into the wallet but this is not done here
- do not use needs_test_with_all_chacha20_implementation,
this is slow and not really useful here.
- split TestPeer class in two classes, depending on the type
of graph we use.
- rename trampoline_forwardings -> final_onion_forwardings,
because this dict is used for both trampoline and hold invoices
- remove timeout from hold_invoice_callbacks (redundant with invoice)
- add test_failure boolean parameter to TestPeer._test_simple_payment,
in order to test correct propagation of OnionRoutingFailures.
- maybe_fulfill_htlc: raise an OnionRoutingFailure if we do not have
the preimage for a payment that does not have a hold invoice callback.
Without this, the above unit tests stall when we use test_failure=True
- introduce SentHtlcInfo named tuple
- some previously unnamed tuples are now much shorter:
create_routes_for_payment no longer returns an 8-tuple!
- sent_htlcs_q (renamed from sent_htlcs), is now keyed on payment_hash+payment_secret
(needed for proper trampoline forwarding)
- add RecvMPPResolution enum for possible states of a pending incoming MPP,
and use it in check_mpp_status
- new state: "FAILED", to allow nicely failing back the whole MPP set
- key more things with payment_hash+payment_secret, for consistency
(just payment_hash is insufficient for trampoline forwarding)
triggered a payment forwarding.
Final onions may trigger a payment forwarding, through the callback
returned by maybe_fulfill_htlc. In that case, we should not fail the
HTLC later; doing so might result in fund loss.
Remove test_simple_payment_with_hold_invoice_timing_out: once we
have accepted to forward a payment HTLC with a hold invoice, we
do not want to time it out, for the same reason.
- maybe_fulfill_htlc returns a forwarding callback that
covers both cases.
- previously, the callback of hold invoices was called as a
side-effect of lnworker.check_mpp_status.
- the same data structures (lnworker.trampoline_forwardings,
lnworker.trampoline_forwarding_errors) are used for both
trampoline forwardings and hold invoices.
- maybe_fulfill_htlc still recursively calls itself to perform
checks on trampoline onion. This is ugly, but ugliness is now
contained to that method.
- fix parameters passed to maybe_forward_trampoline
- use lnworker.trampoline_forwardings as a semaphore for ongoing
trampoline payments
- if a trampoline payment fails, fail all received HTLCs
(invoices for which we do not have the preimage)
Callbacks and timeouts are registered with lnworker. If the
preimage is not known after the timeout has expired, the payment
is failed with MPP_TIMEOUT.
A new config API is introduced, and ~all of the codebase is adapted to it.
The old API is kept but mainly only for dynamic usage where its extra flexibility is needed.
Using examples, the old config API looked this:
```
>>> config.get("request_expiry", 86400)
604800
>>> config.set_key("request_expiry", 86400)
>>>
```
The new config API instead:
```
>>> config.WALLET_PAYREQ_EXPIRY_SECONDS
604800
>>> config.WALLET_PAYREQ_EXPIRY_SECONDS = 86400
>>>
```
The old API operated on arbitrary string keys, the new one uses
a static ~enum-like list of variables.
With the new API:
- there is a single centralised list of config variables, as opposed to
these being scattered all over
- no more duplication of default values (in the getters)
- there is now some (minimal for now) type-validation/conversion for
the config values
closes https://github.com/spesmilo/electrum/pull/5640
closes https://github.com/spesmilo/electrum/pull/5649
Note: there is yet a third API added here, for certain niche/abstract use-cases,
where we need a reference to the config variable itself.
It should only be used when needed:
```
>>> var = config.cv.WALLET_PAYREQ_EXPIRY_SECONDS
>>> var
<ConfigVarWithConfig key='request_expiry'>
>>> var.get()
604800
>>> var.set(3600)
>>> var.get_default_value()
86400
>>> var.is_set()
True
>>> var.is_modifiable()
True
```
Sander van Grieken
SomberNight
MrNaif2018
ThomasV