Fixes#1215.
Prior to this commit it was possible to send a non-coinjoin transaction
while the maker service was running, over the RPC-API call direct-send.
However this was not intentional, as it is not usually safe to perform
UTXO database write actions when the effective 'lock' on write actions
is already held by a long running service/action such as yield generator
or tumbler. This commit prevents that call being successful if the
current state of the JMWalletDaemon is not CJ_NOT_RUNNING.
This commit also disables freeze actions in the same situation, and
changes the HTTP error code for these cases to 400.
Add a new helper module to calculate fidelity bonds values and stats, and a script to use it.
The goal is to give as much information as possible to the user before
committing to a fidelity bond.
Fixes#967. In this test, if the randomized fee calculated from the
tx_fees field in the config happens to be at the bottom of the range,
it's possible that the calculated feerate is very close to 1.1 sats/vB,
but this can still vary slightly due to signature sizes, resulting in a
very rare case where this test could erroneously fail.
This is fixed by bumping the lowest minfeerate by 2%.
Fixes#1294.
Before this commit, calls to query_utxo_set with default arguments
would ignore the mempool and thus return utxos which were spent in
unconfirmed transactions. Thus, takers would continue negotiation of
coinjoins with makers who sent them already-spent utxos, leading to
failure at broadcast time. This was not intended behaviour; we want
takers to reject utxos that are double spent in the mempool.
This commit changes that default argument to True so that utxo set
changes in the mempool are accounted for. It also switches the name of
the includeunconf argument, which was misleading, to include_mempool,
with appropriately updated docstring.
Finally, in this commit we also ensure that callers of this function
check, where necessary, the returned confirmations field to disallow
unconfirmed utxos where that is necessary.
PR #986 introduced two distinct balances at different levels of
the wallet tree; it serialized these balances for CLI and Qt display
but changed the RPC-API json output of the /display endpoint in a
suboptimal way. This commit fixes the json structure.
After this commit, the four levels (wallet, account, branch, entry) now
all still have balances reported under the same key as before #986, that
is, total_balance, account_balance, balance and amount, but the extra
information introduced by that PR, namely the 'available_balance' is
added as an extra key in the json dict.
Tests are added to check this structure.
Before this commit, if Joinmarket-Qt or any other long running script
was started when a new utxo had been created in the wallet, but was not
yet in a block, the transition from unconfirmed to confirmed state would
not be registered, because when confirmation happened and the event was
seen in the transaction monitor, that txid was not in the active_txs
dict. This is now fixed.
For frozen balances, display a label "FROZEN" in the status notes.
For unconfirmed balances, display a label "PENDING" in the status notes.
Both FROZEN and LOCKED balances are considered unavailable balances, while PENDING balance is considered available balance.
Because it is possible to broadcast a transaction that spends unconfirmed balance, while a user has to manually unfreeze
the balance or wait until the timelock has expired before they can use FROZEN or LOCKED balances.
Closes#1247.
This consists of an inclusion of the bond value exponent into the config
that the user can alter, and a change of that from 2 to default 1.3.
Also updates the fidelity bond documentation to account for those
changes, including the units used in ob-watcher, but not the calculation
of fidelity bond attack resistance (which remains a TODO).
Previously the `cert_msg` in the fidelity bond protocol involves a
binary public key. This is additional unnecessary complexity given
that we want other wallets to also implement the fidelity bond
protocol (see issue #993 for full discussion). So its good to move
to a certificate message which is entirely ascii.
This commit has ascii certficiate messages also be accepted as valid
along with the old certificate messages.
Fixes#1214.
Prior to this commit, if a broadcast of a transaction failed locally, or
failed after attempting remote broadcast then falling back to local via
the function jmclient.taker.Taker.handle_unbroadcast_transaction, then
the version of the on_finished_callback found in
jmclient.wallet_rpc.JMWalletDaemon incorrectly asserted that the fromtx
keyword argument must be True, thus failing to update the state of the
daemon to CJ_NOT_RUNNING.
After this commit that assertion is removed, and the coinjoin_in_process
value returned over the API is correctly False after either a
successful, or an unsuccessful, finishing of the coinjoin.
Fixes#1213.
Prior to this commit, if the YieldGeneratorService were started via a
call to /maker/start over the RPC-API, the early check of whether coins
were present in the wallet to allow starting the maker, incorrectly read
the contents of the dict returned by
jmclient.wallet.UTXOManager.get_balance_by_mixdepth, which returns
entries for mixdepths with coins which are all frozen, with a total
value of zero, even if the include_disabled keyword argument is set to
False. Thus only checking for whether the length of the returned dict
was zero was incorrect.
After this commit, we will stop early, as intended, for a wallet which
has no mixdepths with any non-frozen coins.
Fixes#1212.
Before this commit, transactions were kept monitored in
WalletService.transaction_monitor after first being seen, only if the
list of confirmed callbacks was non empty, but this meant that the logic
is process_new_tx was not executed, meaning the height field of the utxo
was never updated from INF_HEIGHT, so the utxo would continue to be seen
as having zero confirmations, even after blocks are mined.
After this commit, we add any transaction that was seen to the
active_txs dict, so that the height field, and thus the number of
confirmations reported, is correct even if the caller never registered a
confirmed callback.
`:80` needs to be provided otherwise parsing will fail with
Failed to load directory nodes: InvalidLocationStringError('3kxw6lf5vf6y26emzwgibzhrzhmhqiw6ekrek3nqfjjmhwznb2moonad.onion')
This is correct in the auto-generated configuration file, but not in the
document.
Also mention that even for UNIX sockets, it's necessary to specify `tor_control_port`.
Prior to this commit, running ob-watcher with
an onion-type message channel configured would result
in the bot attempting to connect to all the makers,
which is bad. This happened because the internal logic
of the onion message channel is that receivers of privmsgs
will be sent peer info from the directory, so that they can
immediately respond p2p if they succeed in outward connecting.
But for bots who do not intend to engage in a coinjoin interactive
protocol, like ob-watchers, this is absolutely not the desired outcome.
After this commit, a bot can specify mode "PASSIVE" in the call to
get_mchannels(), which results in the OnionMessageChannel object only
creating non-directory remote peer objects of type OnionPeerPassive,
instead of OnionPeer, which means they never try to connect to those
remote peers.
Also, exports JMMakerClientProtocol for custom directory node scripts
(stored in the custom-scripts repo).
Modify default config with 2 signet and mainnet directory nodes to
start.
Handles unreachable directory nodes with a human readable error and
adjusts connection timeouts to be realistic.
Changes wording in Qt notifications from "IRC" to message channel.
Updates docs, new directory node information.
In the previous commit, all peers served an onion.
After this commit, taker client instances will automatically
send a config var to the jmdaemon backend that instructs
the OnionMessageChannel instance to not start an onion service,
and the handshake messages sent by these peers replace the
onion location with a placeholder string NOT-SERVING-ONION.
Directories and maker peers will not therefore to connect outbound
to them, but privmsging still happens p2p with connections from
takers to makers after the directory has communicated their
reachable .onion addresses.
This change reduces the configuration requirement for takers and
is better for their privacy and security (without sacrificing
the gain we get from having p2p connections).
The above comments re: takers also apply to ob-watcher bots.
This commit also fixes a large number of minor bugs and errors in
documentation, as well as many Python cleanups after review from
@PulpCattel. A few concrete items are:
It fixes the ob-watcher functionality to work with the new subclass
of MessageChannel (OnionMessageChannel).
It corrects the on_nick_leave trigger to make dynamic nick switching
between MessageChannels (as implemented in MessageChannelCollection)
work correctly.
It corrects the order of events in the add_peer workflow to ensure that
a handshake can always be sent so that the activation of the connection
always works.
It sets a default messaging config with onion, 2 active IRC servers and
one inactive IRC server. The onion config has 2 signet directory nodes,
so this will change to mainnet after the PR is merged to master.
Joinmarket bots run their own onion services allowing inbound connections.
Both takers and makers connect to other makers at the mentioned
onion services, over Tor.
Directory nodes run persistent onion services allowing peers to
find other (maker) peers to connect to, and also forwarding
messages where necessary.
This is implemented as an alternative to IRC, i.e. a new
implementation of the abstract class MessageChannel, in onionmc.py.
Note that using both this *and* IRC servers is supported; Joinmarket
supports multiple, redundant different communication methods,
simultaneously.
Messaging is done with a derived class of twisted's LineReceiver,
and there is an additional layer of syntax, similar to but not the
same as the IRC syntax for ensuring that messages are passed with
the same J5.. nick as is used on IRC. This allows us to keep the
message signing logic the same as before. As well as Joinmarket line
messages, we use additional control messages to communicate peer lists,
and to manage connections.
Peers which send messages not conforming to the syntax are dropped.
See https://github.com/JoinMarket-Org/JoinMarket-Docs/pull/12 for
documentation of the syntax.
Connections to directory nodes are robust as for IRC servers, in
that we use a ReconnectingClientFactory to keep trying to re-establish
broken connections with exponential backoff. Connections to maker
peers do not require this feature, as they will often disconnect
in normal operation.
Multiple directory nodes can and should be configured by bots.
* Migrate callbacks registered under provisional (tx-output tuples) keys
as soon as their txids are known. Leave a txid breadcrumb in place so
register_callbacks and check_callback_called can find it and the
migrated list of callbacks.
* Invoke callbacks via list comprehensions, retaining only the callbacks
that return False. The old code was buggy, as it was removing elements
from the callback lists while iterating over them, which would cause
callbacks to be skipped.
* The existing code would fail to call any "confirmed" callbacks for a
remove-only transaction if no "unconfirmed" callbacks had been
registered for that transaction, and it would discontinue monitoring
of a transaction after just one "confirmed" callback had returned
True, even if other "confirmed" callbacks returned False to remain
registered. This commit overhauls the logic to fix all of these bugs.
* Delete emptied callback lists from the dicts to be tidy.
The transaction_monitor function in WalletService was polling the most
recent 100 transactions from the BlockchainInterface every 5 seconds.
This was egregious and inefficient. Introduce a _yield_new_transactions
function that remembers the newest previously yielded transaction
between invocations and yields only transactions newer than that one.
Use the new function in transaction_monitor.
Avoid repeatedly deserializing confirmed transactions that remain in the
set of monitored transactions after being confirmed either due to
"confirmed" callbacks that return False or because callbacks have timed
out and have been unregistered without cleaning the set of monitored
transactions. Change the active_txids list to an active_txs dict whose
keys are the txids and whose values are the deserialized transactions.
Use this new dict as a cache to avoid repeatedly deserializing confirmed
transactions.
Alter the BitcoinCoreInterface._yield_transactions generator to be
nicer in cases where only a small number of transactions need to be
examined. Geometrically ramp the requested number of transactions in
each successive request, starting from 1. Since new transactions can
appear in the Bitcoin Core wallet between RPC calls, overlap successive
request ranges and resume yielding transactions after finding the
previously yielded transaction to avoid yielding duplicates.
The listtransactions RPC of Bitcoin Core can return the same (txid,vout)
as both a "category":"receive" and a "category":"send". Therefore, use
(txid,vout,category) as the sync key to ensure that all distinct
elements are yielded.
Delete the unused wallet_name parameter from _yield_transactions, per
suggestion by Adam Gibson.
Prior to this commit, the file commitmentlist, which stores
commitments (podle) for makers to let them blacklist/prevent
reuse was stored in the local directory for the script, which
allowed remote running of jmdaemon but was very unhelpful for
any situation where multiple bots are running at once, e.g.
in testing or using multiple wallets against the same codebase.
This could result in incorrect rejection of coinjoins.
After this commit, by default, the commitmentlist file is stored
in datadir/cmtdata/commitmentlist, so it will be local to any
custom data directory as would be the case for running multiple
wallets on the same machine. A user can set the POLICY variable
commitment_list_location to "." to revert to the previous behaviour.
If a user has many UTXOs but wants to spend just one, they would have
to freeze all the UTXOs one-by-one except for one, then after sending
do the same again unfreezing all UTXOs one-by-one. This new command
drastically speeds up this process.
This commit fixes an issue regarding a wrong taker state caused by
initiating a taker operation when no `max_cj_fee` value is present.
Before this change the taker state switches to `running` indefinitely.
After this change the config check is made before updating the state.
Use sats/kilo-vbyte
Use words for large numbers
Improve unit consistency
Improve newline consistency
Improve comment consistency
Improve capitalization consistency
Improve section consistency
Fixes#1170.
Prior to this commit, an attempt to spend using
the direct-send endpoint in the RPC API would
cause a backend crash because the NoFundsException
was not being caught.
This is now fixed.
Adds code to validate bech32m encoded taproot addresses
and scriptPubKeys. The underlying functions are in
python-bitcointx upcoming version 1.1.3.
Adds tests at both the basic address validation layer,
and also at the layer of sending transactions to
taproot outputs.
theborakompanioni
Kristaps Kaupe
Adam Gibson
PulpCattel
dependabot[bot]
Wukong
Daniel
chris-belcher
laanwj
Adam Gibson
Matt Whitlock
unofficial bisq contributor
Thebora Kompanioni
5F43CCDF