This commit creates a new endpoint /getinfo which currently returns only
the version of Joinmarket (JM_CORE_VERSION) running in the backend
joinmarketd. In future commits the returned dict may add more
information, for example the current block height of the connected
Bitcoin Core node.
Additionally, this commit fixes the response type of the
/rescanblockchain endpoint as defined in the OpenAPI spec.
Fixes#1082:
This commit allows recovery of a wallet from a seedphrase
with a new endpoint wallet/recover. 4 parameters are passed in,
the same three as for wallet/create but also a bip39 seedphrase
as the fourth argument.
This commit also adds a rescanblockchain RPC call and status:
This adds a new endpoint /rescanblockchain which is (Core) wallet specific
(due to underlying JM architecture). This action is spawned in a thread,
since the bitcoind RPC call is blocking and can take a very long time.
To monitor the status of the rescan, an extra field `rescanning` is
added to the /session endpoint.
Also adds test of rpc wallet recovery
This allows running tests that do not require, e.g., bitcoind, without having to run these setup fixtures at all.
Signed-off-by: PulpCattel <PulpCattel@users.noreply.github.com>
This may or may not fix#1057 and other test failures. Since the network
delay in receiving the message is unknown (but expected to be very
small, usually), we keep trying to receive notification of the
transaction message in the websocket test 10 times for a total of 2
seconds (on the assumption that if it takes longer than that, something
else is wrong).
Prior to this commit, p2wsh inputs from fidelity bonds resulted in
miscalculation of transaction fees, even in cases where the exact set of
inputs were known (such as a direct send).
In this commit we change the estimation to a model in which the caller
of jmbitcoin.secp256k1_transaction.estimate_tx_size must specify a list
of types, one for each input to the transaction, and the same for
outputs. In some cases, the caller of the function uses the default
script type of the wallet, but in other cases where the caller can know
the exact types of each utxo used as input, and each destination used as
output, they are specified explicitly. In particular, the use of
fidelity bond outputs as input to transactions can be accounted for.
Currently this is only done in direct send payments; coinjoins still
fall back to assuming all inputs the same type (note that it is not
possible to use fidelity bond utxos as inputs to coinjoins).
Note also that the burn destination calculation in direct send is
removed, since it is not used, so the maintenance burden is best
avoided.
Prior to this commit, the tumbler algorithm assumed that destination
mixdepths of INTERNAL transactions were incremented by 1, but the
underlying taker code uses (mod maxmixdepth) logic always. This commit
takes the decision to make the usage of the wallet "purely" cyclic, that
is, not only the Taker object but also the tumbler algorithm now always
treat the wallet as a cycle. This is not problematic in a tumbler
algorith (or any other schedule generation algorithm), as long as we use
the strict rule of "always exit each mixdepth with a sweep", which the
tumbler always did and this commit does not change.
Also, and importantly, several much more detailed tests of the tumbler
schedule generation have been added.
Add a new helper module to calculate fidelity bonds values and stats, and a script to use it.
The goal is to give as much information as possible to the user before
committing to a fidelity bond.
Fixes#967. In this test, if the randomized fee calculated from the
tx_fees field in the config happens to be at the bottom of the range,
it's possible that the calculated feerate is very close to 1.1 sats/vB,
but this can still vary slightly due to signature sizes, resulting in a
very rare case where this test could erroneously fail.
This is fixed by bumping the lowest minfeerate by 2%.
Fixes#1294.
Before this commit, calls to query_utxo_set with default arguments
would ignore the mempool and thus return utxos which were spent in
unconfirmed transactions. Thus, takers would continue negotiation of
coinjoins with makers who sent them already-spent utxos, leading to
failure at broadcast time. This was not intended behaviour; we want
takers to reject utxos that are double spent in the mempool.
This commit changes that default argument to True so that utxo set
changes in the mempool are accounted for. It also switches the name of
the includeunconf argument, which was misleading, to include_mempool,
with appropriately updated docstring.
Finally, in this commit we also ensure that callers of this function
check, where necessary, the returned confirmations field to disallow
unconfirmed utxos where that is necessary.
PR #986 introduced two distinct balances at different levels of
the wallet tree; it serialized these balances for CLI and Qt display
but changed the RPC-API json output of the /display endpoint in a
suboptimal way. This commit fixes the json structure.
After this commit, the four levels (wallet, account, branch, entry) now
all still have balances reported under the same key as before #986, that
is, total_balance, account_balance, balance and amount, but the extra
information introduced by that PR, namely the 'available_balance' is
added as an extra key in the json dict.
Tests are added to check this structure.
In the previous commit, all peers served an onion.
After this commit, taker client instances will automatically
send a config var to the jmdaemon backend that instructs
the OnionMessageChannel instance to not start an onion service,
and the handshake messages sent by these peers replace the
onion location with a placeholder string NOT-SERVING-ONION.
Directories and maker peers will not therefore to connect outbound
to them, but privmsging still happens p2p with connections from
takers to makers after the directory has communicated their
reachable .onion addresses.
This change reduces the configuration requirement for takers and
is better for their privacy and security (without sacrificing
the gain we get from having p2p connections).
The above comments re: takers also apply to ob-watcher bots.
This commit also fixes a large number of minor bugs and errors in
documentation, as well as many Python cleanups after review from
@PulpCattel. A few concrete items are:
It fixes the ob-watcher functionality to work with the new subclass
of MessageChannel (OnionMessageChannel).
It corrects the on_nick_leave trigger to make dynamic nick switching
between MessageChannels (as implemented in MessageChannelCollection)
work correctly.
It corrects the order of events in the add_peer workflow to ensure that
a handshake can always be sent so that the activation of the connection
always works.
It sets a default messaging config with onion, 2 active IRC servers and
one inactive IRC server. The onion config has 2 signet directory nodes,
so this will change to mainnet after the PR is merged to master.
Prior to this commit, the file commitmentlist, which stores
commitments (podle) for makers to let them blacklist/prevent
reuse was stored in the local directory for the script, which
allowed remote running of jmdaemon but was very unhelpful for
any situation where multiple bots are running at once, e.g.
in testing or using multiple wallets against the same codebase.
This could result in incorrect rejection of coinjoins.
After this commit, by default, the commitmentlist file is stored
in datadir/cmtdata/commitmentlist, so it will be local to any
custom data directory as would be the case for running multiple
wallets on the same machine. A user can set the POLICY variable
commitment_list_location to "." to revert to the previous behaviour.
Adds code to validate bech32m encoded taproot addresses
and scriptPubKeys. The underlying functions are in
python-bitcointx upcoming version 1.1.3.
Adds tests at both the basic address validation layer,
and also at the layer of sending transactions to
taproot outputs.
Fixes#1166.
Fixes#1173.
Prior to this commit, in tests, we were calling
JMWalletDaemon.stopService at exit of the python script,
outside the pytest context, leading to various errors because
the test framework had gone through tear down.
After this commit, no attempt to call stopService is made
at exit (this is not necessary, we already call wallet close
and there is no other persistence necessary).
Fix shutdown of already open wallet on new unlock
Prior to this commit, if a JSON-RPC API client called
/unlock on a new wallet while an existing one was currently
loaded, the preexisting wallet would not have its wallet
service shut down, and thus the .lock file would not be removed,
resulting in an inability to open that wallet a second time.
After this commit, we correctly refer to the wallet service
internally such that its service does get shut down, and the
close event of the wallet occurs, removing the .lock file as
required.
Fix re-unlock of currently unlocked wallet
This behaviour was originally intended by #1136. If a wallet
is currently in an unlocked state in the daemon, but the user
loses the auth token, and tries to re-unlock, then before this
commit, the unlock operation would fail because there was an
unsuccessful attempt to open the wallet file.
After this commit, if the unlock request is applied to the wallet
with the same name as the one currently active, then the password
is authenticated by opening the wallet in read-only mode, preventing
the above issue from occurring.
Fix bug that lockwallet doesn't delete wallet_name
Before this commit, the member variable wallet_name in
the JMWalletDaemon object was not being reset to None
in a call to the lockwallet endpoint which meant that the
unlock event erroneously treated the wallet as already being
active. This corrects that error by setting walet_name to
None in lockwallet.
Fixes#1093. This adds a POST method freeze for a wallet,
in which the utxo must be specified as a standard hex txid:n
string in the body, along with a boolean value of 'freeze', to
toggle the frozen/unfrozen state of the given utxo in the wallet.
Fixes#1118.
Before this commit, the json serializtion of a
WalletEntry object was incorrect and missing some
fields. This is now fixed, and the WalletDisplayResponse
in the RPC spec .yaml file correctly reflects the
fields that are returned by the JMWalletDaemon in response
to the /display request.
Fixes#1121.
Prior to this commit, if a user
lost access to the authentication token for
a session with the wallet RPC, they would not
be able to lock the existing wallet before
authenticating again (getting a new token for
the same wallet, or a different one).
After this commit, a call to the /unlock endpoint
will succeed even if an existing wallet is currently
unlocked (whether a different one or the same one).
The existing wallet service, if present, is shut down,
if the attempt to authenticate a new wallet, with a
password is successful (otherwise nothing happens).
A test is added to make sure that this re-unlock can work.
Some of the return codes for the endpoints in
the RPC API were incorrect, these are fixed.
They are also checked in the tests.
Additionally, an extra test of the maker start/
stop function is added (though it tests only
state updates, not actual service start).
Prior to this commit, the sendtomany.py script
would sometimes fail based on the output of the
RPC call gettxout returning only a script, and not
an address. After this commit, the validation will
still work correctly in these cases using the
chosen cryptoengine to convert from script to address.
Also notably, the return value of utxo set does
not include an 'address' key any more.
Tests are updated to reflect this.
1. Moves the JMWalletDaemon service class into
the jmclient package (see the wallet_rpc.py module).
2. Adds dependencies "klein" and "autobahn" to the
jmclient package, as well as "pyjwt".
3. Adds another module websocketserver.py, using
autobahn, to allow the JMWalletDaemon service to
serve subscriptions over a websocket, for e.g.
transaction notifications.
4. Adds tests both for the websocket connection
and for the JSON-RPC HTTP connection.
JmwalletdWebSocketServerFactory.sendTxNotification
sends the json-ified transaction details using
jmbitcoin.human_readable_transaction (as is currently
used in our CLI), along with the txid.
Also adds a coinjoin state update event sent via
the websocket (switch from taker/maker/none).
Require authentication to connect to websocket.
5. Add OpenApi definition of API in yaml;
also auto-create human-readable API docs in markdown.
6. Add fidelity bond function to API
7. Add config read/write route to API
8. Remove snicker rpc calls temporarily
9. Updates to docoinjoin: corrects taker_finished
for this custom case, does not shut down at end.
10. Address detailed review comments of @PulpCattel.
bugfix: use enumerate instead of len
reviewed estimation of transaction sizes
estimates are still a bit conservative with room for improvement;
signatures could still save up to one byte each if using low-r
values. python-bitcointx==1.1.2-dev already supports grinding for
low-r values so when it's stable and referenced version is updated,
this should be reviewed again so as to utilize that benefit.
added utility method `estimate_extra_bytes`
the purpose of this method is for the computation
of extra bytes when the coinjoin or direct send output
type is different from that of the wallet
updated tests to reflect new transaction size
computation
p2pkh transactions are now 1 byte larger for
the inputs hence the change amount should be
less 4 * 30 sats.
add private keys for utxos that we may not be
tracking
some transactions (e.g. opt-in rbf) may require signing
with private keys for utxos that we may have stopped
tracking. this commit will search through all inputs
and for those we own and retrieve their private keys
so we can sign with them.
added support for p2wsh output scripts in
refactored the estimation of the transaction size
when outputs of a different type is the target
Previously, a different method was employed which
was kind of kludgy considering the fact that the
`extra_bytes` parameter is really for `OP_RETURN`
outputs. This method modifies the `estimate_tx_size`
method to accept an optional extra parameter called
`outtype` which is used to estimate the correct
transaction size if the target output is different
from that of the wallet.
added missing import
added a note about preserving the order of wallet type constants
Fix bug with timelocked addrs in receive payjoin
Previously there would be a crash if the wallet receiving a payjoin
had a timelocked UTXO.
Fixes#976. Joinmarket will allow, but warn, when
non-zero change values (which are not included in
the transaction) result from sweeps (due to the
impossibility of knowing the exact value in advance).
However, prior to this commit, if that estimation
inaccuracy resulted in a negative change value that
was to be ignored, there was a crash due to the fact
that `jmbitcoin.amount_to_str` does not allow negative
values. Hence we instead print the number in the warning
without going through this formatting function.
Adds tests case for negative sweep change.
Before this commit, attempting to spend in a taker-
side Joinmarket coinjoin, a utxo which is of the
tiemelock type (after its timelock expired), had
a potential to crash, because it would be selected
for the generation of a PoDLE but this cannot be
supported (even with taker-side code changes, the
maker could not reconstruct it).
After this commit, any non-standard wallet-type
scriptPubKey is filtered out of being a candidate
for PoDLE generation.
Also a trivial variable name typo is fixed (too-old).
Also the test framework `DummyWallet` now inherits
from `LegacyWallet` because its utxos are of that
type; a future commit should update to `SegwitWallet`.
Also add tests cases for no custom-script PoDLEs.
Adam Gibson
roshii
Kristaps Kaupe
PulpCattel
theborakompanioni
Manas Gandy
undeath
Tim Akinbo