The JWT validation update in PR 1480 contained a small error: by calling
the function get_POST_body() twice, the content of the request was
unavailable in the second call. This calls only once, but has the
downside of requiring a specific set of keys in the json request data.
Hence this might be fixed to be more flexible later, see comments in the
PR.
This commit creates a new endpoint /getinfo which currently returns only
the version of Joinmarket (JM_CORE_VERSION) running in the backend
joinmarketd. In future commits the returned dict may add more
information, for example the current block height of the connected
Bitcoin Core node.
Additionally, this commit fixes the response type of the
/rescanblockchain endpoint as defined in the OpenAPI spec.
Prior to this commit, an attempt to change the gap limit used in wallet
syncing, by setting gaplimit in the config via configset, would fail,
since gaplimit was only being read from command line options. After this
commit, the value of gaplimit in the POLICY section of the config,
defaulting to 6, will be read and used in Wallet object creation, either
via the createwallet or recoverwallet endpoints, or via the wallet
opening operation in the unlockwallet endpoint. This can be used in
combination with the rescanblockchain endpoint to allow discovery of
funds in addresses beyond the default gap.
Fixes#1082:
This commit allows recovery of a wallet from a seedphrase
with a new endpoint wallet/recover. 4 parameters are passed in,
the same three as for wallet/create but also a bip39 seedphrase
as the fourth argument.
This commit also adds a rescanblockchain RPC call and status:
This adds a new endpoint /rescanblockchain which is (Core) wallet specific
(due to underlying JM architecture). This action is spawned in a thread,
since the bitcoind RPC call is blocking and can take a very long time.
To monitor the status of the rescan, an extra field `rescanning` is
added to the /session endpoint.
Also adds test of rpc wallet recovery
Prior to this commit, the 'testflag' was set for the Taker object that
was created in running the tumbler via the RPC API. This flag prevents
the function client_protocol.JMTakerClientProtocol.stallMonitor from
running (which is useful in certain cases), which was not intended.
After this commit, this flag is not set, so that when a transaction
fails to go through, the stallMonitor will wake up after some time and
retry the transaction, as should occur.
Fixes#1314.
Prior to this commit, an attempt to start a maker bot which only
contained coins that were expired timelocked utxos (i.e. spendable),
in the RPC would result in the maker service starting but failing to
create offers. Instead, we want the maker service to not start as there
are no coins actually available for it.
Fixes#1215.
Prior to this commit it was possible to send a non-coinjoin transaction
while the maker service was running, over the RPC-API call direct-send.
However this was not intentional, as it is not usually safe to perform
UTXO database write actions when the effective 'lock' on write actions
is already held by a long running service/action such as yield generator
or tumbler. This commit prevents that call being successful if the
current state of the JMWalletDaemon is not CJ_NOT_RUNNING.
This commit also disables freeze actions in the same situation, and
changes the HTTP error code for these cases to 400.
Fixes#1214.
Prior to this commit, if a broadcast of a transaction failed locally, or
failed after attempting remote broadcast then falling back to local via
the function jmclient.taker.Taker.handle_unbroadcast_transaction, then
the version of the on_finished_callback found in
jmclient.wallet_rpc.JMWalletDaemon incorrectly asserted that the fromtx
keyword argument must be True, thus failing to update the state of the
daemon to CJ_NOT_RUNNING.
After this commit that assertion is removed, and the coinjoin_in_process
value returned over the API is correctly False after either a
successful, or an unsuccessful, finishing of the coinjoin.
Fixes#1213.
Prior to this commit, if the YieldGeneratorService were started via a
call to /maker/start over the RPC-API, the early check of whether coins
were present in the wallet to allow starting the maker, incorrectly read
the contents of the dict returned by
jmclient.wallet.UTXOManager.get_balance_by_mixdepth, which returns
entries for mixdepths with coins which are all frozen, with a total
value of zero, even if the include_disabled keyword argument is set to
False. Thus only checking for whether the length of the returned dict
was zero was incorrect.
After this commit, we will stop early, as intended, for a wallet which
has no mixdepths with any non-frozen coins.
In the previous commit, all peers served an onion.
After this commit, taker client instances will automatically
send a config var to the jmdaemon backend that instructs
the OnionMessageChannel instance to not start an onion service,
and the handshake messages sent by these peers replace the
onion location with a placeholder string NOT-SERVING-ONION.
Directories and maker peers will not therefore to connect outbound
to them, but privmsging still happens p2p with connections from
takers to makers after the directory has communicated their
reachable .onion addresses.
This change reduces the configuration requirement for takers and
is better for their privacy and security (without sacrificing
the gain we get from having p2p connections).
The above comments re: takers also apply to ob-watcher bots.
This commit also fixes a large number of minor bugs and errors in
documentation, as well as many Python cleanups after review from
@PulpCattel. A few concrete items are:
It fixes the ob-watcher functionality to work with the new subclass
of MessageChannel (OnionMessageChannel).
It corrects the on_nick_leave trigger to make dynamic nick switching
between MessageChannels (as implemented in MessageChannelCollection)
work correctly.
It corrects the order of events in the add_peer workflow to ensure that
a handshake can always be sent so that the activation of the connection
always works.
It sets a default messaging config with onion, 2 active IRC servers and
one inactive IRC server. The onion config has 2 signet directory nodes,
so this will change to mainnet after the PR is merged to master.
Joinmarket bots run their own onion services allowing inbound connections.
Both takers and makers connect to other makers at the mentioned
onion services, over Tor.
Directory nodes run persistent onion services allowing peers to
find other (maker) peers to connect to, and also forwarding
messages where necessary.
This is implemented as an alternative to IRC, i.e. a new
implementation of the abstract class MessageChannel, in onionmc.py.
Note that using both this *and* IRC servers is supported; Joinmarket
supports multiple, redundant different communication methods,
simultaneously.
Messaging is done with a derived class of twisted's LineReceiver,
and there is an additional layer of syntax, similar to but not the
same as the IRC syntax for ensuring that messages are passed with
the same J5.. nick as is used on IRC. This allows us to keep the
message signing logic the same as before. As well as Joinmarket line
messages, we use additional control messages to communicate peer lists,
and to manage connections.
Peers which send messages not conforming to the syntax are dropped.
See https://github.com/JoinMarket-Org/JoinMarket-Docs/pull/12 for
documentation of the syntax.
Connections to directory nodes are robust as for IRC servers, in
that we use a ReconnectingClientFactory to keep trying to re-establish
broken connections with exponential backoff. Connections to maker
peers do not require this feature, as they will often disconnect
in normal operation.
Multiple directory nodes can and should be configured by bots.
This commit fixes an issue regarding a wrong taker state caused by
initiating a taker operation when no `max_cj_fee` value is present.
Before this change the taker state switches to `running` indefinitely.
After this change the config check is made before updating the state.
Fixes#1170.
Prior to this commit, an attempt to spend using
the direct-send endpoint in the RPC API would
cause a backend crash because the NoFundsException
was not being caught.
This is now fixed.
Fixes#1166.
Fixes#1173.
Prior to this commit, in tests, we were calling
JMWalletDaemon.stopService at exit of the python script,
outside the pytest context, leading to various errors because
the test framework had gone through tear down.
After this commit, no attempt to call stopService is made
at exit (this is not necessary, we already call wallet close
and there is no other persistence necessary).
Fixes#1168.
Before this commit, if an RPC client sent /maker/start
on an active wallet without confirmed coins, it would
raise NotEnoughCoins but this would happen *after* the
call to the setup function which resets the coinjoin
state to CJ_MAKER_RUNNING, even though the maker service
did not successfully start.
After this commit, the raise of NotEnoughCoins happens
first, preventing the update of coinjoin state.
Prior to this commit, a lock of one wallet followed by
an unlock of another, or an unlock of a new wallet, overriding
the old one, if it occurred during a running taker-side
coinjoin, would not reset the coinjoin_state to CJ_NOT_RUNNING,
resulting in an inability of the newly loaded wallet to function
correctly.
After this commit, all subservices, including the quasi-service
of taker-side coinjoin, are fully shut down whenever a wallet is
locked, or a new wallet is unlocked. This may be suboptimal (see
TODO) but is logical for now.
The main effect is to ensure that a new wallet will always start
in the correct coinjoin state (CJ_NOT_RUNNING).
Also worth noting, a running Taker will have its abort parameter
set to True on the wallet lock event, meaning that it will not
proceed to the next step on the next asynchronous message coming
from counterparties.
Fix shutdown of already open wallet on new unlock
Prior to this commit, if a JSON-RPC API client called
/unlock on a new wallet while an existing one was currently
loaded, the preexisting wallet would not have its wallet
service shut down, and thus the .lock file would not be removed,
resulting in an inability to open that wallet a second time.
After this commit, we correctly refer to the wallet service
internally such that its service does get shut down, and the
close event of the wallet occurs, removing the .lock file as
required.
Fix re-unlock of currently unlocked wallet
This behaviour was originally intended by #1136. If a wallet
is currently in an unlocked state in the daemon, but the user
loses the auth token, and tries to re-unlock, then before this
commit, the unlock operation would fail because there was an
unsuccessful attempt to open the wallet file.
After this commit, if the unlock request is applied to the wallet
with the same name as the one currently active, then the password
is authenticated by opening the wallet in read-only mode, preventing
the above issue from occurring.
Fix bug that lockwallet doesn't delete wallet_name
Before this commit, the member variable wallet_name in
the JMWalletDaemon object was not being reset to None
in a call to the lockwallet endpoint which meant that the
unlock event erroneously treated the wallet as already being
active. This corrects that error by setting walet_name to
None in lockwallet.
Fixes#1122. An RPC client can request the list
of coinjoins as shown in the file datadir/logs/yigen-statement.csv.
Authentication is not required, nor a specific wallet, as this is
a global list of all coinjoins done so far. A 404 is returned if
the file has not yet been created (no maker operations).
Fixes#1093. This adds a POST method freeze for a wallet,
in which the utxo must be specified as a standard hex txid:n
string in the body, along with a boolean value of 'freeze', to
toggle the frozen/unfrozen state of the given utxo in the wallet.
Fixes#1123. Prior to this commit, the unlockwallet
method erroneously waited for wallet sync before continuing
and returning the credential. This was not necessary as the
WalletService method startService calls wallet syncing
asynchronously(!) in request_wallet_sync. Hence this initial
syncing step is removed, allowing the RPC call to return
immediately.
Fixes#1120. As per the discussion in that issue, while
we can only return 'processing started' by default, and
the client must monitor progress/state to see if the maker
service has connected successfully, nevertheless if the
maker startup is immediately invalidated by the fact that
the already-synced wallet does not contain confirmed coins,
we can and should return an error message instead of a 202.
This commit does that, adding to the spec a 409-No-Coins
response type which indicates to the API client that the
setup of the maker failed for this specific reason.
Prior to this commit, if a user has attempted
a coinjoin with /taker/coinjoin but a problem
occurs (such as an unresponsive maker), they
are not able to abort the protocol but must wait
until the taker_finished event is triggered.
After this commit a GET request to the endpoint
defined as /wallet/walletname/taker/stop will
force-stop the coinjoin attempt (similar to
quitting the script in CLI). This will reset the
wallet daemon state to CJ_NOT_RUNNING.
Fixes#1121.
Prior to this commit, if a user
lost access to the authentication token for
a session with the wallet RPC, they would not
be able to lock the existing wallet before
authenticating again (getting a new token for
the same wallet, or a different one).
After this commit, a call to the /unlock endpoint
will succeed even if an existing wallet is currently
unlocked (whether a different one or the same one).
The existing wallet service, if present, is shut down,
if the attempt to authenticate a new wallet, with a
password is successful (otherwise nothing happens).
A test is added to make sure that this re-unlock can work.
Some of the return codes for the endpoints in
the RPC API were incorrect, these are fixed.
They are also checked in the tests.
Additionally, an extra test of the maker start/
stop function is added (though it tests only
state updates, not actual service start).
Fixes#1091.
Before this commit, if the unconfirmed transaction
event triggered the taker_finished callback in the
RPC-started coinjoin, the value of fromtx is
'unconfirmed' and not False, hence an assertion error
was raised unless the confirmed event came first
(which is only likely to happen on regtest). This
crashed the jmwalletd daemon backend.
After this commit, the assertion check is only that
the value is not True, which it must not be as long
as we only accept single-join schedules.
1. Moves the JMWalletDaemon service class into
the jmclient package (see the wallet_rpc.py module).
2. Adds dependencies "klein" and "autobahn" to the
jmclient package, as well as "pyjwt".
3. Adds another module websocketserver.py, using
autobahn, to allow the JMWalletDaemon service to
serve subscriptions over a websocket, for e.g.
transaction notifications.
4. Adds tests both for the websocket connection
and for the JSON-RPC HTTP connection.
JmwalletdWebSocketServerFactory.sendTxNotification
sends the json-ified transaction details using
jmbitcoin.human_readable_transaction (as is currently
used in our CLI), along with the txid.
Also adds a coinjoin state update event sent via
the websocket (switch from taker/maker/none).
Require authentication to connect to websocket.
5. Add OpenApi definition of API in yaml;
also auto-create human-readable API docs in markdown.
6. Add fidelity bond function to API
7. Add config read/write route to API
8. Remove snicker rpc calls temporarily
9. Updates to docoinjoin: corrects taker_finished
for this custom case, does not shut down at end.
10. Address detailed review comments of @PulpCattel.
Kristaps Kaupe
theborakompanioni
roshii
Adam Gibson
Daniel
Adam Gibson
Thebora Kompanioni
Manas Gandy