Browse Source
master5cc1695Disconnection logic fixes, add btcnet to handshake (Adam Gibson)830ac22Allow taker peers to not serve onions + bugfixes. (Adam Gibson)fd550eeOnion-based message channels with directory nodes (Adam Gibson)
Adam Gibson
4 years ago
21 changed files with 2228 additions and 162 deletions
@ -0,0 +1,209 @@ |
|||||||
|
# HOW TO SETUP ONION MESSAGE CHANNELS IN JOINMARKET |
||||||
|
|
||||||
|
### Contents |
||||||
|
|
||||||
|
1. [Overview](#overview) |
||||||
|
|
||||||
|
2. [Testing, configuring for signet](#testing) |
||||||
|
|
||||||
|
4. [Directory nodes](#directory) |
||||||
|
|
||||||
|
<a name="overview" /> |
||||||
|
|
||||||
|
## Overview |
||||||
|
|
||||||
|
This is a new way for Joinmarket bots to communicate, namely by serving and connecting to Tor onion services. This does not |
||||||
|
introduce any new requirements to your Joinmarket installation, technically, because the use of Payjoin already required the need |
||||||
|
to run such onion services, and connecting to IRC used a SOCKS5 proxy (used by almost all users) over Tor to |
||||||
|
a remote onion service. |
||||||
|
|
||||||
|
(Note however that taker bots will *not* be required to serve onions; they will only make outbound SOCKS connections, as they currently do on IRC). |
||||||
|
|
||||||
|
The purpose of this new type of message channel is as follows: |
||||||
|
|
||||||
|
* less reliance on any service external to Joinmarket |
||||||
|
* most of the transaction negotiation will be happening directly peer to peer, not passed over a central server ( |
||||||
|
albeit it was and remains E2E encrypted data, in either case) |
||||||
|
* the above can lead to better scalability at large numbers |
||||||
|
* a substantial increase in the speed of transaction negotiation; this is mostly related to the throttling of high bursts of traffic on IRC |
||||||
|
|
||||||
|
The configuration for a user is simple; in their `joinmarket.cfg` they will get a new `[MESSAGING]` section like this, if they start from scratch: |
||||||
|
|
||||||
|
``` |
||||||
|
[MESSAGING:onion] |
||||||
|
# onion based message channels must have the exact type 'onion' |
||||||
|
# (while the section name above can be MESSAGING:whatever), and there must |
||||||
|
# be only ONE such message channel configured (note the directory servers |
||||||
|
# can be multiple, below): |
||||||
|
type = onion |
||||||
|
|
||||||
|
socks5_host = localhost |
||||||
|
socks5_port = 9050 |
||||||
|
|
||||||
|
# the tor control configuration. |
||||||
|
# for most people running the tor daemon |
||||||
|
# on Linux, no changes are required here: |
||||||
|
tor_control_host = localhost |
||||||
|
# or, to use a UNIX socket |
||||||
|
# tor_control_host = unix:/var/run/tor/control |
||||||
|
tor_control_port = 9051 |
||||||
|
|
||||||
|
# the host/port actually serving the hidden service |
||||||
|
# (note the *virtual port*, that the client uses, |
||||||
|
# is hardcoded to 80): |
||||||
|
onion_serving_host = 127.0.0.1 |
||||||
|
onion_serving_port = 8080 |
||||||
|
|
||||||
|
# directory node configuration |
||||||
|
# |
||||||
|
# This is mandatory for directory nodes (who must also set their |
||||||
|
# own *.onion:port as the only directory in directory_nodes, below), |
||||||
|
# but NOT TO BE USED by non-directory nodes (which is you, unless |
||||||
|
# you know otherwise!), as it will greatly degrade your privacy. |
||||||
|
# (note the default is no value, don't replace it with ""). |
||||||
|
hidden_service_dir = |
||||||
|
# |
||||||
|
# This is a comma separated list (comma can be omitted if only one item). |
||||||
|
# Each item has format host:port ; both are required, though port will |
||||||
|
# be 80 if created in this code. |
||||||
|
# for MAINNET: |
||||||
|
directory_nodes = 3kxw6lf5vf6y26emzwgibzhrzhmhqiw6ekrek3nqfjjmhwznb2moonad.onion,qqd22cwgygaxcy6vdw6mzwkyaxg5urb4ptbc5d74nrj25phspajxjbqd.onion |
||||||
|
|
||||||
|
# for SIGNET (testing network): |
||||||
|
# directory_nodes = rr6f6qtleiiwic45bby4zwmiwjrj3jsbmcvutwpqxjziaydjydkk5iad.onion:80,k74oyetjqgcamsyhlym2vgbjtvhcrbxr4iowd4nv4zk5sehw4v665jad.onion:80 |
||||||
|
|
||||||
|
# This setting is ONLY for developer regtest setups, |
||||||
|
# running multiple bots at once. Don't alter it otherwise |
||||||
|
regtest_count = 0,0 |
||||||
|
``` |
||||||
|
|
||||||
|
All of these can be left as default for most users - but most importantly, pay attention to: |
||||||
|
|
||||||
|
* The list of `directory_nodes`, which will be comma separated if multiple directory nodes are configured (we expect there will be 2 or 3 as a normal situation). Make sure to choose the ones for your network (mainnet by default, or signet or otherwise); if it's wrong your bot will just get auto-disconnected. |
||||||
|
* The `onion_serving_port` is the port on the local machine on which the onion service is served; you won't usually need to use it, but it mustn't conflict with some other usage (so if you have something running on port 8080, change it). |
||||||
|
The `type` field must always be `onion` in this case, and distinguishes it from IRC message channels and others. |
||||||
|
|
||||||
|
### Can/should I still run IRC message channels? |
||||||
|
|
||||||
|
In short, yes, at least for now, though you are free to disable any message channel you like. |
||||||
|
|
||||||
|
### Do I need to configure Tor, and if so, how? |
||||||
|
|
||||||
|
To make outbound Tor connections to other onions in the network, you will need to configure the |
||||||
|
SOCKS5 proxy settings (so, only directory nodes may *not* need this; everyone else does). |
||||||
|
This is identical to what we already do for IRC, except that in this case, we disallow clearnet connections. |
||||||
|
|
||||||
|
#### Running/testing as a maker |
||||||
|
|
||||||
|
A maker will additionally allow *inbound* connections to an onion service. |
||||||
|
This onion service will be ephemeral, that is, it will have a different onion address every time |
||||||
|
you restart. This should work automatically, using your existing Tor daemon (here, we are using |
||||||
|
the same code as we use when running the `receive-payjoin` script, essentially). |
||||||
|
|
||||||
|
#### Running/testing as other bots (taker, ob-watcher) |
||||||
|
|
||||||
|
A taker will not attempt to serve an onion; it will only use outbound connections, first to directory |
||||||
|
nodes and then, as according to need, to individual makers, also. |
||||||
|
|
||||||
|
As previously mentioned, both of these features - inbound and outbound, to onion, Tor connections - were already in use in Joinmarket. If you want to run/test as a maker bot, but never served an onion service before, it should work fine as long as you have the Tor service running in the background, |
||||||
|
and the default control port 9051 (if not, change that value in the `joinmarket.cfg`, see above). |
||||||
|
|
||||||
|
#### Why not use Lightning based onions? |
||||||
|
|
||||||
|
(*Feel free to skip this section if you don't know what "Lightning based onions" refers to!*). The reason this architecture is |
||||||
|
proposed as an alternative to the previously suggested Lightning-node-based network (see |
||||||
|
[this PR](https://github.com/JoinMarket-Org/joinmarket-clientserver/pull/1000)), is mostly that: |
||||||
|
|
||||||
|
* the latter has a bunch of extra installation and maintenance dependencies (just one example: pyln-client requires coincurve, which we just |
||||||
|
removed) |
||||||
|
* the latter requires establishing a new node "identity" which can be refreshed, but that creates more concern |
||||||
|
* longer term ideas to integrate Lightning payments to the coinjoin workflow (and vice versa!) are not realizable yet |
||||||
|
* using multi-hop onion messaging in the LN network itself is also a way off, and a bit problematic |
||||||
|
|
||||||
|
So the short version is: the Lightning based alternative is certainly feasible, but has a lot more baggage that can't really be justified |
||||||
|
unless we're actually using it for something. |
||||||
|
|
||||||
|
|
||||||
|
<a name="testing" /> |
||||||
|
|
||||||
|
## Testing, and configuring for signet. |
||||||
|
|
||||||
|
This testing section focuses on signet since that will be the less troublesome way of getting involved in tests for |
||||||
|
the non-hardcore JM developer :) |
||||||
|
|
||||||
|
(For the latter, please use the regtest setup by running `test/e2e-coinjoin-test.py` under `pytest`, |
||||||
|
and pay attention to the settings in `regtest_joinmarket.cfg`.) |
||||||
|
|
||||||
|
There is no separate/special configuration for signet other than the configuration that is already needed for running |
||||||
|
Joinmarket against a signet backend (so e.g. RPC port of 38332). |
||||||
|
|
||||||
|
You can just uncomment the `directory_nodes` entry listed as SIGNET, and comment out the one for MAINNET. |
||||||
|
|
||||||
|
Then just make sure your bot has some signet coins and try running as maker or taker or both. |
||||||
|
|
||||||
|
<a name="directory" /> |
||||||
|
|
||||||
|
## Directory nodes |
||||||
|
|
||||||
|
**This last section is for people with a lot of technical knowledge in this area, |
||||||
|
who would like to help by running a directory node. You can ignore it if that does not apply.**. |
||||||
|
|
||||||
|
This requires a long running bot. It should be on a server you can keep running permanently, so perhaps a VPS, |
||||||
|
but in any case, very high uptime. For reliability it also makes sense to configure to run as a systemd service. |
||||||
|
|
||||||
|
The currently suggested way to run a directory node is to use the script found [here](https://github.com/JoinMarket-Org/custom-scripts/blob/0eda6154265e012b907c43e2ecdacb895aa9e3ab/start-dn.py); you can place it in your `joinmarket-clientserver/scripts` directory and run it *without* arguments, but with one option flag: `--datadir=/your/chosen/datadir` (as you'll see below). |
||||||
|
|
||||||
|
This slightly unobvious approach is based on the following ideas: we run a Joinmarket script, with a Joinmarket python virtualenv, so that we are able to parse messages; this means that the directory node *can* be a bot, e.g. a maker bot, but need not be - and here it is basically a "crippled" maker bot that cannot do anything. This 'crippling' is actually very useful because (a) we use the `no-blockchain` argument (it is forced in-code; you don't need to set it) so we don't need a running Bitcoin node (of whatever flavour), and (b) we don't need a wallet either. |
||||||
|
|
||||||
|
#### Joinmarket-specific configuration |
||||||
|
|
||||||
|
Add a non-empty `hidden_service_dir` entry to your `[MESSAGING:onion]` with a directory accessible to your user. You may want to lock this down |
||||||
|
a bit, but be careful changing permissions from what is created by the script, because Tor is very finicky about this. |
||||||
|
|
||||||
|
The hostname for your onion service will not change and will be stored permanently in that directory. |
||||||
|
|
||||||
|
The point to understand is: Joinmarket's `jmbase.JMHiddenService` will, if configured with a non-empty `hidden_service_dir` |
||||||
|
field, actually start an *independent* instance of Tor specifically for serving this, under the current user. |
||||||
|
(our Tor interface library `txtorcon` needs read access to the Tor HS dir, so it's troublesome to do this another way). |
||||||
|
|
||||||
|
##### Question: How to configure the `directory-nodes` list in our `joinmarket.cfg` for this directory node bot? |
||||||
|
|
||||||
|
Answer: **you must only enter your own node in this list!**. This way your bot will recognize that it is a directory node and it avoids weird edge case behaviour (so don't add *other* known directory nodes; you won't be talking to them). |
||||||
|
|
||||||
|
A natural retort is: but I don't know my own node's onion service hostname before I start it the first time. Indeed. So, just run it once with the default `directory_nodes` entries, then note down the new onion service hostname you created, and insert that as the only entry in the list. |
||||||
|
|
||||||
|
|
||||||
|
#### Suggested setup of a systemd service: |
||||||
|
|
||||||
|
The most basic bare-bones service seems to work fine here: |
||||||
|
|
||||||
|
``` |
||||||
|
[Unit] |
||||||
|
Description=My JM signet directory node |
||||||
|
Requires=network-online.target |
||||||
|
After=network-online.target |
||||||
|
|
||||||
|
[Service] |
||||||
|
Type=simple |
||||||
|
ExecStart=/bin/bash -c 'cd /path/to/joinmarket-clientserver && source jmvenv/bin/activate && cd scripts && python start-dn.py --datadir=/path/to/chosen/datadir' |
||||||
|
User=user |
||||||
|
|
||||||
|
[Install] |
||||||
|
WantedBy=multi-user.target |
||||||
|
``` |
||||||
|
|
||||||
|
... however, you need to kind of 'bootstrap' it the first time. For example: |
||||||
|
|
||||||
|
* run once with systemctl start |
||||||
|
|
||||||
|
* look at log with `journalctl`, service fails due to default `joinmarket.cfg` and quit. |
||||||
|
* go to that cfg file. Remove the IRC settings, they serve no purpose here. Change the `hidden_service_dir` to `/yourlocation/hidserv` (the actual directory need not exist, it's better if it doesn't, this first time). Edit the `network` field in `BLOCKCHAIN` to whatever network (mainnet, signet) you intend to support - it can be only one for one directory node, for now. |
||||||
|
|
||||||
|
* `systemctl start` again, now note the onion hostname created from the log or the directory |
||||||
|
|
||||||
|
* set that hostname in `directory_nodes` in `joinmarket.cfg` |
||||||
|
|
||||||
|
* now the service should start correctly |
||||||
|
|
||||||
|
TODO: add some material on network hardening/firewalls here, I guess. |
||||||
|
|
||||||
@ -0,0 +1,340 @@ |
|||||||
|
#! /usr/bin/env python |
||||||
|
'''Creates wallets and yield generators in regtest, |
||||||
|
then runs both them and a JMWalletDaemon instance |
||||||
|
for the taker, injecting the newly created taker |
||||||
|
wallet into it and running sendpayment once. |
||||||
|
Number of ygs is configured in the joinmarket.cfg |
||||||
|
with `regtest-count` in the `ln-onion` type MESSAGING |
||||||
|
section. |
||||||
|
See notes below for more detail on config. |
||||||
|
Run it like: |
||||||
|
pytest \ |
||||||
|
--btcroot=/path/to/bitcoin/bin/ \ |
||||||
|
--btcpwd=123456abcdef --btcconf=/blah/bitcoin.conf \ |
||||||
|
-s test/e2e-coinjoin-test.py |
||||||
|
''' |
||||||
|
from twisted.internet import reactor, defer |
||||||
|
from twisted.web.client import readBody, Headers |
||||||
|
from common import make_wallets |
||||||
|
import pytest |
||||||
|
import random |
||||||
|
import json |
||||||
|
from datetime import datetime |
||||||
|
from jmbase import (get_nontor_agent, BytesProducer, jmprint, |
||||||
|
get_log, stop_reactor) |
||||||
|
from jmclient import (YieldGeneratorBasic, load_test_config, jm_single, |
||||||
|
JMClientProtocolFactory, start_reactor, SegwitWallet, get_mchannels, |
||||||
|
SegwitLegacyWallet, JMWalletDaemon) |
||||||
|
from jmclient.wallet_utils import wallet_gettimelockaddress |
||||||
|
from jmclient.wallet_rpc import api_version_string |
||||||
|
|
||||||
|
log = get_log() |
||||||
|
|
||||||
|
# For quicker testing, restrict the range of timelock |
||||||
|
# addresses to avoid slow load of multiple bots. |
||||||
|
# Note: no need to revert this change as test runs |
||||||
|
# in isolation. |
||||||
|
from jmclient import FidelityBondMixin |
||||||
|
FidelityBondMixin.TIMELOCK_ERA_YEARS = 2 |
||||||
|
FidelityBondMixin.TIMELOCK_EPOCH_YEAR = datetime.now().year |
||||||
|
FidelityBondMixin.TIMENUMBERS_PER_PUBKEY = 12 |
||||||
|
|
||||||
|
wallet_name = "test-onion-yg-runner.jmdat" |
||||||
|
|
||||||
|
mean_amt = 2.0 |
||||||
|
|
||||||
|
directory_node_indices = [1] |
||||||
|
|
||||||
|
def get_onion_messaging_config_regtest(run_num: int, dns=[1], hsd="", mode="TAKER"): |
||||||
|
""" Sets a onion messaging channel section for a regtest instance |
||||||
|
indexed by `run_num`. The indices to be used as directory nodes |
||||||
|
should be passed as `dns`, as a list of ints. |
||||||
|
""" |
||||||
|
def location_string(directory_node_run_num): |
||||||
|
return "127.0.0.1:" + str( |
||||||
|
8080 + directory_node_run_num) |
||||||
|
if run_num in dns: |
||||||
|
# means *we* are a dn, and dns currently |
||||||
|
# do not use other dns: |
||||||
|
dns_to_use = [location_string(run_num)] |
||||||
|
else: |
||||||
|
dns_to_use = [location_string(a) for a in dns] |
||||||
|
dn_nodes_list = ",".join(dns_to_use) |
||||||
|
log.info("For node: {}, set dn list to: {}".format(run_num, dn_nodes_list)) |
||||||
|
cf = {"type": "onion", |
||||||
|
"btcnet": "testnet", |
||||||
|
"socks5_host": "127.0.0.1", |
||||||
|
"socks5_port": 9050, |
||||||
|
"tor_control_host": "127.0.0.1", |
||||||
|
"tor_control_port": 9051, |
||||||
|
"onion_serving_host": "127.0.0.1", |
||||||
|
"onion_serving_port": 8080 + run_num, |
||||||
|
"hidden_service_dir": "", |
||||||
|
"directory_nodes": dn_nodes_list, |
||||||
|
"regtest_count": "1, 1"} |
||||||
|
if mode == "MAKER": |
||||||
|
cf["serving"] = True |
||||||
|
else: |
||||||
|
cf["serving"] = False |
||||||
|
if run_num in dns: |
||||||
|
# only directories need to use fixed hidden service directories: |
||||||
|
cf["hidden_service_dir"] = hsd |
||||||
|
return cf |
||||||
|
|
||||||
|
|
||||||
|
class RegtestJMClientProtocolFactory(JMClientProtocolFactory): |
||||||
|
i = 1 |
||||||
|
def set_directory_nodes(self, dns): |
||||||
|
# a list of integers representing the directory nodes |
||||||
|
# for this test: |
||||||
|
self.dns = dns |
||||||
|
|
||||||
|
def get_mchannels(self, mode="TAKER"): |
||||||
|
# swaps out any existing onionmc configs |
||||||
|
# in the config settings on startup, for one |
||||||
|
# that's indexed to the regtest counter var: |
||||||
|
default_chans = get_mchannels(mode=mode) |
||||||
|
new_chans = [] |
||||||
|
onion_found = False |
||||||
|
hsd = "" |
||||||
|
for c in default_chans: |
||||||
|
if "type" in c and c["type"] == "onion": |
||||||
|
onion_found = True |
||||||
|
if c["hidden_service_dir"] != "": |
||||||
|
hsd = c["hidden_service_dir"] |
||||||
|
continue |
||||||
|
else: |
||||||
|
new_chans.append(c) |
||||||
|
if onion_found: |
||||||
|
new_chans.append(get_onion_messaging_config_regtest( |
||||||
|
self.i, self.dns, hsd, mode=mode)) |
||||||
|
return new_chans |
||||||
|
|
||||||
|
class JMWalletDaemonT(JMWalletDaemon): |
||||||
|
def check_cookie(self, request): |
||||||
|
if self.auth_disabled: |
||||||
|
return True |
||||||
|
return super().check_cookie(request) |
||||||
|
|
||||||
|
class TWalletRPCManager(object): |
||||||
|
""" Base class for set up of tests of the |
||||||
|
Wallet RPC calls using the wallet_rpc.JMWalletDaemon service. |
||||||
|
""" |
||||||
|
# the port for the jmwallet daemon |
||||||
|
dport = 28183 |
||||||
|
# the port for the ws |
||||||
|
wss_port = 28283 |
||||||
|
|
||||||
|
def __init__(self): |
||||||
|
# a client connnection object which is often but not always |
||||||
|
# instantiated: |
||||||
|
self.client_connector = None |
||||||
|
self.daemon = JMWalletDaemonT(self.dport, self.wss_port, tls=False) |
||||||
|
self.daemon.auth_disabled = True |
||||||
|
# because we sync and start the wallet service manually here |
||||||
|
# (and don't use wallet files yet), we won't have set a wallet name, |
||||||
|
# so we set it here: |
||||||
|
self.daemon.wallet_name = wallet_name |
||||||
|
|
||||||
|
def start(self): |
||||||
|
r, s = self.daemon.startService() |
||||||
|
self.listener_rpc = r |
||||||
|
self.listener_ws = s |
||||||
|
|
||||||
|
def get_route_root(self): |
||||||
|
addr = "http://127.0.0.1:" + str(self.dport) |
||||||
|
addr += api_version_string |
||||||
|
return addr |
||||||
|
|
||||||
|
def stop(self): |
||||||
|
for dc in reactor.getDelayedCalls(): |
||||||
|
dc.cancel() |
||||||
|
d1 = defer.maybeDeferred(self.listener_ws.stopListening) |
||||||
|
d2 = defer.maybeDeferred(self.listener_rpc.stopListening) |
||||||
|
if self.client_connector: |
||||||
|
self.client_connector.disconnect() |
||||||
|
# only fire if everything is finished: |
||||||
|
return defer.gatherResults([d1, d2]) |
||||||
|
|
||||||
|
@defer.inlineCallbacks |
||||||
|
def do_request(self, agent, method, addr, body, handler, token=None): |
||||||
|
if token: |
||||||
|
headers = Headers({"Authorization": ["Bearer " + self.jwt_token]}) |
||||||
|
else: |
||||||
|
headers = None |
||||||
|
response = yield agent.request(method, addr, headers, bodyProducer=body) |
||||||
|
yield self.response_handler(response, handler) |
||||||
|
|
||||||
|
@defer.inlineCallbacks |
||||||
|
def response_handler(self, response, handler): |
||||||
|
body = yield readBody(response) |
||||||
|
# these responses should always be 200 OK. |
||||||
|
#assert response.code == 200 |
||||||
|
# handlers check the body is as expected; no return. |
||||||
|
yield handler(body) |
||||||
|
return True |
||||||
|
|
||||||
|
def test_start_yg_and_taker_setup(setup_onion_ygrunner): |
||||||
|
"""Set up some wallets, for the ygs and 1 taker. |
||||||
|
Then start LN and the ygs in the background, then fire |
||||||
|
a startup of a wallet daemon for the taker who then |
||||||
|
makes a coinjoin payment. |
||||||
|
""" |
||||||
|
if jm_single().config.get("POLICY", "native") == "true": |
||||||
|
walletclass = SegwitWallet |
||||||
|
else: |
||||||
|
# TODO add Legacy |
||||||
|
walletclass = SegwitLegacyWallet |
||||||
|
|
||||||
|
start_bot_num, end_bot_num = [int(x) for x in jm_single().config.get( |
||||||
|
"MESSAGING:onion", "regtest_count").split(",")] |
||||||
|
num_ygs = end_bot_num - start_bot_num |
||||||
|
# specify the number of wallets and bots of each type: |
||||||
|
wallet_services = make_wallets(num_ygs + 1, |
||||||
|
wallet_structures=[[1, 3, 0, 0, 0]] * (num_ygs + 1), |
||||||
|
mean_amt=2.0, |
||||||
|
walletclass=walletclass) |
||||||
|
#the sendpayment bot uses the last wallet in the list |
||||||
|
wallet_service = wallet_services[end_bot_num - 1]['wallet'] |
||||||
|
jmprint("\n\nTaker wallet seed : " + wallet_services[end_bot_num - 1]['seed']) |
||||||
|
# for manual audit if necessary, show the maker's wallet seeds |
||||||
|
# also (note this audit should be automated in future, see |
||||||
|
# test_full_coinjoin.py in this directory) |
||||||
|
jmprint("\n\nMaker wallet seeds: ") |
||||||
|
for i in range(start_bot_num, end_bot_num): |
||||||
|
jmprint("Maker seed: " + wallet_services[i - 1]['seed']) |
||||||
|
jmprint("\n") |
||||||
|
wallet_service.sync_wallet(fast=True) |
||||||
|
ygclass = YieldGeneratorBasic |
||||||
|
|
||||||
|
# As per previous note, override non-default command line settings: |
||||||
|
options = {} |
||||||
|
for x in ["ordertype", "txfee_contribution", "txfee_contribution_factor", |
||||||
|
"cjfee_a", "cjfee_r", "cjfee_factor", "minsize", "size_factor"]: |
||||||
|
options[x] = jm_single().config.get("YIELDGENERATOR", x) |
||||||
|
ordertype = options["ordertype"] |
||||||
|
txfee_contribution = int(options["txfee_contribution"]) |
||||||
|
txfee_contribution_factor = float(options["txfee_contribution_factor"]) |
||||||
|
cjfee_factor = float(options["cjfee_factor"]) |
||||||
|
size_factor = float(options["size_factor"]) |
||||||
|
if ordertype == 'reloffer': |
||||||
|
cjfee_r = options["cjfee_r"] |
||||||
|
# minimum size is such that you always net profit at least 20% |
||||||
|
#of the miner fee |
||||||
|
minsize = max(int(1.2 * txfee_contribution / float(cjfee_r)), |
||||||
|
int(options["minsize"])) |
||||||
|
cjfee_a = None |
||||||
|
elif ordertype == 'absoffer': |
||||||
|
cjfee_a = int(options["cjfee_a"]) |
||||||
|
minsize = int(options["minsize"]) |
||||||
|
cjfee_r = None |
||||||
|
else: |
||||||
|
assert False, "incorrect offertype config for yieldgenerator." |
||||||
|
|
||||||
|
txtype = wallet_service.get_txtype() |
||||||
|
if txtype == "p2wpkh": |
||||||
|
prefix = "sw0" |
||||||
|
elif txtype == "p2sh-p2wpkh": |
||||||
|
prefix = "sw" |
||||||
|
elif txtype == "p2pkh": |
||||||
|
prefix = "" |
||||||
|
else: |
||||||
|
assert False, "Unsupported wallet type for yieldgenerator: " + txtype |
||||||
|
|
||||||
|
ordertype = prefix + ordertype |
||||||
|
|
||||||
|
for i in range(start_bot_num, end_bot_num): |
||||||
|
cfg = [txfee_contribution, cjfee_a, cjfee_r, ordertype, minsize, |
||||||
|
txfee_contribution_factor, cjfee_factor, size_factor] |
||||||
|
wallet_service_yg = wallet_services[i - 1]["wallet"] |
||||||
|
|
||||||
|
wallet_service_yg.startService() |
||||||
|
|
||||||
|
yg = ygclass(wallet_service_yg, cfg) |
||||||
|
clientfactory = RegtestJMClientProtocolFactory(yg, proto_type="MAKER") |
||||||
|
# This ensures that the right rpc/port config is passed into the daemon, |
||||||
|
# for this specific bot: |
||||||
|
clientfactory.i = i |
||||||
|
# This ensures that this bot knows which other bots are directory nodes: |
||||||
|
clientfactory.set_directory_nodes(directory_node_indices) |
||||||
|
nodaemon = jm_single().config.getint("DAEMON", "no_daemon") |
||||||
|
daemon = bool(nodaemon) |
||||||
|
#rs = True if i == num_ygs - 1 else False |
||||||
|
start_reactor(jm_single().config.get("DAEMON", "daemon_host"), |
||||||
|
jm_single().config.getint("DAEMON", "daemon_port"), |
||||||
|
clientfactory, daemon=daemon, rs=False) |
||||||
|
reactor.callLater(1.0, start_test_taker, wallet_services[end_bot_num - 1]['wallet'], end_bot_num, num_ygs) |
||||||
|
reactor.run() |
||||||
|
|
||||||
|
@defer.inlineCallbacks |
||||||
|
def start_test_taker(wallet_service, i, num_ygs): |
||||||
|
# this rpc manager has auth disabled, |
||||||
|
# and the wallet_service is set manually, |
||||||
|
# so no unlock etc. |
||||||
|
mgr = TWalletRPCManager() |
||||||
|
mgr.daemon.services["wallet"] = wallet_service |
||||||
|
# because we are manually setting the wallet_service |
||||||
|
# of the JMWalletDaemon instance, we do not follow the |
||||||
|
# usual flow of `initialize_wallet_service`, we do not set |
||||||
|
# the auth token or start the websocket; so we must manually |
||||||
|
# sync the wallet, including bypassing any restart callback: |
||||||
|
def dummy_restart_callback(msg): |
||||||
|
log.warn("Ignoring rescan request from backend wallet service: " + msg) |
||||||
|
mgr.daemon.services["wallet"].add_restart_callback(dummy_restart_callback) |
||||||
|
mgr.daemon.wallet_name = wallet_name |
||||||
|
mgr.daemon.services["wallet"].startService() |
||||||
|
def get_client_factory(): |
||||||
|
clientfactory = RegtestJMClientProtocolFactory(mgr.daemon.taker, |
||||||
|
proto_type="TAKER") |
||||||
|
clientfactory.i = i |
||||||
|
clientfactory.set_directory_nodes(directory_node_indices) |
||||||
|
return clientfactory |
||||||
|
|
||||||
|
mgr.daemon.get_client_factory = get_client_factory |
||||||
|
# before preparing the RPC call to the wallet daemon, |
||||||
|
# we decide a coinjoin destination, counterparty count and amount. |
||||||
|
# Choosing a destination in the wallet is a bit easier because |
||||||
|
# we can query the mixdepth balance at the end. |
||||||
|
coinjoin_destination = mgr.daemon.services["wallet"].get_internal_addr(4) |
||||||
|
cj_amount = 22000000 |
||||||
|
def n_cps_from_n_ygs(n): |
||||||
|
if n > 4: |
||||||
|
return n - 2 |
||||||
|
if n > 2: |
||||||
|
return 2 |
||||||
|
assert False, "Need at least 3 yield generators to test" |
||||||
|
n_cps = n_cps_from_n_ygs(num_ygs) |
||||||
|
# once the taker is finished we sanity check before |
||||||
|
# shutting down: |
||||||
|
def dummy_taker_finished(res, fromtx=False, |
||||||
|
waittime=0.0, txdetails=None): |
||||||
|
jmprint("Taker is finished") |
||||||
|
# check that the funds have arrived. |
||||||
|
mbal = mgr.daemon.services["wallet"].get_balance_by_mixdepth()[4] |
||||||
|
assert mbal == cj_amount |
||||||
|
jmprint("Funds: {} sats successfully arrived into mixdepth 4.".format(cj_amount)) |
||||||
|
stop_reactor() |
||||||
|
mgr.daemon.taker_finished = dummy_taker_finished |
||||||
|
mgr.start() |
||||||
|
agent = get_nontor_agent() |
||||||
|
addr = mgr.get_route_root() |
||||||
|
addr += "/wallet/" |
||||||
|
addr += mgr.daemon.wallet_name |
||||||
|
addr += "/taker/coinjoin" |
||||||
|
addr = addr.encode() |
||||||
|
body = BytesProducer(json.dumps({"mixdepth": "1", |
||||||
|
"amount_sats": cj_amount, |
||||||
|
"counterparties": str(n_cps), |
||||||
|
"destination": coinjoin_destination}).encode()) |
||||||
|
yield mgr.do_request(agent, b"POST", addr, body, |
||||||
|
process_coinjoin_response) |
||||||
|
|
||||||
|
def process_coinjoin_response(response): |
||||||
|
json_body = json.loads(response.decode("utf-8")) |
||||||
|
print("coinjoin response: {}".format(json_body)) |
||||||
|
|
||||||
|
@pytest.fixture(scope="module") |
||||||
|
def setup_onion_ygrunner(): |
||||||
|
load_test_config() |
||||||
|
jm_single().bc_interface.tick_forward_chain_interval = 10 |
||||||
|
jm_single().bc_interface.simulate_blocks() |
||||||
Loading…
Reference in new issue