Allow user to constrain coinjoin sweep fee.

Fixes #784. In `Taker.receive_utxos` we check, in the
sweep case, whether the fee violates the user config
setting `max_sweep_fee_change`; if so, the tx is aborted,
and we shut down for single-shot coinjoins, but wait for
stallMonitor to restart for multi-schedules. The value is
defaulted to 80% to give plenty of breathing room to avoid
using up too many commitments.

docs/USAGE.md

@@ -455,6 +455,8 @@ There are two different types of fee; bitcoin network transaction fees and fees
 This is controlled using the setting of `tx_fees` in the `[POLICY]` section in your `joinmarket.cfg` file in the current directory. If you set it to a number between 1 and 1000 it is treated as the targeted number of blocks for confirmation; e.g. if you set it to 20 you are asking to use whatever Bitcoin Core thinks is a realistic fee to get confirmation within the next 20 blocks. By default it is 3. If you set it to a number > 1000, don't set it lower than about 1200, it will be interpreted as "number of satoshis per kilobyte for the transaction fee". 1000 equates to 1 satoshi per byte (ignoring technical details of vbyte), which is usually the minimum fee that nodes on the network will relay. Note that Joinmarket will deliberately vary your choice randomly, in this case, by 20% either side, to avoid you watermarking all your transactions with the exact same fee rate. As an example, if you prefer to use an approximate rate of 20 sats/byte rather than rely on Bitcoin Core's estimated target for 3 or 6 blocks, then set `tx_fees` to 20000.
 Note that some liquidity providers (Makers) will offer very small contributions to the tx fee, but mostly you should consider that you must pay the whole Bitcoin network fee yourself, as an instigator of a coinjoin (a Taker). Note also that if you set 7 counterparties, you are effectively paying for approximately 7 normal sized transactions; be cognizant of that!
 
+An additional note for coinjoin sweeps: we must *estimate* the fee in this case (due to not having a change output). See the comments to the `joinmarket.cfg` setting `max_sweep_fee_change` for how you can control the variance of the fee, for this specific case.
+
 #### CoinJoin fees.
 
 Individual Makers will offer to do CoinJoin at different rates. Some set their fee as a percentage of the amount, and others set it as a fixed number of satoshis. Most set their rates very low, so in most (but not all) cases the overall CoinJoin fee will be lower than the bitcoin network fees discussed above. When starting to do a CoinJoin you will be prompted to set the *maximum* relative (percentage) and absolute (number of satoshis) fees that you're willing to accept from one participant; your bot will then choose randomly from those that are below at least one of those limits. Please read these instructions carefully and update your config file accordingly to avoid having to answer the questions repeatedly.

jmclient/jmclient/client_protocol.py

@@ -393,7 +393,7 @@ class JMTakerClientProtocol(JMClientProtocol):
             return
         if not self.client.txid:
             #txid is set on pushing; if it's not there, we have failed.
-            jlog.info("Stall detected. Regenerating transactions and retrying.")
+            jlog.info("Stall detected. Retrying transaction if possible ...")
             self.client.on_finished_callback(False, True, 0.0)
         else:
             #This shouldn't really happen; if the tx confirmed,
@@ -442,6 +442,10 @@ class JMTakerClientProtocol(JMClientProtocol):
             if not retval[0]:
                 jlog.info("Taker is not continuing, phase 2 abandoned.")
                 jlog.info("Reason: " + str(retval[1]))
+                if len(self.client.schedule) == 1:
+                    # see comment for the same invocation in on_JM_OFFERS;
+                    # the logic here is the same.
+                    self.client.on_finished_callback(False, False, 0.0)
                 return {'accepted': False}
             else:
                 nick_list, txhex = retval[1:]

jmclient/jmclient/configure.py

@@ -217,6 +217,22 @@ tx_fees = 3
 # 1,000,000 satoshis.
 absurd_fee_per_kb = 350000
 
+# In decimal, the maximum allowable change either lower or
+# higher, that the fee rate used for coinjoin sweeps is
+# allowed to be.
+# (note: coinjoin sweeps *must estimate* fee rates;
+# they cannot be exact due to the lack of change output.)
+#
+# Example: max_sweep_fee_change = 0.4, with tx_fees = 10000,
+# means actual fee rate achieved in the sweep can be as low
+# as 6000 sats/kilo-vbyte up to 14000 sats/kilo-vbyte.
+#
+# If this is not achieved, the transaction is aborted. For tumbler,
+# it will then be retried until successful.
+# WARNING: too-strict setting may result in using up a lot
+# of PoDLE commitments, hence the default 0.8 (80%).
+max_sweep_fee_change = 0.8
+
 # Maximum absolute coinjoin fee in satoshi to pay to a single
 # market maker for a transaction. Both the limits given in
 # max_cj_fee_abs and max_cj_fee_rel must be exceeded in order

jmclient/jmclient/taker.py

@@ -504,8 +504,28 @@ class Taker(object):
                 # seems you wont always get exactly zero because of integer
                 # rounding so 1 satoshi extra or fewer being spent as miner
                 # fees is acceptable
-                jlog.info(('WARNING CHANGE NOT BEING '
-                           'USED\nCHANGEVALUE = {}').format(btc.amount_to_str(my_change_value)))
+                jlog.info(
+                ('WARNING CHANGE NOT BEING USED\nCHANGEVALUE = {}').format(
+                    btc.amount_to_str(my_change_value)))
+            # we need to check whether the *achieved* txfee-rate is outside
+            # the range allowed by the user in config; if not, abort the tx.
+            # this is done with using the same estimate fee function and comparing
+            # the totals; this ratio will correspond to the ratio of the feerates.
+            num_ins = len([u for u in sum(self.utxos.values(), [])])
+            num_outs = len(self.outputs) + 2
+            new_total_fee = estimate_tx_fee(num_ins, num_outs,
+                                    txtype=self.wallet_service.get_txtype())
+            feeratio = self.total_txfee/new_total_fee
+            jlog.debug("Ratio of actual to estimated sweep fee: {}".format(
+                feeratio))
+            sweep_delta = float(jm_single().config.get("POLICY",
+                                                       "max_sweep_fee_change"))
+            if feeratio < 1 - sweep_delta or feeratio > 1 + sweep_delta:
+                jlog.warn("Transaction fee for sweep: {} too far from expected:"
+                          " {}; check the setting 'max_sweep_fee_change'"
+                          " in joinmarket.cfg. Aborting this attempt.".format(
+                              self.total_txfee, new_total_fee))
+                return (False, "Unacceptable feerate for sweep, giving up.")
         else:
             self.outputs.append({'address': self.my_change_addr,
                                  'value': my_change_value})