Replace links to instructions in original Wiki with updated
and corrected instructions within USAGE.md.
Add YIELDGENERATOR.md to docs/ with updated and corrected
instructions from original wiki.
(You have either followed the Quick Install on the readme (`./install.sh`), or have
(You have either followed the Quick Install on the readme (`./install.sh`) (RECOMMENDED), or have
followed a manual installation as per [here](INSTALL.md)).
(If you want to run Joinmarket-Qt, start with the [walkthrough](JOINMARKET-QT-GUIDE.md) instead of this.)
### Zeroth step: configuring for Bitcoin Core.
# Contents
1. [Configuring for Bitcoin Core](#configure)
2. [Using the wallet-tool.py script](#wallet-tool)
a. [Creating a Wallet](#generate)
b. [Funding Wallet and Displaying Balance](#funding)
c. [Spending from the wallet directly (without Coinjoin)](#spending)
d. [Recovering a Wallet from mnemonic](#recover)
e. [Recovering Mnemonic from a Wallet](#mnemonic)
f. [Recovering Private Keys](#privkeys)
g. [Wallet History](#history)
h. [Importing External Private Keys](#importprivkeys)
i. [What is the BIP32 wallet structure](#structure)
j. [What is the Gap Limit](#gaplimit)
3. [Try out a coinjoin; using sendpayment.py](#try-coinjoin)
4. [Running a "Maker" or "yield generator"](#run-maker)
5. [Running the tumbler script to boost privacy of owned coins](#run-tumbler)
<aname="configure"/>
## Configuring for Bitcoin Core.
Bitcoin Core is required to use Joinmarket; note that the node *can* be pruned.
Configuring Joinmarket for Core is now reduced, since there is no longer any `walletnotify` used.
First thing to do: in `scripts/`, run:
First thing to do: go into`scripts/`, and run:
python wallet-tool.py generate
(jmvenv)$ python wallet-tool.py generate
This *should* quit with an error, because the rpc is not configured. Open the newly created file `joinmarket.cfg`,
and edit:
@ -22,64 +56,336 @@ and edit:
rpc_host = localhost #default usually correct
rpc_port = 8332 # default for mainnet
Note, you can also use a cookie file by setting, in this section, a variable `rpc_cookie_file` to the location of the file,
as an alternative to using user/password.
Note, you can also use a cookie file by setting, in this section, a variable `rpc_cookie_file` to the location of the file, as an alternative to using user/password.
If you use Bitcoin Core's multiwallet feature, you can edit the value of rpc_wallet_file to your chosen wallet file.
If you use Bitcoin Core's multiwallet feature, you can edit the value of `rpc_wallet_file` to your chosen wallet file.
Then retry the same `generate` command; it should now not error (see below).
Then retry the same `generate` command; it should now not error (see [below](#generate)).
If you still get rpc connection errors, make sure you can connect to your Core node using the command line first.
### First step: make a Joinmarket wallet
<aname="wallet-tool"/>
## Using the `wallet-tool.py` script
This section leads you through the basics of creating, funding, recovering and displaying information about the Joinmarket wallet using the `wallet-tool.py` script. A reminder that, at this point, you should be on the command line and in the `scripts/` subdirectory of the Joinmarket directory.
The Joinmarket wallet is hierarchical and deterministic, it can be entirely recovered from a single seed. At any point, you can use `python wallet-tool.py --help` on the command line for all options. We'll now go through all the main functions:
<aname="generate"/>
### Creating a Wallet
Run wallet-tool.py with the `generate` method. Write down the 12 word seed on paper.
(jmvenv)$ python wallet-tool.py generate
Write down this wallet recovery mnemonic on paper:
matter aim multiply december stove march wolf nuclear yard boost worth supreme
Enter wallet encryption passphrase:
Reenter wallet encryption passphrase:
Input wallet file name (default: wallet.jmdat):
saved to wallet.jmdat
$
So far you have just created the wallet. But to do the next step, you'll need to see the addresses.
Run `python wallet-tool.py wallet.jmdat` (or replace the wallet name with whatever you chose). You'll get this message:
**restart Bitcoin Core with -rescan or use `bitcoin-cli rescanblockchain` if you're recovering an existing wallet from backup seed**
**Otherwise just restart this joinmarket application.**
Notice that here you **don't** need to rescan, because you are not "recovering an existing wallet" but creating a new one. Because it is new, none of the addresses are used yet. You only need to rescan (or use `rescanblockchain` as mentioned, which by the way can be a more efficient way of rescanning) if the addresses *have* been used ...
<aname="funding"/>
### Funding Wallet and Displaying Balance
... so, ignore this, and just run `python wallet-tool.py wallet.jmdat` again. You should get a large output like this:
m/49'/0'/4'/0/0 39n1tYKnPQ47bgCmsPWDHxuk2Mxo6GE4LZ 0.00000000 new
m/49'/0'/4'/0/1 3BoLqHDSHdMHyrSEAP31bBzzw45bFwZhy7 0.00000000 new
m/49'/0'/4'/0/2 31mo7D9UDmoStafYcyHpwV9sWY7oP9jUVQ 0.00000000 new
m/49'/0'/4'/0/3 3JMVGEyZ5nyJR9NVsfNY93c34xx9rEtzmq 0.00000000 new
m/49'/0'/4'/0/4 3AdD86dw59Q5EGHVupkHzR8pM4sW45bKqX 0.00000000 new
m/49'/0'/4'/0/5 3Gu7jTxcmh5dHJXgCE7Z5fdMgUWuenE2GE 0.00000000 new
Balance: 0.00000000
internal addresses m/49'/0'/4'/1
Balance: 0.00000000
Balance for mixdepth 4: 0.00000000
Total balance: 0.00000000
```
The [BIP32](https://github.com/bitcoin/bips/blob/master/bip-0049.mediawiki) xpub keys of each external branch are shown in case that is needed.
The BIP32 derivation paths are also shown; for Joinmarket they are defined by [BIP49](https://github.com/bitcoin/bips/blob/master/bip-0049.mediawiki); for more on this see [below](#structure).
**Bitcoins should be sent to empty external addresses** (sometimes known as receive addresses). You'll notice in
the above that while there are fields also for *internal addresses*, they are empty. That's because zero-balance internal addresses (also known as change addresses) are hidden by default. Using 'displayall' as the second argument will show all addresses in wallet, including empty used ones.
Read the instructions [here](https://github.com/JoinMarket-Org/joinmarket/wiki/Sourcing-commitments-for-joins) before funding the wallet.
Once you've funded it, run `python wallet-tool.py wallet.jmdat` again, enter your passphrase, and you should see the deposited coins.
<aname="spending"/>
### Spending from the wallet directly (without Coinjoin)
Note that Joinmarket's wallet supports spending to all address types, including legacy addresses ('1'), P2SH addresses ('3') and bech32 addresses ('bc1'). To learn more about how to spend directly without using CoinJoin, see [here](#no-coinjoin-sending-funds).
<aname="recover"/>
### Recovering a Wallet from mnemonic
In the event of loss of encrypted wallet file, use the 12 word seed to recover by running wallet-tool.py with 'recover' as first argument:
(jmvenv)$python wallet-tool.py recover
Input mnemonic recovery phrase: matter aim multiply december stove march wolf nuclear yard boost worth supreme
Input mnemonic extension, leave blank if there isnt one:
Enter wallet file encryption passphrase:
Reenter wallet file encryption passphrase:
Input wallet file name (default: wallet.jmdat):
Write down this wallet recovery mnemonic
matter aim multiply december stove march wolf nuclear yard boost worth supreme
Recovered wallet OK
#### Help! I recovered but some of my money is missing
Try increasing the gap limit up from its default of 6. This is more likely to happen if you ran a yield generator bot for a while (see below).
Another possible cause is you were running the tumbler script and it was stopped halfway through, with your coins currently at mixing depth 8. In that case increase the maximum mixdepth:
The `showseed` command will display the mnemonic for an existing wallet file, in case you've forgotten it. **It is highly recommended to keep a written backup of this phrase, lest you forget it!**
It is possible to recover a Joinmarket wallet in Trezor, Electrum or a number of other wallets, because it uses BIP49. But note that this is not ideal for privacy reasons (if you contact a server with your addresses), and also that because Joinmarket uses multiple accounts (=mixdepths), it may be more difficult than expected.
In difficult recovery situations, the `-p` command line flag can be used to print out private keys. Be very careful and consider this advanced usage:
m/49'/0'/1'/0/0 3DdhEr9GCoMDVRLNGAwi9rb8F4HQX8newY 0.00000000 new KzNosXo1sDfSJr7JcyqwYqGzTKTgAeQ9iEfiWA5YVE3Y6Tn3iCJs
.... etc
```
The above method (`-p`) still requires synchronizing the JoinMarket wallet. In the case where this isn't possible, individual private keys can still be exported:
You can create a csv file for opening with spreadsheet software:
(jmvenv)$ python wallet-tool.py --csv wallet.json history > history.csv
<aname="importprivkeys"/>
### Importing External Private Keys
Individual private keys can be imported into JoinMarket wallets by using the 'importprivkey' method. Multiple private keys are imported by separating them with commas or spaces. Use the `-M` flag to control which mixing depth the private keys are imported into. Be warned that handling raw private keys like this is dangerous.
You should read the warnings and understand the non-intuitive behaviour before using.
With [this](https://bitcoin.stackexchange.com/questions/29948/why-doc-says-importing-private-keys-is-so-dangerous) and [this](https://bitcoin.stackexchange.com/questions/18619/why-so-many-warnings-about-importing-private-keys) page.
It is best to avoid importing private keys if you can.
WARNING: This imported key will not be recoverable with your 12 word mnemonic seed. Make sure you have backups.
WARNING: Handling of raw ECDSA bitcoin private keys can lead to non-intuitive behaviour and loss of funds.
Recommended instead is to use the 'sweep' feature of sendpayment.py
Enter private key(s) to import: KzHJDZrSmmwkZKdLNS8L91qGsL9By6b48deaZRExBg4vAiyiBE7V Kxo3mHpUcx6KcLsyGTETh3ZJHEeU73tNCwYM1Yk7MMoTcW4jZ7Mi L3SdjpTu8tGdtht74wwsUX37bqGmr44AoyvZSqvrhTieN2GhbP7e
Private key(s) successfully imported
$
<aname="structure"/>
### What is the BIP32 wallet structure
#### The Mixing Depth Concept
The point of JoinMarket is to improve privacy. Merged transaction inputs are damaging to privacy because they provide evidence of common ownership. Each mixing depth is a different identity, coins are never merged in the same transaction across mixing depths, but may be merged within mixing depths. Coins move between mixing depths through coinjoins.
A change output stays in the same mixing depth. This prevents the situation where a change output is merged with a coinjoin output in a later transaction, which would render the coinjoin easily unmixable.
An example of the different identities being used is to not leak a lower limit of your wallet balance. Imagine if someone pays you $10 and sees it combined with $1 million, they could deduce you own at least that much. If instead those two payments go to different mixing levels then this analysis becomes harder. As coins move up the mixing levels via coinjoin, their identity becomes more uncertain. To introduce more uncertainty, have the coins separated by more mixing levels. E.G. A coin in level 0 and a second coin with level 1 will be merged with one set of coinjoins between them, the second coin at level 5 will be merged with 5 sets of coinjoins.
#### BIP32 Structure
m - generated from seed
m/49' - purpose: this is specified by [BIP49](https://github.com/bitcoin/bips/blob/master/bip-0049.mediawiki) for P2SH-wrapped segwit P2WPKH addresses
m/49'/0' - coin type 0 : see [BIP44](https://github.com/bitcoin/bips/blob/master/bip-0044.mediawiki) which specifies this as the coin type for Bitcoin mainnet
m/49'/0'/n' - nth mixing depth (nth account) (as per BIP44)
m/49'/0'/n'/0/k - kth external address, for mixing depth n
m/49'/0'/n'/1/k - kth internal address, for mixing depth n
Note that the quote (') indicates hardened derivation. See [BIP32](https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki) for technical details.
<aname="gaplimit"/>
### What is the Gap Limit?
With a deterministic wallet you create a sequence of bitcoin addresses and private keys from an initial seed. This list is the same every time it's generated, it means the entire wallet can be backed up by saving only the initial seed.
You can create as many addresses as you like, but not all of them will appear the blockchain. For instance I might create one especially for you to give me 1,000,000 BTC. That is (alas!) probably not going to be used so will likely never appear on the blockchain.
When you are starting JoinMarket it does not know which is the last address used. So you start at the beginning and see what is on the blockchain. Then you look for the next one in the sequence. The gap limit is how many *misses* you accept before you give up and stop looking. The same concept is used in other deterministic wallets like Electrum.
<aname="try-coinjoin"/>
## Try out a coinjoin; using `sendpayment.py`
**Doing single coinjoins doesn't really achieve a significant privacy boost; so you can skip much of this if that's your only interest, and go [here](#run-tumbler) instead to learn about the tumbler script; but do read here about how to spend coins from the wallet.**.
A single coinjoin *can* make destination (payment) outputs unclear, and can degrade automated blockchain surveillance significantly. We recommend using these opportunistically when making payments but on no account rely on any specific privacy gain from doing so.
Single coinjoins can be done using the script `sendpayment.py`. As with all Joinmarket user scripts, use `--help` to see a full list of options.
This sends 5000000 satoshi (0.05btc) to the address *mprGzBA9rQk82Ly41TsmpQGa8UPpZb2w8c* (testnet), with the default 5-7 (randomized) other parties from the default 0-th mixing depth from the wallet contained in the file *wallet.jmdat*. This will take some time, since Joinmarket will connect to remote messaging servers and do end to end encrypted communication with other bots, and also you will be paying some fees (more on this later in this section).
<aname="no-coinjoin-sending-funds"/>
#### No coinjoin, sending funds
Suppose you simply wanted to pay that address without coinjoin. Easy, just set the number of counterparties to zero:
in the previously linked page. Don't neglect the point mentioned that, if you are planning to do your own coinjoins
(act as "Taker", you must fund multiple different addresses to avoid inconvenience in future. For Makers, that's not necessary.
Setting the *amount* to zero will cause the bot to sweep. Meaning it will empty that mixing depth, calculating the coinjoin fee so that no coins are left.
If you are upgrading from non-Segwit JM, you'll want to read this [upgrade guide](SEGWIT-UPGRADE.md).
... sends all coins in mixdepth 0 (default), minus whatever is needed as coinjoin and network fees, to the destination address, via a CoinJoin with 7 other parties. Note that CoinJoins that are sweeps are particularly powerful because no change is left behind to make future linkages.
**Doing single coinjoins doesn't really achieve a significant privacy boost;
so you can skip this step if that's your only interest, and go to step 4b instead
to learn about the tumbler script, which we strongly recommend to achieve this goal**.
### Fee settings
A single coinjoin *can* make destination (payment) outputs unclear, and can degrade automated blockchain surveillance significantly.
We recommend using these opportunistically when making payments but on no account rely on any specific privacy gain from doing so.
There are two different types of fee; bitcoin network transaction fees and fees paid to counterparties for providing liquidity for coinjoins (that latter being the central idea of Joinmarket).
To try doing one coinjoin, use the `sendpayment.py` script. See [here](https://github.com/JoinMarket-Org/joinmarket/wiki/Sending-payments-with-CoinJoin#send-payment)
#### Bitcoin network fees.
(Note that the `patientsendpayment` option, also described on that page, is not yet implemented in this version, but probably will be soon).
This is controlled using the setting of `tx_fees` in the `[POLICY]` section in your `joinmarket.cfg` file in the current directory. If you set it to a number between 1 and 144 it is treated as the targeted number of blocks for confirmation; e.g. if you set it to 20 you are asking to use whatever Bitcoin Core things is a realistic fee to get confirmation within the next 20 blocks. By default it is 3. If you set it to a number > 144, don't set it lower than about 1200, it will be interpreted as "number of satoshis per kilobyte for the transaction fee". 1000 equates to 1 satoshi per byte (ignoring technical details of vbyte), which is usually the minimum fee that nodes on the network will relay. Note that Joinmarket will deliberately vary your choice randomly, in this case, by 20% either side, to avoid you watermarking all your transactions with the exact same fee rate. As an example, if you prefer to use an approximate rate of 20 sats/byte rather than rely on Bitcoin Core's estimated target for 3 or 6 blocks, then set `tx_fees` to 20000.
Note that some liquidity providers (Makers) will offer very small contributions to the tx fee, but mostly you should consider that you must pay the whole Bitcoin network fee yourself, as an instigator of a coinjoin (a Taker). Note also that if you set 7 counterparties, you are effectively paying for approximately 7 normal sized transactions; be cognizant of that!
(Note also that you can use 'schedule' files in this new version, but that's more advanced so ignore it for now.)
#### CoinJoin fees.
### 4a step: if you want to be a "Maker" or "yield generator".
Individual Makers will offer to do CoinJoin at different rates. Some set their fee as a percentage of the amount, and others set it as a fixed number of satoshis. Most set their rates very low, so in most (but not all) cases the overall CoinJoin fee will be lower than the bitcoin network fees discussed above. When starting to do a CoinJoin you will be prompted to set the *maximum* relative (percentage) and absolute (number of satoshis) fees that you're willing to accept from one participant; your bot will then choose randomly from those that are below at least one of those limits. Please read these instructions carefully and update your config file accordingly to avoid having to answer the questions repeatedly.
Read the introductory guide [here](https://github.com/JoinMarket-Org/joinmarket/wiki/Running-a-Yield-Generator).
Note also that you can use 'schedule' files, but that's more advanced so ignore it for now; see the README under scripts/ for more details.
You can use (recommended) the `yg-privacyenhanced.py` script in the `scripts/` directory.
You can also instead use the `yield-generator-basic.py` which is almost the same but the former has
some slight behaviour tweaks to make the privacy properties of the bot better and is more clearly documented.
<aname="run-maker"/>
If you're new to this, note the slightly strange/surprising fact that the settings are editable in the source file;
at the top of those two script files (`yield-generator-basic.py` and `yg-privacyenhanced.py`), there is a user-editable section;
the comments tell you what to change and what it means.
## Running a "Maker" or "yield generator".
For veteran users, note the new offertypes are 'swreloffer', 'swabsoffer' - they function the same, but use segwit.
Follow the guide [here](YIELDGENERATOR.md).
<aname="run-tumbler"/>
### 4b step: Running the tumbler script to boost privacy of owned coins.
### Running the tumbler script to boost privacy of owned coins.
A "yield generator" is a JoinMarket bot which does market-making of CoinJoins to produce an income as a join "Maker". The bot connects to the JoinMarket trading pit on the available messaging channels, announces its offers and waits. Market "Takers" will communicate with it to create a CoinJoin transaction.
## A few words about incentives
The first thing to understand is that this is not a bank account with a guaranteed interest rate. You are selling a product, namely coinjoins, and your customers can take it or leave it depending on your offered fee, range of available coinjoin amount, internet speed, latency and so on. Most of the actual decision-making is done by software bots rather than humans but the same principles apply. The algorithm for market takers remembers yield generators they previously dealt with and avoid those who did not offer a good experience. You have an incentive to be on your best behavior.
### Things You Need
+ A local Bitcoin full node running with JoinMarket. No other Bitcoin blockchain access is currently supported. Your node can be pruned.
+ A reliable, relatively fast, always-on internet connection. CoinJoins require multiple people to take part, if one yield generator times-out or is slow, the entire process is held up or must be restarted. People are looking for joins 24/7, so you'll need to run Joinmarket all the time to make joins as well. The entire point of a market maker is to offer a deal immediately, without any waiting or hold-ups.
+ A significant enough Bitcoin balance in your joinmarket wallet to offer a wide range of coinjoin amounts. If you can only offer from zero to 1 million satoshis (0.01 BTC), you're unlikely to get many coinjoin deals. The bigger your balance, the better.
+ Update often. This software is still in the very early stages. Keep an eye out of updates. Perhaps subscribe to the [twitter](https://twitter.com/joinmarket/) or [subreddit](https://www.reddit.com/r/joinmarket) to get news about updates.
## Requirements / How-to
You will need to:
+ Install JoinMarket (see the README file of this repository, then follow the [usage guide](USAGE.md)).
+ If you've followed the usage guide and funded your wallet with some coins (anywhere is fine, but the 0th mixdepth is most logical), you can then configure your yield generator script:
+ Edit `yg-privacyenhanced.py` or `yield-generator-basic.py` (former is recommended) at the top of the file with your chosen fee selections, then run this variant yield-generator.py on a permanently-on computer with a stable internet connection. For example:
+ Wait and be patient. The volume of joins is always dependent on the number of takers. A successful join may take a day or longer to appear.
## Configuring
Open one of the two built-in yield generators: `yield-generator-basic.py` or `yg-privacyenhanced.py` in a text editor and edit the configuration. Currently these are the fields. Most of them can just be left at the default values.
txfee = 1000
cjfee_a = 2000
cjfee_r = '0.0002' # 0.02% fee
ordertype = 'swreloffer'
minsize = int(1.2 * txfee / float(cjfee)) #minimum size is such that you always net profit at least 20% of the miner fee
Meaning of fields:
+ txfee. The yield generators contribution to the miner fee.
+ ordertype. One of 'swreloffer' or 'swabsoffer'.
+ cjfee_r. Your offered coinjoin fee as a fraction of the coinjoin amount (for 'swreloffer'). Takers pay this to you.
+ cjfee_a. Your offered coinjoin fee as an absolute number of satoshis (for 'swabsoffer')
## Keeping Track of Returns
As well as simply viewing your balance in-wallet occasionally as explained in the [usage guide](USAGE.md), the history can also be calculated with the `history` method, also explained in that document.
## I deposited X btc into yield generator, why is it only offering Y (<X)btcinthemarket?
Short answer: privacy
Long answer: https://bitcointalk.org/index.php?topic=919116.msg11465848#msg11465848
## My bitcoins have been split up into many small UTXOs
You may wish to change the input merging policy in `joinmarket.cfg`, see this page: https://github.com/JoinMarket-Org/joinmarket/wiki/Configuring-with-joinmarket.cfg#input-merging-policy
The value of `gradual` or `greedy` is usually good for keeping the number of inputs from getting too large.
## Modifying the algorithm
If you are a programmer you could try creating your own algorithm for the yield generator.
Other yield-generator algorithms are in a separate github repository: https://github.com/JoinMarket-Org/custom-scripts Many more people can work on them without the possibility of putting the main JoinMarket code in danger.
AdamISZ
6 years ago