Browse Source
For detailed description of functionality in this commit, see the newly created docs/SNICKER.mdmaster
Adam Gibson
5 years ago
32 changed files with 2800 additions and 183 deletions
@ -0,0 +1,349 @@
|
||||
# HOW TO USE THE SNICKER FEATURES IN JOINMARKET |
||||
|
||||
# Contents |
||||
|
||||
1. [Basic concepts and definition](#basic) |
||||
|
||||
a. [Quick read: advice](#quick) |
||||
|
||||
b. [Slightly longer description](#longer) |
||||
|
||||
c. [Proof of work for anti-spam](#pow) |
||||
|
||||
d. [Servers](#servers) |
||||
|
||||
e. [New script tools](#scripts) |
||||
|
||||
2. [Updating config](#configure) |
||||
|
||||
3. [Alternative to mainnet usage](#network) |
||||
|
||||
4. [Running SNICKER as a receiver](#receiver) |
||||
|
||||
a. [Manually running the receiver](#manually) |
||||
|
||||
b. [Running a yield-generator with SNICKER active](#yieldgen) |
||||
|
||||
5. [Use of wallets](#wallets) |
||||
|
||||
6. [Checking for SNICKER coinjoins](#checking) |
||||
|
||||
7. [A testing workflow](#testing) |
||||
|
||||
8. [Appendix: Example SNICKER transaction](#appendix1) |
||||
|
||||
<a name="basic" /> |
||||
|
||||
## Basic concepts and definition |
||||
|
||||
<a name="quick" /> |
||||
|
||||
### Quick read: advice |
||||
|
||||
For the time constraints, consider these points: |
||||
|
||||
* This is *somewhat* experimental; better run it on signet for now, mainnet is not advised. |
||||
* If you do run it on mainnet, make an effort to keep backups of your jmdat wallet file; recovery with seed only is possible (a tool is provided), but it's a pain. |
||||
* This basically allows coinjoins to be proposed and executed without any interaction by the participants, even over a message channel. You can run it passsively in a yield generator, for example. You can even be paid some small amount of sats for that to happen. But the coinjoins are only 2-party. |
||||
|
||||
<a name="longer" /> |
||||
|
||||
### Slightly longer read on what this is: |
||||
|
||||
For formal specs as currently implemented, please use [this](https://gist.github.com/AdamISZ/2c13fb5819bd469ca318156e2cf25d79). Less technical description [here](https://joinmarket.me/blog/blog/snicker/). |
||||
|
||||
Essentially, this is a two party but non-interactive protocol. The **proposer** will identify, on the blockchain, a candidate transaction where he has some confidence that one or more inputs are owned by the same party as one output, and that that party has SNICKER receiver functionality. |
||||
Given those conditions, he'll create one or more **proposals** which are of form `base64-encoded-ECIES-encrypted-PSBT,hex-encoded-pubkey` (the separator is literally a comma; this is ASCII encoded), and send them to a **snicker-server** which is hosted at an onion address (possibly TLS but let's stick with onion for now, it's easier). They could also be sent manually. We'll talk more about these two possibilities below. |
||||
|
||||
To understand the step-by-step of how this is done "under the hood", you may find the [section on testing](#testing) a useful read. If you're only interested in "switching on this feature", notice this is not advised on mainnet (see the section on [alternatives to mainnet](#network)), but read more below about editing the config and switching it on in a yieldgenerator. |
||||
|
||||
<a name="pow" /> |
||||
|
||||
### Proof of work for mild anti-spam |
||||
|
||||
As implemented here, in fact, the proposer attaches a proof of work in the form of a 10-byte nonce appended to the end of the above string (hex encoded; so in fact: `base64-encoded-ECIES-encrypted-PSBT,hex-encoded-pubkey,hex-encoded-nonce` is what is sent over the wire). This nonce is grinded to get a 32-byte-truncated-hash512 of that string to be less than a target calculated by a request number of bits from the server. The target bits is requested by the proposer with a `GET /target` request to the server, before sending the proposals themselves with a `POST /` request with the proposals in the body. For the proof of work, see `jmbase/jmbase/proof_of_work.py`; it's pretty elementary. Note of course this is no defence against a serious attempt to jam the system, it's only a "script-kiddie-defence", so to speak. |
||||
|
||||
<a name="servers" /> |
||||
|
||||
### Servers |
||||
|
||||
The **snicker-server** just hosts the proposals and lets others read them. For the purpose of testing it's fine that we don't have a very sophisticated version of this, but for now note: |
||||
* You can run tests using the `-n` option of `create-snicker-proposal.py` to just output to terminal instead of uploading to server; this may be in particular more useful for local tests where it's only your own wallets involved. |
||||
* The server serves only over a Tor hidden service. |
||||
* It stores the accepted proposals in a sqlite3 database `proposals.db`. The table is `proposals` and has only two fields: `pubkey` and `proposal` (see above). |
||||
* Defends against spam with proof of work as per above (this is very limited but better than nothing). |
||||
* Currently has NO maintenance or performance feature such as flushing out proposals after a time limit, or allowing filtered queries. This seems the most obvious way to improve what exists, here. |
||||
|
||||
<a name="scripts" /> |
||||
|
||||
### New script tools |
||||
|
||||
The new tools for SNICKER are in the directory `scripts/snicker`, consisting of: |
||||
* `snicker-seed-tx.py` - create a fake SNICKER transaction in your own wallet. |
||||
* `snicker-finder.py` - scan recent blocks for Joinmarket or SNICKER candidate transactions. |
||||
* `create-snicker-proposal.py` - takes transactions found from the above and makes proposals, uploading them to a server or outputting to command line. |
||||
* `snicker-server.py` - implements a simple server serving over *.onion, with a sqlite database to store proposals, and defends against spam only mildly with a proof of work requirement (see below). |
||||
* `receive-snicker.py` - polls above server to read new proposals, parse them and broadcasts completed SNICKER coinjoins when found, storing the new keys as imports (see details on wallet handling below). |
||||
* `snicker-recovery.py` - can be used to recover a wallet from seedphrase which contains SNICKER utxos, though it needs (possibly multiple) rescanblockchain calls (and informs the user how to do this, including blockheights). |
||||
|
||||
<a name="configure" /> |
||||
|
||||
## Updating config |
||||
|
||||
Recreate your joinmarket.cfg in the usual way. You can then edit the new `[SNICKER]` section as desired: |
||||
|
||||
``` |
||||
[SNICKER] |
||||
|
||||
# any other value than 'true' will be treated as False, |
||||
# and no SNICKER actions will be enabled in that case: |
||||
enabled = false |
||||
|
||||
# in satoshis, we require any SNICKER to pay us at least |
||||
# this much (can be negative), otherwise we will refuse |
||||
# to sign it: |
||||
lowest_net_gain = 0 |
||||
|
||||
# comma separated list of servers (if port is omitted as :port, it |
||||
# is assumed to be 80) which we will poll against (all, in sequence); note |
||||
# that they are allowed to be *.onion or cleartext servers, and no |
||||
# scheme (http(s) etc) needs to be added to the start. |
||||
servers = cn5lfwvrswicuxn3gjsxoved6l2gu5hdvwy5l3ev7kg6j7lbji2k7hqd.onion, |
||||
|
||||
# how many minutes between each polling event to each server above: |
||||
polling_interval_minutes = 60 |
||||
``` |
||||
|
||||
Notice that it is of course *NOT* enabled by default, so switch that to `true`. |
||||
|
||||
If you are running tests, a 60 minute polling interval is slow, feel free to cut it down to a minute or two. |
||||
|
||||
The default server is currently running on a VPS. As mentioned above, you can easily run your own server with `snicker-server.py`. It is possible to poll multiple servers, comma separated in the list. |
||||
|
||||
<a name="network" /> |
||||
|
||||
## Alternative to mainnet usage |
||||
|
||||
Choosing the network for this function means editing the `[BLOCKCHAIN]` section in `joinmarket.cfg`, just as for other Joinmarket functionality. |
||||
|
||||
**Regtest** is a good option if you are interested in testing functionality quickly, on your own. See [here](TESTING.md) for some info on regtest setup. |
||||
|
||||
**Signet** (a new testnet) will be helpful especially if you intend to test with others. This can be done by simply sharing proposals as per the `-n` comments above, or by sharing proposal server locations as onion addresses, and possibly communicating with other testers to identify candidate transactions (obviously this strays far from the intended way SNICKER will be used, but it is convenient to test workflow). |
||||
For more information about using Joinmarket with signet, see the [0.8.1 release notes](release-notes/release-notes-0.8.1.md) and [this gist](https://gist.github.com/AdamISZ/325716a66c7be7dd3fc4acdfce449fb1). |
||||
|
||||
**Testnet3** - it may be possible but it will be far less convenient than signet. |
||||
|
||||
**Mainnet** - as of Feb 2021, this isn't recommended yet; it should in theory work no different, but any usage would be at your own risk. |
||||
|
||||
<a name="receiver" /> |
||||
|
||||
## Running SNICKER as a receiver |
||||
|
||||
The **receiver** (unless handling manually with `-n` as per above) polls this server (for testing, you can make the polling loop fast; in real usage it should be slow), reads all the existing proposals using a `GET /` request with no parameters, and if it can decrypt and sanity check the transaction OK, it co-signs it and broadcasts it. Note: *the receiver wallet will store its new coins output from the coinjoin, as imported keys; they are not part of the HD tree, although derivable from history*. See `use of wallets` below for important notes on this aspect. |
||||
|
||||
<a name="manually" /> |
||||
|
||||
### Manually running the receiver |
||||
|
||||
For this you can use the `receive-snicker.py` script as detailed above, passing the chosen wallet file as argument, and it will "one-shot" poll for proposals and process them, or, you can pass the base64 proposal manually instead. |
||||
|
||||
<a name="yieldgen" /> |
||||
|
||||
### Running SNICKER in a yield generator. |
||||
|
||||
This will presumably be the most normal way to be a SNICKER receiver over time; if `enabled=true` for SNICKER is in the above config settings, this will happen automatically, under the hood. Make sure the polling loop interval is not too fast if you leave this running longer term (even if a test bot). If valid proposals are found that follow our requirements, the transactions are broadcast. |
||||
|
||||
<a name="wallets" /> |
||||
|
||||
## Use of wallets |
||||
|
||||
**Wallet type** - please stick with native segwit (`native=true` in config *before* you generate), but you can also choose p2sh-p2wpkh, it should work. No other script type (including p2pkh) will work here. We don't want mixed script type SNICKER coinjoins. |
||||
|
||||
**Persistence in the wallet** - this is very important and not at all obvious! But, on regtest and testnet *by default*, we use hex seeds instead of wallet files and `VolatileStorage` (wallet storage in memory; wiped on shutdown). This is fine and convenient for many tests, but will not work for a key part of SNICKER - imported keys. |
||||
|
||||
The upshot - **make sure you actually generate wallet files for all wallets you're going to test SNICKER with**, otherwise you will not even see the created coins on the receiver side. |
||||
|
||||
Additionally, when you view the wallet with wallet-tool, you need to use `--recoversync`, as the default fast sync won't see imported keys. If you took these two steps, your tests should correctly show the post-SNICKER created coins. |
||||
|
||||
<a name="checking" /> |
||||
|
||||
## Checking for SNICKER coinjoins |
||||
|
||||
SNICKER is logged to: |
||||
|
||||
`~/.joinmarket/logs/SNICKER/SNICKER-joinmarket-wallet-xxxxxx.log` |
||||
|
||||
i.e. there is one SNICKER log file for each wallet. |
||||
This should show all transactions that were detected and broadcast. |
||||
|
||||
<a name="testing" /> |
||||
|
||||
## A testing workflow |
||||
|
||||
This is a scenario for a single user, using either regtest or signet. |
||||
|
||||
Generate a minimum of two joinmarket wallets with `python wallet-tool.py generate`, as noted above, native (or at least, both the same type). |
||||
|
||||
Fund them both. The receiver needs at least two utxos to create the seed transaction. |
||||
|
||||
Create a seed fake-SNICKER transaction in the receiver wallet, using the script `snicker-seed-tx.py`. |
||||
|
||||
Start the test server. Navigate to `scripts/snicker` and run `python snicker-server.py` - no arguments should be needed, and this will generate an onion running serving on port 80; the onion hostname is displayed: |
||||
|
||||
``` |
||||
(jmvenv) waxwing@here~/testjminstall/joinmarket-clientserver/scripts/snicker$ python snicker-server.py |
||||
User data location: |
||||
Attempting to start onion service on port: 80 ... |
||||
Your hidden service is available: |
||||
xpkqk2cy2h2ay5iecwcod5ka36nxj2tsiyczk2w5c6o7h5g57w3xg4id.onion |
||||
``` |
||||
|
||||
This is ephemeral, obviously we intend the real servers to be long-running. The one in the default config should exist and be long-running already, but of course your tests don't need to rely on this. Add: |
||||
|
||||
``` |
||||
|
||||
[SNICKER] |
||||
enabled = true |
||||
servers= xpkqk2cy2h2ay5iecwcod5ka36nxj2tsiyczk2w5c6o7h5g57w3xg4id.onion, |
||||
``` |
||||
... to a `joinmarket.cfg` that you add inside `scripts/snicker`, by copying it from `scripts/` or wherever you keep your testing `joinmarket.cfg` file. (This manual annoyance is part of testing, it won't be needed in mainnet usage of course ... alternatively just put a signet/regtest custom `joinmarket.cfg` in your `.joinmarket` folder, but this is hardly less annoying). |
||||
|
||||
`servers=` requires a comma separated list. |
||||
|
||||
You're now ready to do the two steps: (a) create a proposal and upload it, (b) download proposals (as the receiver identity/wallet) and complete coinjoins. It could be different people doing (a) and (b) of course but here we're assuming one tester doing everything (see two wallets above). |
||||
|
||||
### Creating one or more proposals |
||||
|
||||
Having done the above seed transaction, do a scan operation to find the candidate: |
||||
|
||||
`cd script/snicker; python snicker-finder.py --datadir=. 330` |
||||
|
||||
Here, 330 is the starting block number on my regtest blockchain; the ending block is the current block. On signet use a block explorer to find current height (or `bitcoin-cli getblockchaininfo`). It will look for all transactions with a SNICKER pattern and you should see returned something like this: |
||||
|
||||
``` |
||||
2020-12-28 16:09:04,329 [INFO] Finished processing block: 790 |
||||
2020-12-28 16:09:04,334 [INFO] Found SNICKER transaction: 32f80807b3ba4ca477b25e8ab608a8a3134a34c8c3787cad95c653d1805d7533 in block: 791 |
||||
2020-12-28 16:09:04,338 [INFO] Finished processing block: 791 |
||||
2020-12-28 16:09:04,340 [INFO] Finished processing block: 792 |
||||
done |
||||
``` |
||||
|
||||
Then look in `./candidates.txt` to find the details of the identified transaction, including its full hex, which you need to copy: |
||||
|
||||
``` |
||||
2020-12-16 11:13:01,708 [INFO] { |
||||
"hex": "0200000000010138e8a90b3df7740b9d5f5ae9af2cf6769f314d290b2e12bf25bfa4aae2c0cbe20000000000feffffff0280ba8c010000000016001471b09afbac6204627225c10f3a8d4a0749364fdb6d7c36220000000016001447ae59f32c504cbb56e18b77f7842fb58b55025b02473044022075354351ad4c619ba662f9abd25e8ee434f8381795001606a29fa959d36aeb7f022018f8bf1ec0407dad586baeb7e4d977aaacbd8fb15579293e6d739ad69ac3c6cf012103f8e827464fb83209c194376c53ae8f4e7ab5f1baf0948705fec6dd421f2b65c37a020000", |
||||
"inputs": [ |
||||
................. |
||||
Full transaction hex for creating a proposal is found in the above. |
||||
The unspent indices are: 0 1 2 |
||||
``` |
||||
|
||||
Copy that fully signed transaction hex, and note the unspent outputs. You're going to assume (in this case correctly of course) that all of the inputs are valid options for the SNICKER public key (why? because it was a *seed* transaction, it was a fake SNICKER, and so all the inputs belonged to the same wallet). |
||||
|
||||
At this point you're ready to run the proposal creator: |
||||
|
||||
``` |
||||
python create-encrypted-proposal.py --datadir=. proposerwallet.jmdat "0200000000010138e8a90b3df7740b9d5f5ae9af2cf6769f314d290b2e12bf25bfa4aae2c0cbe20000000000feffffff0280ba8c010000000016001471b09afbac6204627225c10f3a8d4a0749364fdb6d7c36220000000016001447ae59f32c504cbb56e18b77f7842fb58b55025b02473044022075354351ad4c619ba662f9abd25e8ee434f8381795001606a29fa959d36aeb7f022018f8bf1ec0407dad586baeb7e4d977aaacbd8fb15579293e6d739ad69ac3c6cf012103f8e827464fb83209c194376c53ae8f4e7ab5f1baf0948705fec6dd421f2b65c37a020000" 0 1 100 -m1 -n |
||||
``` |
||||
Obviously see the `--help` for details, but in this example we chose: |
||||
|
||||
* input index 0 to source the pubkey for the encryption. |
||||
* output index 1 for the coin we want the receiver to spend with us in the coinjoin. |
||||
* 100 sats as the amount we will bump their output by as an incentive to do the coinjoin (you *can* make this number negative, to receive, instead - the proposer is paying the tx fee otherwise, note). |
||||
* mixdepth 1 as the mixdepth from which we source *our* coins for the coinjoin. Make sure of course that mixdepth 1 has at least a little bit more bitcoin than the size of that output at index 1 aforementioned. |
||||
* The `-n` option can be used to output the proposal to stdout (it is base64+hex so copy-pasteable). |
||||
|
||||
If you choose not to use -n, but instead use a proposals server as above, then assuming it connects to the server OK, you will see: |
||||
|
||||
``` |
||||
Response from server: http://xpkqk2cy2h2ay5iecwcod5ka36nxj2tsiyczk2w5c6o7h5g57w3xg4id.onion was: 1 proposals-accepted |
||||
``` |
||||
|
||||
### Receiving the created proposals. |
||||
|
||||
The last phase is pretty simple, if it works - just run the receiver script (from `scripts/snicker`) as follows: |
||||
|
||||
``` |
||||
python receive-snicker.py --datadir=. receiver.jmdat [proposal] |
||||
User data location: . |
||||
2020-12-16 11:43:03,779 [DEBUG] rpc: getblockchaininfo [] |
||||
2020-12-16 11:43:03,781 [DEBUG] rpc: getnewaddress [] |
||||
Enter passphrase to decrypt wallet: |
||||
2020-12-16 11:43:07,501 [DEBUG] rpc: listaddressgroupings [] |
||||
2020-12-16 11:43:07,562 [DEBUG] Fast sync in progress. Got this many used addresses: 3 |
||||
2020-12-16 11:43:08,075 [DEBUG] rpc: listunspent [0] |
||||
2020-12-16 11:43:08,216 [DEBUG] bitcoind sync_unspent took 0.14214825630187988sec |
||||
2020-12-16 11:43:08,280 [WARNING] Cannot listen on port 27183, trying next port |
||||
2020-12-16 11:43:08,281 [WARNING] Cannot listen on port 27184, trying next port |
||||
2020-12-16 11:43:08,281 [WARNING] Cannot listen on port 27185, trying next port |
||||
2020-12-16 11:43:08,281 [INFO] Listening on port 27186 |
||||
2020-12-16 11:43:08,282 [INFO] (SNICKER) Listening on port 26186 |
||||
2020-12-16 11:43:08,282 [INFO] Starting transaction monitor in walletservice |
||||
2020-12-16 11:43:08,339 [INFO] Starting SNICKER polling loop |
||||
2020-12-16 11:43:22,676 [DEBUG] rpc: sendrawtransaction ['020000000001028ffa6a6f0184ed8123993273ecbb4af82d1b1c0963c815fec4e92525eaba56b30000000000ffffffffa5015509e0e241ef25ee7ccc1936295c908e572cb222105e16c197d66f0599640000000000ffffffff03e4ba8c0100000000160014190ec76b7843f47bc367b65119b98c32074536255dfd5e0a00000000160014d38fa4a6ac8db7495e5e2b5d219dccd412dd9baee4ba8c01000000001600147b4676f859b993257bc8d5880650fcab470db8a1024830450221008480d553177a020f58ca0e45b9e20aa027305a279a3de1014f55ff22909b89b1022054e848285ee60c169b5de19bb4d3637b606ff14bc4cca4506ad05a42fff6af400121029a82a00f05d023f188dfd1db82ef8ec136b0500bbd33bb1f65930c5b74e3199802463043021f01d3f4567c32fc0c5c0cd33db233a3c74100a36940d743b72042b55e60b89d022073ab203ad0fee389f2a2c9e62197244cea95b07ae78a5516ca9f866a8e348d2c01210245d8623c4b06505dffd21bdd314a84b73afe2b9d49a93fe89397b48a85b718bd00000000'] |
||||
2020-12-16 11:43:22,678 [INFO] Successfully broadcast SNICKER coinjoin: 33ec857df09030140391529295412434cced8191626024f937426b7859a21947 |
||||
2020-12-16 11:43:23,359 [INFO] Removed utxos= |
||||
b356baea2525e9c4fe15c863091c1b2df84abbec7332992381ed84016f6afa8f:0 - path: m/84'/1'/4'/0/0, address: bcrt1qwxcf47avvgzxyu39cy8n4r22qaynvn7mc359ap, value: 26000000 |
||||
2020-12-16 11:43:23,360 [INFO] Added utxos= |
||||
33ec857df09030140391529295412434cced8191626024f937426b7859a21947:0 - path: imported/1/0, address: bcrt1qry8vw6mcg068hsm8keg3nwvvxgr52d3923gg45, value: 26000100 |
||||
``` |
||||
|
||||
Obviously this is the ideal case: if no errors occur. If invalid proposals, or proposals on coins that no longer exist because you already spent them, are encountered, logging messages are displayed to that effect. |
||||
|
||||
If you did choose the `-n` option then you can pass the copy-pasted proposal on the command line and it will just process that instead of polling. |
||||
|
||||
### Other kinds of testing |
||||
|
||||
The above is the baseline workflow. Additionally, you can test: |
||||
|
||||
#### Wallet recovery from seed |
||||
|
||||
This case is already somewhat tricky in Joinmarket, but for the worst possible scenario of only having a seedphrase and no address imports in the wallet and no Joinmarket jmdat file, and having used SNICKER recently and not spent those coins, recovery is particularly tricky (which is why users who enable this feature must be warned that it's very important not to lose the jmdat file). However even in this case full recovery is possible, using the script `snicker-recovery.py`. To fully test this try making multiple SNICKER transactions in a wallet, then deleting the jmdat and creating a new Core wallet (on the same regtest instance of course!), enabling it in joinmarket.cfg, then running `wallet-tool.py recover` with the seedphrase, then running the aforementioned snicker recovery script; it will prompt you to `rescanblockchain` from certain heights, potentially more than once; the reason for this is that Core cannot find arbitrarily the transactions which spend custom keys which we discover during wallet recovery, we need to import and rescan before going to the next step, although this will only be an edge case. |
||||
|
||||
<a name="appendix1" /> |
||||
|
||||
## Appendix: Example SNICKER transaction |
||||
|
||||
This is what is produced by `print(jmbitcoin.human_readable_transaction(jmbitcoin.CTransaction.deserialize(jmbase.hextobin('020000000001028ffa6a6f0184ed8123993273ecbb4af82d1b1c0963c815fec4e92525eaba56b30000000000ffffffffa5015509e0e241ef25ee7ccc1936295c908e572cb222105e16c197d66f0599640000000000ffffffff03e4ba8c0100000000160014190ec76b7843f47bc367b65119b98c32074536255dfd5e0a00000000160014d38fa4a6ac8db7495e5e2b5d219dccd412dd9baee4ba8c01000000001600147b4676f859b993257bc8d5880650fcab470db8a1024830450221008480d553177a020f58ca0e45b9e20aa027305a279a3de1014f55ff22909b89b1022054e848285ee60c169b5de19bb4d3637b606ff14bc4cca4506ad05a42fff6af400121029a82a00f05d023f188dfd1db82ef8ec136b0500bbd33bb1f65930c5b74e3199802463043021f01d3f4567c32fc0c5c0cd33db233a3c74100a36940d743b72042b55e60b89d022073ab203ad0fee389f2a2c9e62197244cea95b07ae78a5516ca9f866a8e348d2c01210245d8623c4b06505dffd21bdd314a84b73afe2b9d49a93fe89397b48a85b718bd00000000'))))`: |
||||
|
||||
``` |
||||
{ |
||||
"hex": "02000000000102578770b2732aed421ffe62d54fd695cf281ca336e4f686d2adbb2e8c3bedb2570000000000ffffffff4719a259786b4237f92460629181edcc3424419592529103143090f07d85ec330100000000ffffffff0324fd9b0100000000160014d38fa4a6ac8db7495e5e2b5d219dccd412dd9bae24fd9b0100000000160014564aead56de8f4d445fc5b74a61793b5c8a819667af6c208000000001600146ec55c2e1d1a7a868b5ec91822bf40bba842bac502473044022078f8106a5645cc4afeef36d4addec391a5b058cc51053b42c89fcedf92f4db1002200cdf1b66a922863fba8dc1b1b1a0dce043d952fa14dcbe86c427fda25e930a53012102f1f750bfb73dbe4c7faec2c9c301ad0e02176cd47bcc909ff0a117e95b2aad7b02483045022100b9a6c2295a1b0f7605381d416f6ed8da763bd7c20f2402dd36b62dd9dd07375002207d40eaff4fc6ee219a7498abfab6bdc54b7ce006ac4b978b64bff960fbf5f31e012103c2a7d6e44acdbd503c578ec7d1741a44864780be0186e555e853eee86e06f11f00000000", |
||||
"inputs": [ |
||||
{ |
||||
"outpoint": "57b2ed3b8c2ebbadd286f6e436a31c28cf95d64fd562fe1f42ed2a73b2708757:0", |
||||
"scriptSig": "", |
||||
"nSequence": 4294967295, |
||||
"witness": "02473044022078f8106a5645cc4afeef36d4addec391a5b058cc51053b42c89fcedf92f4db1002200cdf1b66a922863fba8dc1b1b1a0dce043d952fa14dcbe86c427fda25e930a53012102f1f750bfb73dbe4c7faec2c9c301ad0e02176cd47bcc909ff0a117e95b2aad7b" |
||||
}, |
||||
{ |
||||
"outpoint": "33ec857df09030140391529295412434cced8191626024f937426b7859a21947:1", |
||||
"scriptSig": "", |
||||
"nSequence": 4294967295, |
||||
"witness": "02483045022100b9a6c2295a1b0f7605381d416f6ed8da763bd7c20f2402dd36b62dd9dd07375002207d40eaff4fc6ee219a7498abfab6bdc54b7ce006ac4b978b64bff960fbf5f31e012103c2a7d6e44acdbd503c578ec7d1741a44864780be0186e555e853eee86e06f11f" |
||||
} |
||||
], |
||||
"outputs": [ |
||||
{ |
||||
"value_sats": 27000100, |
||||
"scriptPubKey": "0014d38fa4a6ac8db7495e5e2b5d219dccd412dd9bae", |
||||
"address": "bc1q6w86ff4v3km5jhj79dwjr8wv6sfdmxaw2dytjc" |
||||
}, |
||||
{ |
||||
"value_sats": 27000100, |
||||
"scriptPubKey": "0014564aead56de8f4d445fc5b74a61793b5c8a81966", |
||||
"address": "bc1q2e9w44tdar6dg30utd62v9unkhy2sxtxtqrthh" |
||||
}, |
||||
{ |
||||
"value_sats": 146994810, |
||||
"scriptPubKey": "00146ec55c2e1d1a7a868b5ec91822bf40bba842bac5", |
||||
"address": "bc1qdmz4ctsarfagdz67eyvz906qhw5y9wk9dqpuea" |
||||
} |
||||
], |
||||
"txid": "ca606efc5ba8f6669ba15e9262e5d38e745345ea96106d5a919688d1ff0da0cc", |
||||
"nLockTime": 0, |
||||
"nVersion": 2 |
||||
} |
||||
``` |
||||
|
||||
@ -0,0 +1,41 @@
|
||||
import hashlib |
||||
from .support import bintohex |
||||
|
||||
def get_pow(data, noncelen=10, hashfn=hashlib.sha512, |
||||
hashlen=64, nbits=1, truncate=0, maxiterations=10**8): |
||||
""" Arguments: |
||||
data - a string of bytes. |
||||
noncelen - an int, the number of additional bytes to be appended |
||||
to the bytestring `data` which will be used for grinding. |
||||
hashfn - a function that outputs a finalized hash state that can |
||||
be converted to a bytestring with .digest() (see hashlib). |
||||
hashlen - the length of the bytestring created with the .digest() |
||||
call just mentioned. |
||||
nbits - an integer, the number of bits of proof of work required. |
||||
truncate - an integer number of bytes to be truncated from the end |
||||
of the hash digest created. |
||||
maxiterations - an integer, how many grinding attempts maximum allowed |
||||
to attempt to reach the target, before giving up. |
||||
Returns: |
||||
(nonceval, pow-preimage, niterations) |
||||
where pow-preimage is data+nonce-in-bytes |
||||
or |
||||
(None, failure-reason, None) |
||||
""" |
||||
maxbits = (hashlen-truncate)*8 |
||||
pow_target = 2 ** (maxbits - nbits) |
||||
# note since we are using a trivial counter, two |
||||
# elements of returned tuple are the same, this needn't be the case. |
||||
for nonceval in range(maxiterations): |
||||
x = data + bintohex(nonceval.to_bytes(noncelen, "big")).encode( |
||||
"utf-8") |
||||
pow_candidate = hashfn(x).digest()[:truncate] |
||||
if int.from_bytes(pow_candidate, "big") < pow_target: |
||||
return (nonceval, x, nonceval) |
||||
return (None, "exceeded max-iterations: {}".format(maxiterations), None) |
||||
|
||||
def verify_pow(data, hashfn=hashlib.sha512, hashlen=64, nbits=1, truncate=0): |
||||
return int.from_bytes(hashfn(data).digest()[:truncate], |
||||
"big") < 2 ** ((hashlen - truncate) * 8 - nbits) |
||||
|
||||
|
||||
@ -0,0 +1,65 @@
|
||||
""" |
||||
Module implementing actions that can be taken on |
||||
network-serialized bitcoin blocks. |
||||
""" |
||||
|
||||
import struct |
||||
from jmbase import hextobin, bintohex |
||||
from bitcointx.core import CBitcoinTransaction |
||||
from bitcointx.core.serialize import VarIntSerializer, VarBytesSerializer |
||||
|
||||
def decode_varint(data): |
||||
n, tail = VarIntSerializer.deserialize_partial(data) |
||||
head = data[0: len(data) - len(tail)] |
||||
return n, len(head) |
||||
|
||||
def get_transactions_in_block(block): |
||||
""" `block` is hex output from RPC `getblock`. |
||||
Return: |
||||
yields the block's transactions, type CBitcoinTransaction |
||||
""" |
||||
block = hextobin(block) |
||||
|
||||
# Skipping the header |
||||
transaction_data = block[80:] |
||||
|
||||
# Decoding the number of transactions, offset is the size of |
||||
# the varint (1 to 9 bytes) |
||||
n_transactions, offset = decode_varint(transaction_data) |
||||
|
||||
for i in range(n_transactions): |
||||
# This 'strat' of reading in small chunks optimistically is taken from: |
||||
# https://github.com/alecalve/python-bitcoin-blockchain-parser/blob/7a9e15c236b10d2a6dff5e696801c0641af72628/blockchain_parser/utils.py |
||||
# Try from 1024 (1KiB) -> 1073741824 (1GiB) slice widths |
||||
for j in range(0, 20): |
||||
try: |
||||
offset_e = offset + (1024 * 2 ** j) |
||||
transaction = CBitcoinTransaction.deserialize( |
||||
transaction_data[offset:offset_e], allow_padding=True) |
||||
yield transaction |
||||
break |
||||
except: |
||||
continue |
||||
|
||||
# Skipping to the next transaction |
||||
offset += len(transaction.serialize()) |
||||
|
||||
""" Example block from a regtest: |
||||
|
||||
# Found using `getblockhash 222` followed by `getblock <resultinghash> 0`: |
||||
|
||||
0000003066327ecf2f3e72ec43f358c9c7b34f47374f23f4fcce965d4e18273a5b98f325d11b3b9c3a592c830d49f6281d4055f5732a79a19f9bd8d4afad729772cbf393fa7bdf5fffff7f2000000000010200000 |
||||
00001010000000000000000000000000000000000000000000000000000000000000000ffffffff0502de000101ffffffff0200f902950000000017a914d2e1d0ea5135f0cbeb4aef06e3cee785d394876a870000000000000000266a24aa21a9ede2f61c3f71d1defd3fa999dfa36953755c690689799962b48bebd836974e8cf90120000000000000000000000000000000000000000000000000000000000000000000000000 |
||||
|
||||
Gives (x = above block): |
||||
|
||||
y = list(btc.get_block_transactions(x)) |
||||
>>> y[0] |
||||
CBitcoinTransaction([CBitcoinTxIn(CBitcoinOutPoint(), CBitcoinScript([x('de00'), x('01')]), 0xffffffff)], |
||||
[CBitcoinTxOut(25.0*COIN, CBitcoinScript([OP_HASH160, x('d2e1d0ea5135f0cbeb4aef06e3cee785d394876a'), OP_EQUAL])), |
||||
CBitcoinTxOut(0.0*COIN, CBitcoinScript([OP_RETURN, x('aa21a9ede2f61c3f71d1defd3fa999dfa36953755c690689799962b48bebd836974e8cf9')]))], |
||||
0, 2, CBitcoinTxWitness([CBitcoinTxInWitness(CScriptWitness([x('0000000000000000000000000000000000000000000000000000000000000000')]))])) |
||||
|
||||
(coinbase transaction) |
||||
""" |
||||
|
||||
@ -0,0 +1,96 @@
|
||||
import pytest |
||||
import copy |
||||
import jmbitcoin as btc |
||||
|
||||
""" Awkward test module name `test_btc_snicker` is to avoid |
||||
conflicts with snicker tests in jmclient. |
||||
""" |
||||
|
||||
@pytest.mark.parametrize( |
||||
"our_input_val, their_input_val, network_fee, script_type, net_transfer", [ |
||||
(24000000, 20000000, 2000, "p2wpkh", 100), |
||||
(124000000, 20000000, 800, "p2wpkh", -100), |
||||
(24000000, 20000000, 2000, "p2sh-p2wpkh", 100), |
||||
(124000000, 20000000, 800, "p2sh-p2wpkh", -100), |
||||
]) |
||||
def test_is_snicker_tx(our_input_val, their_input_val, network_fee, |
||||
script_type, net_transfer): |
||||
our_input = (bytes([1])*32, 0) |
||||
their_input = (bytes([2])*32, 1) |
||||
assert our_input_val - their_input_val - network_fee > 0 |
||||
total_input_amount = our_input_val + their_input_val |
||||
total_output_amount = total_input_amount - network_fee |
||||
receiver_output_amount = their_input_val + net_transfer |
||||
proposer_output_amount = total_output_amount - receiver_output_amount |
||||
|
||||
# all keys are just made up; only the script type will be checked |
||||
privs = [bytes([i])*32 + bytes([1]) for i in range(1,4)] |
||||
pubs = [btc.privkey_to_pubkey(x) for x in privs] |
||||
|
||||
if script_type == "p2wpkh": |
||||
spks = [btc.pubkey_to_p2wpkh_script(x) for x in pubs] |
||||
elif script_type == "p2sh-p2wpkh": |
||||
spks = [btc.pubkey_to_p2sh_p2wpkh_script(x) for x in pubs] |
||||
else: |
||||
assert False |
||||
tweaked_addr, our_addr, change_addr = [str( |
||||
btc.CCoinAddress.from_scriptPubKey(x)) for x in spks] |
||||
# now we must construct the three outputs with correct output amounts. |
||||
outputs = [{"address": tweaked_addr, "value": receiver_output_amount}] |
||||
outputs.append({"address": our_addr, "value": receiver_output_amount}) |
||||
outputs.append({"address": change_addr, |
||||
"value": total_output_amount - 2 * receiver_output_amount}) |
||||
assert all([x["value"] > 0 for x in outputs]) |
||||
|
||||
# make_shuffled_tx mutates ordering (yuck), work with copies only: |
||||
outputs1 = copy.deepcopy(outputs) |
||||
# version and locktime as currently specified in the BIP |
||||
# for 0/1 version SNICKER. (Note the locktime is partly because |
||||
# of expected delays). |
||||
tx = btc.make_shuffled_tx([our_input, their_input], outputs1, |
||||
version=2, locktime=0) |
||||
assert btc.is_snicker_tx(tx) |
||||
|
||||
# construct variants which will be invalid. |
||||
|
||||
# mixed script types in outputs |
||||
wrong_tweaked_spk = btc.pubkey_to_p2pkh_script(pubs[1]) |
||||
wrong_tweaked_addr = str(btc.CCoinAddress.from_scriptPubKey( |
||||
wrong_tweaked_spk)) |
||||
outputs2 = copy.deepcopy(outputs) |
||||
outputs2[0] = {"address": wrong_tweaked_addr, |
||||
"value": receiver_output_amount} |
||||
tx2 = btc.make_shuffled_tx([our_input, their_input], outputs2, |
||||
version=2, locktime=0) |
||||
assert not btc.is_snicker_tx(tx2) |
||||
|
||||
# nonequal output amounts |
||||
outputs3 = copy.deepcopy(outputs) |
||||
outputs3[1] = {"address": our_addr, "value": receiver_output_amount - 1} |
||||
tx3 = btc.make_shuffled_tx([our_input, their_input], outputs3, |
||||
version=2, locktime=0) |
||||
assert not btc.is_snicker_tx(tx3) |
||||
|
||||
# too few outputs |
||||
outputs4 = copy.deepcopy(outputs) |
||||
outputs4 = outputs4[:2] |
||||
tx4 = btc.make_shuffled_tx([our_input, their_input], outputs4, |
||||
version=2, locktime=0) |
||||
assert not btc.is_snicker_tx(tx4) |
||||
|
||||
# too many outputs |
||||
outputs5 = copy.deepcopy(outputs) |
||||
outputs5.append({"address": change_addr, "value": 200000}) |
||||
tx5 = btc.make_shuffled_tx([our_input, their_input], outputs5, |
||||
version=2, locktime=0) |
||||
assert not btc.is_snicker_tx(tx5) |
||||
|
||||
# wrong nVersion |
||||
tx6 = btc.make_shuffled_tx([our_input, their_input], outputs, |
||||
version=1, locktime=0) |
||||
assert not btc.is_snicker_tx(tx6) |
||||
|
||||
# wrong nLockTime |
||||
tx7 = btc.make_shuffled_tx([our_input, their_input], outputs, |
||||
version=2, locktime=1) |
||||
assert not btc.is_snicker_tx(tx7) |
||||
@ -0,0 +1,226 @@
|
||||
#!/usr/bin/env python3 |
||||
|
||||
description="""A rudimentary implementation of creation of a SNICKER proposal. |
||||
|
||||
**THIS TOOL DOES NOT SCAN FOR CANDIDATE TRANSACTIONS** |
||||
|
||||
It only creates proposals on candidate transactions (individually) |
||||
that you have already found. |
||||
|
||||
Input: the user's wallet, mixdepth to source their (1) coin from, |
||||
and a hex encoded pre-existing bitcoin transaction (fully signed) |
||||
as target. |
||||
User chooses the input to source the pubkey from, and the output |
||||
to use to create the SNICKER coinjoin. Tx fees are sourced from |
||||
the config, and the user specifies interactively the number of sats |
||||
to award the receiver (can be negative). |
||||
|
||||
Once the proposal is created, it is uploaded to the servers as per |
||||
the `servers` setting in `joinmarket.cfg`, unless the -n option is |
||||
specified (see help for options), in which case the proposal is |
||||
output to stdout in the same string format: base64proposal,hexpubkey. |
||||
""" |
||||
|
||||
import sys |
||||
from optparse import OptionParser |
||||
from jmbase import BytesProducer, bintohex, jmprint, hextobin, \ |
||||
EXIT_ARGERROR, EXIT_FAILURE, EXIT_SUCCESS, get_pow |
||||
import jmbitcoin as btc |
||||
from jmclient import (RegtestBitcoinCoreInterface, process_shutdown, |
||||
jm_single, load_program_config, check_regtest, select_one_utxo, |
||||
estimate_tx_fee, SNICKERReceiver, add_base_options, get_wallet_path, |
||||
open_test_wallet_maybe, WalletService, SNICKERClientProtocolFactory, |
||||
start_reactor, JMPluginService) |
||||
from jmclient.configure import get_log |
||||
|
||||
log = get_log() |
||||
|
||||
def main(): |
||||
parser = OptionParser( |
||||
usage= |
||||
'usage: %prog [options] walletname hex-tx input-index output-index net-transfer', |
||||
description=description |
||||
) |
||||
add_base_options(parser) |
||||
parser.add_option('-m', |
||||
'--mixdepth', |
||||
action='store', |
||||
type='int', |
||||
dest='mixdepth', |
||||
help='mixdepth/account to spend from, default=0', |
||||
default=0) |
||||
parser.add_option( |
||||
'-g', |
||||
'--gap-limit', |
||||
action='store', |
||||
type='int', |
||||
dest='gaplimit', |
||||
default = 6, |
||||
help='gap limit for Joinmarket wallet, default 6.' |
||||
) |
||||
parser.add_option( |
||||
'-n', |
||||
'--no-upload', |
||||
action='store_true', |
||||
dest='no_upload', |
||||
default=False, |
||||
help="if set, we don't upload the new proposal to the servers" |
||||
) |
||||
parser.add_option( |
||||
'-f', |
||||
'--txfee', |
||||
action='store', |
||||
type='int', |
||||
dest='txfee', |
||||
default=-1, |
||||
help='Bitcoin miner tx_fee to use for transaction(s). A number higher ' |
||||
'than 1000 is used as "satoshi per KB" tx fee. A number lower than that ' |
||||
'uses the dynamic fee estimation of your blockchain provider as ' |
||||
'confirmation target. This temporarily overrides the "tx_fees" setting ' |
||||
'in your joinmarket.cfg. Works the same way as described in it. Check ' |
||||
'it for examples.') |
||||
parser.add_option('-a', |
||||
'--amtmixdepths', |
||||
action='store', |
||||
type='int', |
||||
dest='amtmixdepths', |
||||
help='number of mixdepths in wallet, default 5', |
||||
default=5) |
||||
(options, args) = parser.parse_args() |
||||
snicker_plugin = JMPluginService("SNICKER") |
||||
load_program_config(config_path=options.datadir, |
||||
plugin_services=[snicker_plugin]) |
||||
if len(args) != 5: |
||||
jmprint("Invalid arguments, see --help") |
||||
sys.exit(EXIT_ARGERROR) |
||||
wallet_name, hextx, input_index, output_index, net_transfer = args |
||||
input_index, output_index, net_transfer = [int(x) for x in [ |
||||
input_index, output_index, net_transfer]] |
||||
check_regtest() |
||||
|
||||
# If tx_fees are set manually by CLI argument, override joinmarket.cfg: |
||||
if int(options.txfee) > 0: |
||||
jm_single().config.set("POLICY", "tx_fees", str(options.txfee)) |
||||
max_mix_depth = max([options.mixdepth, options.amtmixdepths - 1]) |
||||
wallet_path = get_wallet_path(wallet_name, None) |
||||
wallet = open_test_wallet_maybe( |
||||
wallet_path, wallet_name, max_mix_depth, |
||||
wallet_password_stdin=options.wallet_password_stdin, |
||||
gap_limit=options.gaplimit) |
||||
wallet_service = WalletService(wallet) |
||||
if wallet_service.rpc_error: |
||||
sys.exit(EXIT_FAILURE) |
||||
snicker_plugin.start_plugin_logging(wallet_service) |
||||
# in this script, we need the wallet synced before |
||||
# logic processing for some paths, so do it now: |
||||
while not wallet_service.synced: |
||||
wallet_service.sync_wallet(fast=not options.recoversync) |
||||
# the sync call here will now be a no-op: |
||||
wallet_service.startService() |
||||
|
||||
# now that the wallet is available, we can construct a proposal |
||||
# before encrypting it: |
||||
originating_tx = btc.CMutableTransaction.deserialize(hextobin(hextx)) |
||||
txid1 = originating_tx.GetTxid()[::-1] |
||||
# the proposer wallet needs to choose a single utxo, from his selected |
||||
# mixdepth, that is bigger than the output amount of tx1 at the given |
||||
# index. |
||||
fee_est = estimate_tx_fee(2, 3, txtype=wallet_service.get_txtype()) |
||||
amt_required = originating_tx.vout[output_index].nValue + fee_est |
||||
|
||||
prop_utxo_dict = wallet_service.select_utxos(options.mixdepth, |
||||
amt_required) |
||||
prop_utxos = list(prop_utxo_dict) |
||||
prop_utxo_vals = [prop_utxo_dict[x] for x in prop_utxos] |
||||
# get the private key for that utxo |
||||
priv = wallet_service.get_key_from_addr( |
||||
wallet_service.script_to_addr(prop_utxo_vals[0]['script'])) |
||||
# construct the arguments for the snicker proposal: |
||||
our_input_utxos = [btc.CMutableTxOut(x['value'], |
||||
x['script']) for x in prop_utxo_vals] |
||||
|
||||
# destination must be a different mixdepth: |
||||
prop_destn_spk = wallet_service.get_new_script(( |
||||
options.mixdepth + 1) % (wallet_service.mixdepth + 1), 1) |
||||
change_spk = wallet_service.get_new_script(options.mixdepth, 1) |
||||
their_input = (txid1, output_index) |
||||
# we also need to extract the pubkey of the chosen input from |
||||
# the witness; we vary this depending on our wallet type: |
||||
pubkey, msg = btc.extract_pubkey_from_witness(originating_tx, input_index) |
||||
if not pubkey: |
||||
log.error("Failed to extract pubkey from transaction: {}".format(msg)) |
||||
sys.exit(EXIT_FAILURE) |
||||
encrypted_proposal = wallet_service.create_snicker_proposal( |
||||
prop_utxos, their_input, |
||||
our_input_utxos, |
||||
originating_tx.vout[output_index], |
||||
net_transfer, |
||||
fee_est, |
||||
priv, |
||||
pubkey, |
||||
prop_destn_spk, |
||||
change_spk, |
||||
version_byte=1) + b"," + bintohex(pubkey).encode('utf-8') |
||||
if options.no_upload: |
||||
jmprint(encrypted_proposal.decode("utf-8")) |
||||
sys.exit(EXIT_SUCCESS) |
||||
|
||||
nodaemon = jm_single().config.getint("DAEMON", "no_daemon") |
||||
daemon = True if nodaemon == 1 else False |
||||
snicker_client = SNICKERPostingClient([encrypted_proposal]) |
||||
servers = jm_single().config.get("SNICKER", "servers").split(",") |
||||
snicker_pf = SNICKERClientProtocolFactory(snicker_client, servers) |
||||
start_reactor(jm_single().config.get("DAEMON", "daemon_host"), |
||||
jm_single().config.getint("DAEMON", "daemon_port"), |
||||
None, snickerfactory=snicker_pf, |
||||
daemon=daemon) |
||||
|
||||
class SNICKERPostingClient(object): |
||||
""" A client object which stores proposals |
||||
ready to be sent to the server/servers, and appends |
||||
proof of work to them according to the server's rules. |
||||
""" |
||||
def __init__(self, pre_nonce_proposals, info_callback=None, |
||||
end_requests_callback=None): |
||||
# the encrypted proposal without the nonce appended for PoW |
||||
self.pre_nonce_proposals = pre_nonce_proposals |
||||
|
||||
self.proposals_with_nonce = [] |
||||
|
||||
# callback for conveying information messages |
||||
if not info_callback: |
||||
self.info_callback = self.default_info_callback |
||||
else: |
||||
self.info_callback = info_callback |
||||
|
||||
# callback for action at the end of a set of |
||||
# submissions to servers; by default, this |
||||
# is "one-shot"; we submit to all servers in the |
||||
# config, then shut down the script. |
||||
if not end_requests_callback: |
||||
self.end_requests_callback = \ |
||||
self.default_end_requests_callback |
||||
|
||||
def default_end_requests_callback(self, response): |
||||
process_shutdown() |
||||
|
||||
def default_info_callback(self, msg): |
||||
jmprint(msg) |
||||
|
||||
def get_proposals(self, targetbits): |
||||
# the data sent to the server is base64encryptedtx,key,nonce; the nonce |
||||
# part is generated in get_pow(). |
||||
for p in self.pre_nonce_proposals: |
||||
nonceval, preimage, niter = get_pow(p+b",", nbits=targetbits, |
||||
truncate=32) |
||||
log.debug("Got POW preimage: {}".format(preimage.decode("utf-8"))) |
||||
if nonceval is None: |
||||
log.error("Failed to generate proof of work, message:{}".format( |
||||
preimage)) |
||||
sys.exit(EXIT_FAILURE) |
||||
self.proposals_with_nonce.append(preimage) |
||||
return self.proposals_with_nonce |
||||
|
||||
if __name__ == "__main__": |
||||
main() |
||||
jmprint('done', "success") |
||||
@ -0,0 +1,89 @@
|
||||
#!/usr/bin/env python3 |
||||
|
||||
from optparse import OptionParser |
||||
import sys |
||||
from jmbase import get_log, jmprint |
||||
from jmclient import (jm_single, load_program_config, WalletService, |
||||
open_test_wallet_maybe, get_wallet_path, |
||||
check_regtest, add_base_options, start_reactor, |
||||
SNICKERClientProtocolFactory, SNICKERReceiver, |
||||
JMPluginService) |
||||
from jmbase.support import EXIT_ARGERROR |
||||
|
||||
jlog = get_log() |
||||
|
||||
def receive_snicker_main(): |
||||
usage = """ Use this script to receive proposals for SNICKER |
||||
coinjoins, parse them and then broadcast coinjoins |
||||
that fit your criteria. See the SNICKER section of |
||||
joinmarket.cfg to set your criteria. |
||||
The only argument to this script is the (JM) wallet |
||||
file against which to check. |
||||
Once all proposals have been parsed, the script will |
||||
quit. |
||||
Usage: %prog [options] wallet file [proposal] |
||||
""" |
||||
parser = OptionParser(usage=usage) |
||||
add_base_options(parser) |
||||
parser.add_option('-g', '--gap-limit', action='store', type="int", |
||||
dest='gaplimit', default=6, |
||||
help='gap limit for wallet, default=6') |
||||
parser.add_option('-m', '--mixdepth', action='store', type='int', |
||||
dest='mixdepth', default=0, |
||||
help="mixdepth to source coins from") |
||||
parser.add_option('-a', |
||||
'--amtmixdepths', |
||||
action='store', |
||||
type='int', |
||||
dest='amtmixdepths', |
||||
help='number of mixdepths in wallet, default 5', |
||||
default=5) |
||||
parser.add_option( |
||||
'-n', |
||||
'--no-upload', |
||||
action='store_true', |
||||
dest='no_upload', |
||||
default=False, |
||||
help="if set, we read the proposal from the command line" |
||||
) |
||||
|
||||
(options, args) = parser.parse_args() |
||||
if len(args) < 1: |
||||
parser.error('Needs a wallet file as argument') |
||||
sys.exit(EXIT_ARGERROR) |
||||
wallet_name = args[0] |
||||
snicker_plugin = JMPluginService("SNICKER") |
||||
load_program_config(config_path=options.datadir, |
||||
plugin_services=[snicker_plugin]) |
||||
|
||||
check_regtest() |
||||
|
||||
wallet_path = get_wallet_path(wallet_name, None) |
||||
max_mix_depth = max([options.mixdepth, options.amtmixdepths - 1]) |
||||
wallet = open_test_wallet_maybe( |
||||
wallet_path, wallet_name, max_mix_depth, |
||||
wallet_password_stdin=options.wallet_password_stdin, |
||||
gap_limit=options.gaplimit) |
||||
wallet_service = WalletService(wallet) |
||||
snicker_plugin.start_plugin_logging(wallet_service) |
||||
while not wallet_service.synced: |
||||
wallet_service.sync_wallet(fast=not options.recoversync) |
||||
wallet_service.startService() |
||||
|
||||
nodaemon = jm_single().config.getint("DAEMON", "no_daemon") |
||||
daemon = True if nodaemon == 1 else False |
||||
snicker_r = SNICKERReceiver(wallet_service) |
||||
if options.no_upload: |
||||
proposal = args[1] |
||||
snicker_r.process_proposals([proposal]) |
||||
return |
||||
servers = jm_single().config.get("SNICKER", "servers").split(",") |
||||
snicker_pf = SNICKERClientProtocolFactory(snicker_r, servers, oneshot=True) |
||||
start_reactor(jm_single().config.get("DAEMON", "daemon_host"), |
||||
jm_single().config.getint("DAEMON", "daemon_port"), |
||||
None, snickerfactory=snicker_pf, |
||||
daemon=daemon) |
||||
|
||||
if __name__ == "__main__": |
||||
receive_snicker_main() |
||||
jmprint('done') |
||||
@ -0,0 +1,115 @@
|
||||
#!/usr/bin/env python3 |
||||
|
||||
description="""Find SNICKER candidate transactions on |
||||
the blockchain. |
||||
|
||||
Using a connection to Bitcoin Core, which allows retrieving |
||||
full blocks, this script will list the transaction IDs of |
||||
transactions that fit the pattern of SNICKER, as codified in |
||||
https://gist.github.com/AdamISZ/2c13fb5819bd469ca318156e2cf25d79 |
||||
and as checked in the `jmbitcoin.snicker` module function |
||||
`is_snicker_tx`, and also optionally, transactions that fit the |
||||
pattern of Joinmarket coinjoins (see -j). |
||||
|
||||
Pass a starting and finishing block value as argument. If the |
||||
finishing block is not provided, it is assumed to be the latest |
||||
block. |
||||
|
||||
**Note that this is slow.** |
||||
|
||||
This script does *NOT* require a wallet, but it does require |
||||
a connection to Core, so does not work with `no-blockchain`. |
||||
Note that this script obviates the need to have txindex enabled |
||||
in Bitcoin Core in order to get full transactions, since it |
||||
parses the raw blocks. |
||||
""" |
||||
|
||||
from optparse import OptionParser |
||||
from jmbase import bintohex, EXIT_ARGERROR, EXIT_FAILURE, jmprint |
||||
import jmbitcoin as btc |
||||
from jmclient import (jm_single, add_base_options, load_program_config, |
||||
check_regtest) |
||||
from jmclient.configure import get_log |
||||
|
||||
log = get_log() |
||||
|
||||
def found_str(ttype, tx, b): |
||||
return "Found {} transaction: {} in block: {}".format( |
||||
ttype, bintohex(tx.GetTxid()[::-1]), b) |
||||
|
||||
def write_candidate_to_file(ttype, candidate, blocknum, unspents, filename): |
||||
""" Appends the details for the candidate |
||||
transaction to the chosen textfile. |
||||
""" |
||||
with open(filename, "a") as f: |
||||
f.write(found_str(ttype, candidate, blocknum) + "\n") |
||||
f.write(btc.human_readable_transaction(candidate)+"\n") |
||||
f.write("Full transaction hex for creating a proposal is " |
||||
"found in the above.\n") |
||||
f.write("The unspent indices are: " + " ".join( |
||||
(str(u) for u in unspents)) + "\n") |
||||
def main(): |
||||
parser = OptionParser( |
||||
usage= |
||||
'usage: %prog [options] startingblock [endingblock]', |
||||
description=description |
||||
) |
||||
add_base_options(parser) |
||||
parser.add_option('-f', |
||||
'--filename', |
||||
action='store', |
||||
type='str', |
||||
dest='candidate_file_name', |
||||
help='filename to write details of candidate ' |
||||
'transactions, default ./candidates.txt', |
||||
default='candidates.txt') |
||||
parser.add_option( |
||||
'-j', |
||||
'--include-jm', |
||||
action='store_true', |
||||
dest='include_joinmarket', |
||||
default=True, |
||||
help="scan for Joinmarket coinjoin outputs, as well as SNICKER.") |
||||
|
||||
(options, args) = parser.parse_args() |
||||
load_program_config(config_path=options.datadir) |
||||
if len(args) not in [1,2]: |
||||
log.error("Invalid arguments, see --help") |
||||
sys.exit(EXIT_ARGERROR) |
||||
|
||||
startblock = int(args[0]) |
||||
if len(args) == 1: |
||||
endblock = jm_single().bc_interface.get_current_block_height() |
||||
else: |
||||
endblock = int(args[1]) |
||||
|
||||
check_regtest() |
||||
|
||||
for b in range(startblock, endblock + 1): |
||||
block = jm_single().bc_interface.get_block(b) |
||||
for t in btc.get_transactions_in_block(block): |
||||
if btc.is_snicker_tx(t): |
||||
log.info(found_str("SNICKER", t, b)) |
||||
# get list of unspent outputs; if empty, skip, |
||||
# otherwise, persist to candidate file with unspents |
||||
# marked. |
||||
unspents = jm_single().bc_interface.get_unspent_indices(t) |
||||
if len(unspents) == 0: |
||||
continue |
||||
write_candidate_to_file("SNICKER", t, b, unspents, |
||||
options.candidate_file_name) |
||||
# note elif avoids wasting computation if we already found SNICKER: |
||||
elif options.include_joinmarket: |
||||
cj_amount, n = btc.is_jm_tx(t) |
||||
# here we don't care about the stats; the tx is printed anyway. |
||||
if cj_amount: |
||||
log.info(found_str("Joinmarket coinjoin", t, b)) |
||||
unspents = jm_single().bc_interface.get_unspent_indices(t) |
||||
if len(unspents) == 0: |
||||
continue |
||||
write_candidate_to_file("Joinmarket coinjoin", t, b, |
||||
unspents, options.candidate_file_name) |
||||
log.info("Finished processing block: {}".format(b)) |
||||
if __name__ == "__main__": |
||||
main() |
||||
jmprint('done', "success") |
||||
@ -0,0 +1,206 @@
|
||||
#!/usr/bin/env python3 |
||||
|
||||
description="""This tool is to be used in a case where |
||||
a user has a BIP39 seedphrase but has no wallet file and no |
||||
backup of imported keys, and they had earlier used SNICKER. |
||||
|
||||
This will usually not be needed as you should keep a backup |
||||
of your *.jmdat joinmarket wallet file, which contains all |
||||
this information. |
||||
|
||||
Before using this tool, you need to do: |
||||
`python wallet-tool.py recover` to recover the wallet from |
||||
seed, and then: |
||||
`bitcoin-cli rescanblockchain ...` |
||||
for an appropriate range of blocks in order for Bitcoin Core |
||||
to get a record of the transactions that happened with your |
||||
HD addresses. |
||||
|
||||
Then, you can run this script to find all the SNICKER-generated |
||||
imported addresses that either did have, or still do have, keys |
||||
and have them imported back into the wallet. |
||||
(Note that this of course won't find any other non-SNICKER imported |
||||
keys, so as a reminder, *always* back up either jmdat wallet files, |
||||
or at least, the imported keys themselves.) |
||||
""" |
||||
|
||||
from optparse import OptionParser |
||||
from jmbase import bintohex, EXIT_ARGERROR, EXIT_FAILURE, jmprint |
||||
import jmbitcoin as btc |
||||
from jmclient import (jm_single, add_base_options, load_program_config, |
||||
check_regtest, get_wallet_path, open_test_wallet_maybe, |
||||
WalletService) |
||||
from jmclient.configure import get_log |
||||
|
||||
log = get_log() |
||||
|
||||
def get_pubs_and_indices_of_inputs(tx, wallet_service, ours): |
||||
""" Returns a list of items (pubkey, index), |
||||
one per input at index index, in transaction |
||||
tx, spending pubkey pubkey, if the input is ours |
||||
if ours is True, else returns the complementary list. |
||||
""" |
||||
our_ins = [] |
||||
not_our_ins = [] |
||||
for i in range(len(tx.vin)): |
||||
pub, msg = btc.extract_pubkey_from_witness(tx, i) |
||||
if not pub: |
||||
continue |
||||
if not wallet_service.is_known_script( |
||||
wallet_service.pubkey_to_script(pub)): |
||||
not_our_ins.append((pub, i)) |
||||
else: |
||||
our_ins.append((pub, i)) |
||||
if ours: |
||||
return our_ins |
||||
else: |
||||
return not_our_ins |
||||
|
||||
def get_pubs_and_indices_of_ancestor_inputs(txin, wallet_service, ours): |
||||
""" For a transaction input txin, retrieve the spent transaction, |
||||
and iterate over its inputs, returning a list of items |
||||
(pubkey, index) all of which belong to us if ours is True, |
||||
or else the complementary set. |
||||
Note: the ancestor transactions must be in the dict txlist, which is |
||||
keyed by txid and values are CTransaction; if not, |
||||
an error occurs. This is assumed to be the case because all ancestors |
||||
must be either in the set returned by wallet_sync, or else in the set |
||||
of SNICKER transactions found so far. |
||||
""" |
||||
tx = wallet_service.get_transaction(txin.prevout.hash[::-1]) |
||||
return get_pubs_and_indices_of_inputs(tx, wallet_service, ours=ours) |
||||
|
||||
def main(): |
||||
parser = OptionParser( |
||||
usage= |
||||
'usage: %prog [options] walletname', |
||||
description=description |
||||
) |
||||
parser.add_option('-m', '--mixdepth', action='store', type='int', |
||||
dest='mixdepth', default=0, |
||||
help="mixdepth to source coins from") |
||||
parser.add_option('-a', |
||||
'--amtmixdepths', |
||||
action='store', |
||||
type='int', |
||||
dest='amtmixdepths', |
||||
help='number of mixdepths in wallet, default 5', |
||||
default=5) |
||||
parser.add_option('-g', |
||||
'--gap-limit', |
||||
type="int", |
||||
action='store', |
||||
dest='gaplimit', |
||||
help='gap limit for wallet, default=6', |
||||
default=6) |
||||
add_base_options(parser) |
||||
(options, args) = parser.parse_args() |
||||
load_program_config(config_path=options.datadir) |
||||
check_regtest() |
||||
if len(args) != 1: |
||||
log.error("Invalid arguments, see --help") |
||||
sys.exit(EXIT_ARGERROR) |
||||
wallet_name = args[0] |
||||
wallet_path = get_wallet_path(wallet_name, None) |
||||
max_mix_depth = max([options.mixdepth, options.amtmixdepths - 1]) |
||||
wallet = open_test_wallet_maybe( |
||||
wallet_path, wallet_name, max_mix_depth, |
||||
wallet_password_stdin=options.wallet_password_stdin, |
||||
gap_limit=options.gaplimit) |
||||
wallet_service = WalletService(wallet) |
||||
|
||||
# step 1: do a full recovery style sync. this will pick up |
||||
# all addresses that we expect to match transactions against, |
||||
# from a blank slate Core wallet that originally had no imports. |
||||
if not options.recoversync: |
||||
jmprint("Recovery sync was not set, but using it anyway.") |
||||
while not wallet_service.synced: |
||||
wallet_service.sync_wallet(fast=False) |
||||
# Note that the user may be interrupted above by the rescan |
||||
# request; this is as for normal scripts; after the rescan is done |
||||
# (usually, only once, but, this *IS* needed here, unlike a normal |
||||
# wallet generation event), we just try again. |
||||
|
||||
# Now all address from HD are imported, we need to grab |
||||
# all the transactions for those addresses; this includes txs |
||||
# that *spend* as well as receive our coins, so will include |
||||
# "first-out" SNICKER txs as well as ordinary spends and JM coinjoins. |
||||
seed_transactions = wallet_service.get_all_transactions() |
||||
|
||||
# Search for SNICKER txs and add them if they match. |
||||
# We proceed recursively; we find all one-out matches, then |
||||
# all 2-out matches, until we find no new ones and stop. |
||||
|
||||
if len(seed_transactions) == 0: |
||||
jmprint("No transactions were found for this wallet. Did you rescan?") |
||||
return False |
||||
|
||||
new_txs = [] |
||||
current_block_heights = set() |
||||
for tx in seed_transactions: |
||||
if btc.is_snicker_tx(tx): |
||||
jmprint("Found a snicker tx: {}".format(bintohex(tx.GetTxid()[::-1]))) |
||||
equal_outs = btc.get_equal_outs(tx) |
||||
if not equal_outs: |
||||
continue |
||||
if all([wallet_service.is_known_script( |
||||
x.scriptPubKey) == False for x in [a[1] for a in equal_outs]]): |
||||
# it is now *very* likely that one of the two equal |
||||
# outputs is our SNICKER custom output |
||||
# script; notice that in this case, the transaction *must* |
||||
# have spent our inputs, since it didn't recognize ownership |
||||
# of either coinjoin output (and if it did recognize the change, |
||||
# it would have recognized the cj output also). |
||||
# We try to regenerate one of the outputs, but warn if |
||||
# we can't. |
||||
my_indices = get_pubs_and_indices_of_inputs(tx, wallet_service, ours=True) |
||||
for mypub, mi in my_indices: |
||||
for eo in equal_outs: |
||||
for (other_pub, i) in get_pubs_and_indices_of_inputs(tx, wallet_service, ours=False): |
||||
for (our_pub, j) in get_pubs_and_indices_of_ancestor_inputs(tx.vin[mi], wallet_service, ours=True): |
||||
our_spk = wallet_service.pubkey_to_script(our_pub) |
||||
our_priv = wallet_service.get_key_from_addr( |
||||
wallet_service.script_to_addr(our_spk)) |
||||
tweak_bytes = btc.ecdh(our_priv[:-1], other_pub) |
||||
tweaked_pub = btc.snicker_pubkey_tweak(our_pub, tweak_bytes) |
||||
tweaked_spk = wallet_service.pubkey_to_script(tweaked_pub) |
||||
if tweaked_spk == eo[1].scriptPubKey: |
||||
# TODO wallet.script_to_addr has a dubious assertion, that's why |
||||
# we use btc method directly: |
||||
address_found = str(btc.CCoinAddress.from_scriptPubKey(btc.CScript(tweaked_spk))) |
||||
#address_found = wallet_service.script_to_addr(tweaked_spk) |
||||
jmprint("Found a new SNICKER output belonging to us.") |
||||
jmprint("Output address {} in the following transaction:".format( |
||||
address_found)) |
||||
jmprint(btc.human_readable_transaction(tx)) |
||||
jmprint("Importing the address into the joinmarket wallet...") |
||||
# NB for a recovery we accept putting any imported keys all into |
||||
# the same mixdepth (0); TODO investigate correcting this, it will |
||||
# be a little complicated. |
||||
success, msg = wallet_service.check_tweak_matches_and_import(wallet_service.script_to_addr(our_spk), |
||||
tweak_bytes, tweaked_pub, wallet_service.mixdepth) |
||||
if not success: |
||||
jmprint("Failed to import SNICKER key: {}".format(msg), "error") |
||||
return False |
||||
else: |
||||
jmprint("... success.") |
||||
# we want the blockheight to track where the next-round rescan |
||||
# must start from |
||||
current_block_heights.add(wallet_service.get_transaction_block_height(tx)) |
||||
# add this transaction to the next round. |
||||
new_txs.append(tx) |
||||
if len(new_txs) == 0: |
||||
return True |
||||
seed_transactions.extend(new_txs) |
||||
earliest_new_blockheight = min(current_block_heights) |
||||
jmprint("New SNICKER addresses were imported to the Core wallet; " |
||||
"do rescanblockchain again, starting from block {}, before " |
||||
"restarting this script.".format(earliest_new_blockheight)) |
||||
return False |
||||
|
||||
if __name__ == "__main__": |
||||
res = main() |
||||
if not res: |
||||
jmprint("Script finished, recovery is NOT complete.", level="warning") |
||||
else: |
||||
jmprint("Script finished, recovery is complete.") |
||||
@ -0,0 +1,213 @@
|
||||
#!/usr/bin/env python3 |
||||
|
||||
description="""Make fake SNICKER transactions to aid discovery. |
||||
|
||||
Use this script to send money to yourself in a transaction which |
||||
fits the format of SNICKER v1 (so it will have two equal sized |
||||
outputs and a change output, also obeying the other minor rules |
||||
for SNICKER). |
||||
|
||||
Having done this your transaction will be picked up by blockchain |
||||
scanners looking for the SNICKER "fingerprint", allowing them |
||||
to propose coinjoins with your coins. |
||||
|
||||
The transaction is generated with at least TWO utxos from your chosen |
||||
source mixdepth/account (-m), so it must contain at least two. |
||||
The reason for using one account, not two, is to prevent violating |
||||
the principle of not co-spending from different accounts; even though |
||||
this is a simulated coinjoin, it may be deducible that it is only really |
||||
a *signalling* fake coinjoin, so it is better not to violate the principle. |
||||
""" |
||||
|
||||
import sys |
||||
import random |
||||
from optparse import OptionParser |
||||
from jmbase import BytesProducer, bintohex, jmprint, hextobin, \ |
||||
EXIT_ARGERROR, EXIT_FAILURE, EXIT_SUCCESS |
||||
import jmbitcoin as btc |
||||
from jmclient import (RegtestBitcoinCoreInterface, process_shutdown, |
||||
jm_single, load_program_config, check_regtest, select_one_utxo, |
||||
estimate_tx_fee, SNICKERReceiver, add_base_options, get_wallet_path, |
||||
open_test_wallet_maybe, WalletService, SNICKERClientProtocolFactory, |
||||
start_reactor, JMPluginService) |
||||
from jmclient.support import select_greedy, NotEnoughFundsException |
||||
from jmclient.configure import get_log |
||||
|
||||
log = get_log() |
||||
|
||||
def main(): |
||||
parser = OptionParser( |
||||
usage= |
||||
'usage: %prog [options] walletname', |
||||
description=description |
||||
) |
||||
add_base_options(parser) |
||||
parser.add_option('-m', |
||||
'--mixdepth', |
||||
action='store', |
||||
type='int', |
||||
dest='mixdepth', |
||||
help='mixdepth/account, default 0', |
||||
default=0) |
||||
parser.add_option( |
||||
'-g', |
||||
'--gap-limit', |
||||
action='store', |
||||
type='int', |
||||
dest='gaplimit', |
||||
default = 6, |
||||
help='gap limit for Joinmarket wallet, default 6.' |
||||
) |
||||
parser.add_option( |
||||
'-f', |
||||
'--txfee', |
||||
action='store', |
||||
type='int', |
||||
dest='txfee', |
||||
default=-1, |
||||
help='Bitcoin miner tx_fee to use for transaction(s). A number higher ' |
||||
'than 1000 is used as "satoshi per KB" tx fee. A number lower than that ' |
||||
'uses the dynamic fee estimation of your blockchain provider as ' |
||||
'confirmation target. This temporarily overrides the "tx_fees" setting ' |
||||
'in your joinmarket.cfg. Works the same way as described in it. Check ' |
||||
'it for examples.') |
||||
parser.add_option('-a', |
||||
'--amtmixdepths', |
||||
action='store', |
||||
type='int', |
||||
dest='amtmixdepths', |
||||
help='number of mixdepths in wallet, default 5', |
||||
default=5) |
||||
parser.add_option('-N', |
||||
'--net-transfer', |
||||
action='store', |
||||
type='int', |
||||
dest='net_transfer', |
||||
help='how many sats are sent to the "receiver", default randomised.', |
||||
default=-1000001) |
||||
(options, args) = parser.parse_args() |
||||
snicker_plugin = JMPluginService("SNICKER") |
||||
load_program_config(config_path=options.datadir, |
||||
plugin_services=[snicker_plugin]) |
||||
if len(args) != 1: |
||||
log.error("Invalid arguments, see --help") |
||||
sys.exit(EXIT_ARGERROR) |
||||
wallet_name = args[0] |
||||
check_regtest() |
||||
# If tx_fees are set manually by CLI argument, override joinmarket.cfg: |
||||
if int(options.txfee) > 0: |
||||
jm_single().config.set("POLICY", "tx_fees", str(options.txfee)) |
||||
max_mix_depth = max([options.mixdepth, options.amtmixdepths - 1]) |
||||
wallet_path = get_wallet_path(wallet_name, None) |
||||
wallet = open_test_wallet_maybe( |
||||
wallet_path, wallet_name, max_mix_depth, |
||||
wallet_password_stdin=options.wallet_password_stdin, |
||||
gap_limit=options.gaplimit) |
||||
wallet_service = WalletService(wallet) |
||||
if wallet_service.rpc_error: |
||||
sys.exit(EXIT_FAILURE) |
||||
snicker_plugin.start_plugin_logging(wallet_service) |
||||
# in this script, we need the wallet synced before |
||||
# logic processing for some paths, so do it now: |
||||
while not wallet_service.synced: |
||||
wallet_service.sync_wallet(fast=not options.recoversync) |
||||
# the sync call here will now be a no-op: |
||||
wallet_service.startService() |
||||
fee_est = estimate_tx_fee(2, 3, txtype=wallet_service.get_txtype()) |
||||
|
||||
# first, order the utxos in the mixepth by size. Then (this is the |
||||
# simplest algorithm; we could be more sophisticated), choose the |
||||
# *second* largest utxo as the receiver utxo; this ensures that we |
||||
# have enough for the proposer to cover. We consume utxos greedily, |
||||
# meaning we'll at least some of the time, be consolidating. |
||||
utxo_dict = wallet_service.get_utxos_by_mixdepth()[options.mixdepth] |
||||
if not len(utxo_dict) >= 2: |
||||
log.error("Cannot create fake SNICKER tx without at least two utxos, quitting") |
||||
sys.exit(EXIT_ARGERROR) |
||||
# sort utxos by size |
||||
sorted_utxos = sorted(list(utxo_dict.keys()), |
||||
key=lambda k: utxo_dict[k]['value'], |
||||
reverse=True) |
||||
# receiver is the second largest: |
||||
receiver_utxo = sorted_utxos[1] |
||||
receiver_utxo_val = utxo_dict[receiver_utxo] |
||||
# gather the other utxos into a list to select from: |
||||
nonreceiver_utxos = [sorted_utxos[0]] + sorted_utxos[2:] |
||||
# get the net transfer in our fake coinjoin: |
||||
if options.net_transfer < -1000001: |
||||
log.error("Net transfer must be greater than negative 1M sats") |
||||
sys.exit(EXIT_ARGERROR) |
||||
if options.net_transfer == -1000001: |
||||
# default; low-ish is more realistic and avoids problems |
||||
# with dusty utxos |
||||
options.net_transfer = random.randint(-1000, 1000) |
||||
|
||||
# select enough to cover: receiver value + fee + transfer + breathing room |
||||
# we select relatively greedily to support consolidation, since |
||||
# this transaction does not pretend to isolate the coins. |
||||
try: |
||||
available = [{'utxo': utxo, 'value': utxo_dict[utxo]["value"]} |
||||
for utxo in nonreceiver_utxos] |
||||
# selection algos return [{"utxo":..,"value":..}]: |
||||
prop_utxos = {x["utxo"] for x in select_greedy(available, |
||||
receiver_utxo_val["value"] + fee_est + options.net_transfer + 1000)} |
||||
prop_utxos = list(prop_utxos) |
||||
prop_utxo_vals = [utxo_dict[prop_utxo] for prop_utxo in prop_utxos] |
||||
except NotEnoughFundsException as e: |
||||
log.error(repr(e)) |
||||
sys.exit(EXIT_FAILURE) |
||||
|
||||
# Due to the fake nature of this transaction, and its distinguishability |
||||
# (not only in trivial output pattern, but also in subset-sum), there |
||||
# is little advantage in making it use different output mixdepths, so |
||||
# here to prevent fragmentation, everything is kept in the same mixdepth. |
||||
receiver_addr, proposer_addr, change_addr = (wallet_service.script_to_addr( |
||||
wallet_service.get_new_script(options.mixdepth, 1)) for _ in range(3)) |
||||
# persist index update: |
||||
wallet_service.save_wallet() |
||||
outputs = btc.construct_snicker_outputs( |
||||
sum([x["value"] for x in prop_utxo_vals]), |
||||
receiver_utxo_val["value"], |
||||
receiver_addr, |
||||
proposer_addr, |
||||
change_addr, |
||||
fee_est, |
||||
options.net_transfer) |
||||
tx = btc.make_shuffled_tx(prop_utxos + [receiver_utxo], |
||||
outputs, |
||||
version=2, |
||||
locktime=0) |
||||
# before signing, check we satisfied the criteria, otherwise |
||||
# this is pointless! |
||||
if not btc.is_snicker_tx(tx): |
||||
log.error("Code error, created non-SNICKER tx, not signing.") |
||||
sys.exit(EXIT_FAILURE) |
||||
|
||||
# sign all inputs |
||||
# scripts: {input_index: (output_script, amount)} |
||||
our_inputs = {} |
||||
for index, ins in enumerate(tx.vin): |
||||
utxo = (ins.prevout.hash[::-1], ins.prevout.n) |
||||
script = utxo_dict[utxo]['script'] |
||||
amount = utxo_dict[utxo]['value'] |
||||
our_inputs[index] = (script, amount) |
||||
success, msg = wallet_service.sign_tx(tx, our_inputs) |
||||
if not success: |
||||
log.error("Failed to sign transaction: " + msg) |
||||
sys.exit(EXIT_FAILURE) |
||||
# TODO condition on automatic brdcst or not |
||||
if not jm_single().bc_interface.pushtx(tx.serialize()): |
||||
# this represents an error about state (or conceivably, |
||||
# an ultra-short window in which the spent utxo was |
||||
# consumed in another transaction), but not really |
||||
# an internal logic error, so we do NOT return False |
||||
log.error("Failed to broadcast fake SNICKER coinjoin: " +\ |
||||
bintohex(tx.GetTxid()[::-1])) |
||||
log.info(btc.human_readable_transaction(tx)) |
||||
sys.exit(EXIT_FAILURE) |
||||
log.info("Successfully broadcast fake SNICKER coinjoin: " +\ |
||||
bintohex(tx.GetTxid()[::-1])) |
||||
|
||||
if __name__ == "__main__": |
||||
main() |
||||
jmprint('done', "success") |
||||
@ -0,0 +1,344 @@
|
||||
#!/usr/bin/env python3 |
||||
|
||||
""" |
||||
A rudimentary implementation of a server, allowing POST of proposals |
||||
in base64 format, with POW attached required, |
||||
and GET of all current proposals, for SNICKER. |
||||
Serves only over Tor onion service. |
||||
For persistent onion services, specify public port, local port and |
||||
hidden service directory: |
||||
|
||||
`python snicker-server.py 80 7080 /my/hiddenservicedir` |
||||
|
||||
... and (a) make sure these settings match those in your Tor config, |
||||
and also (b) note that the hidden service hostname may not be displayed |
||||
if the running user, understandably, do not have permissions to read that |
||||
directory. |
||||
|
||||
If you only want an ephemeral onion service, for testing, just run without |
||||
arguments: |
||||
|
||||
`python snicker-server.py` |
||||
|
||||
""" |
||||
|
||||
from twisted.internet import reactor |
||||
from twisted.internet.defer import Deferred |
||||
from twisted.web.server import Site |
||||
from twisted.web.resource import Resource |
||||
from twisted.internet.endpoints import TCP4ClientEndpoint, UNIXClientEndpoint, serverFromString |
||||
import txtorcon |
||||
import sys |
||||
import base64 |
||||
import json |
||||
import sqlite3 |
||||
import threading |
||||
import hashlib |
||||
from io import BytesIO |
||||
from jmbase import jmprint, hextobin, bintohex, verify_pow |
||||
from jmclient import process_shutdown, jm_single, load_program_config |
||||
from jmclient.configure import get_log |
||||
|
||||
# Note: this is actually a duplication of the |
||||
# string in jmbitcoin.secp256k1_ecies, but this is deliberate, |
||||
# as we want this tool to have no dependency on jmbitcoin. |
||||
ECIES_MAGIC_BYTES = b'BIE1' |
||||
|
||||
log = get_log() |
||||
|
||||
database_file_name = "proposals.db" |
||||
database_table_name = "proposals" |
||||
|
||||
class SNICKERServer(Resource): |
||||
# rudimentary: flat file, TODO location of file |
||||
DATABASE = "snicker-proposals.txt" |
||||
|
||||
def __init__(self): |
||||
self.dblock = threading.Lock() |
||||
self.conn = sqlite3.connect(database_file_name, check_same_thread=False) |
||||
# TODO: ? |
||||
#con.row_factory = dict_factory |
||||
|
||||
self.cursor = self.conn.cursor() |
||||
try: |
||||
self.dblock.acquire(True) |
||||
# note the pubkey is *NOT* a primary key, by |
||||
# design; we need to be able to create multiple |
||||
# proposals against one key. |
||||
self.cursor.execute("CREATE TABLE IF NOT EXISTS {}(" |
||||
"pubkey TEXT NOT NULL, proposal TEXT NOT NULL, " |
||||
"unique (pubkey, proposal));".format(database_table_name)) |
||||
finally: |
||||
self.dblock.release() |
||||
|
||||
# initial PoW setting; todo, change this: |
||||
self.set_pow_target_bits(8) |
||||
self.nonce_length = 10 |
||||
super().__init__() |
||||
|
||||
isLeaf = True |
||||
|
||||
def set_pow_target_bits(self, nbits): |
||||
self.nbits = nbits |
||||
|
||||
def get_pow_target_bits(self): |
||||
return self.nbits |
||||
|
||||
def return_error(self, request, error_meaning, |
||||
error_code="unavailable", http_code=400): |
||||
""" |
||||
We return, to the sender, stringified json in the body as per the above. |
||||
""" |
||||
request.setResponseCode(http_code) |
||||
request.setHeader(b"content-type", b"text/html; charset=utf-8") |
||||
log.debug("Returning an error: " + str( |
||||
error_code) + ": " + str(error_meaning)) |
||||
return json.dumps({"errorCode": error_code, |
||||
"message": error_meaning}).encode("utf-8") |
||||
|
||||
def render_GET(self, request): |
||||
"""GET request to "/" retrieves the entire current data set. |
||||
GET "/target" retrieves the current nbits target for PoW. |
||||
It's intended that proposers request the target in real time |
||||
before each submission, so that the server can dynamically update |
||||
it at any time. |
||||
""" |
||||
log.debug("GET request, path: {}".format(request.path)) |
||||
if request.path == b"/target": |
||||
return self.serve_pow_target(request) |
||||
if request.path != b"/": |
||||
return self.return_error(request, "Invalid request path", |
||||
"invalid-request-path") |
||||
proposals = self.get_all_current_proposals() |
||||
request.setHeader(b"content-length", |
||||
("%d" % len(proposals)).encode("ascii")) |
||||
return proposals.encode("ascii") |
||||
|
||||
def serve_pow_target(self, request): |
||||
targetbits = ("%d" % self.nbits).encode("ascii") |
||||
request.setHeader(b"content-length", |
||||
("%d" % len(targetbits)).encode("ascii")) |
||||
return targetbits |
||||
|
||||
def render_POST(self, request): |
||||
""" An individual proposal may be submitted in base64, with key |
||||
appended after newline separator in hex. |
||||
""" |
||||
log.debug("The server got this POST request: ") |
||||
# unfortunately the twisted Request object is not |
||||
# easily serialized: |
||||
log.debug(request) |
||||
log.debug(request.method) |
||||
log.debug(request.uri) |
||||
log.debug(request.args) |
||||
sender_parameters = request.args |
||||
log.debug(request.path) |
||||
# defer logging of raw request content: |
||||
proposals = request.content |
||||
if not isinstance(proposals, BytesIO): |
||||
return self.return_error(request, "Invalid request format", |
||||
"invalid-request-format") |
||||
proposals = proposals.read() |
||||
# for now, only allowing proposals of form "base64ciphertext,hexkey", |
||||
#newline separated: |
||||
proposals = proposals.split(b"\n") |
||||
log.debug("Client send proposal list of length: " + str( |
||||
len(proposals))) |
||||
accepted_proposals = [] |
||||
for proposal in proposals: |
||||
if len(proposal) == 0: |
||||
continue |
||||
try: |
||||
encryptedtx, key, nonce = proposal.split(b",") |
||||
bin_key = hextobin(key.decode('utf-8')) |
||||
bin_nonce = hextobin(nonce.decode('utf-8')) |
||||
base64.b64decode(encryptedtx) |
||||
except: |
||||
log.warn("This proposal was not accepted: " + proposal.decode( |
||||
"utf-8")) |
||||
# give up immediately in case of format error: |
||||
return self.return_error(request, "Invalid request format", |
||||
"invalid-request-format") |
||||
if not verify_pow(proposal, nbits=self.nbits, truncate=32): |
||||
return self.return_error(request, "Insufficient PoW", |
||||
"insufficient proof of work") |
||||
accepted_proposals.append((key, encryptedtx)) |
||||
|
||||
# the proposals are valid format-wise; add them to the database |
||||
for p in accepted_proposals: |
||||
# note we will ignore errors here and continue; |
||||
# warning will be shown in logs from called fn. |
||||
self.add_proposal(p) |
||||
content = "{} proposals accepted".format(len(accepted_proposals)) |
||||
request.setHeader(b"content-length", ("%d" % len(content)).encode( |
||||
"ascii")) |
||||
return content.encode("ascii") |
||||
|
||||
def add_proposal(self, p): |
||||
proposal_to_add = tuple(x.decode("utf-8") for x in p) |
||||
try: |
||||
self.cursor.execute('INSERT INTO {} VALUES(?, ?);'.format( |
||||
database_table_name),proposal_to_add) |
||||
except sqlite3.Error as e: |
||||
log.warn("Error inserting data into table: {}".format( |
||||
" ".join(e.args))) |
||||
return False |
||||
self.conn.commit() |
||||
return True |
||||
|
||||
def dbquery(self, querystr, params, return_results=False): |
||||
try: |
||||
self.dblock.acquire(True) |
||||
if return_results: |
||||
return self.cursor.execute( |
||||
querystr, params).fetchall() |
||||
self.cursor.execute(querystr, params) |
||||
finally: |
||||
self.dblock.release() |
||||
|
||||
def get_all_keys(self): |
||||
rows = self.dbquery('SELECT DISTINCT pubkey FROM {};'.format( |
||||
database_table_name), (), True) |
||||
if not rows: |
||||
return [] |
||||
return list([x[0] for x in rows]) |
||||
|
||||
@classmethod |
||||
def db_row_to_proposal_string(cls, row): |
||||
assert len(row) == 2 |
||||
key, proposal = row |
||||
return proposal + "," + key |
||||
|
||||
def get_all_current_proposals(self): |
||||
rows = self.dbquery('SELECT * from {};'.format( |
||||
database_table_name), (), True) |
||||
return "\n".join([self.db_row_to_proposal_string(x) for x in rows]) |
||||
|
||||
def get_proposals_for_key(self, key): |
||||
rows = self.dbquery('SELECT proposal FROM {} WHERE pubkey=?'.format( |
||||
database_table_name), (key,), True) |
||||
if not rows: |
||||
return [] |
||||
return rows |
||||
|
||||
class SNICKERServerManager(object): |
||||
|
||||
def __init__(self, port, local_port=None, |
||||
hsdir=None, |
||||
control_port=9051, |
||||
uri_created_callback=None, |
||||
info_callback=None, |
||||
shutdown_callback=None): |
||||
# port is the *public* port, default 80 |
||||
# if local_port is None, we follow the process |
||||
# to create an ephemeral hidden service. |
||||
# if local_port is a valid port, we start the |
||||
# hidden service configured at directory hsdir. |
||||
# In the latter case, note the patch described at |
||||
# https://github.com/meejah/txtorcon/issues/347 is required. |
||||
self.port = port |
||||
self.local_port = local_port |
||||
if self.local_port is not None: |
||||
assert hsdir is not None |
||||
self.hsdir = hsdir |
||||
self.control_port = control_port |
||||
if not uri_created_callback: |
||||
self.uri_created_callback = self.default_info_callback |
||||
else: |
||||
self.uri_created_callback = uri_created_callback |
||||
if not info_callback: |
||||
self.info_callback = self.default_info_callback |
||||
else: |
||||
self.info_callback = info_callback |
||||
|
||||
self.shutdown_callback =shutdown_callback |
||||
|
||||
def default_info_callback(self, msg): |
||||
jmprint(msg) |
||||
|
||||
def start_snicker_server_and_tor(self): |
||||
""" Packages the startup of the receiver side. |
||||
""" |
||||
self.server = SNICKERServer() |
||||
self.site = Site(self.server) |
||||
self.site.displayTracebacks = False |
||||
jmprint("Attempting to start onion service on port: " + str( |
||||
self.port) + " ...") |
||||
self.start_tor() |
||||
|
||||
def setup_failed(self, arg): |
||||
errmsg = "Setup failed: " + str(arg) |
||||
log.error(errmsg) |
||||
self.info_callback(errmsg) |
||||
process_shutdown() |
||||
|
||||
def create_onion_ep(self, t): |
||||
if self.local_port: |
||||
endpointString = "onion:{}:controlPort={}:localPort={}:hiddenServiceDir={}".format( |
||||
self.port, self.control_port,self.local_port, self.hsdir) |
||||
return serverFromString(reactor, endpointString) |
||||
else: |
||||
# ephemeral onion: |
||||
self.tor_connection = t |
||||
return t.create_onion_endpoint(self.port, version=3) |
||||
|
||||
def onion_listen(self, onion_ep): |
||||
return onion_ep.listen(self.site) |
||||
|
||||
def print_host(self, ep): |
||||
""" Callback fired once the HS is available; |
||||
receiver user needs a BIP21 URI to pass to |
||||
the sender: |
||||
""" |
||||
self.info_callback("Your hidden service is available: ") |
||||
# Note that ep,getHost().onion_port must return the same |
||||
# port as we chose in self.port; if not there is an error. |
||||
assert ep.getHost().onion_port == self.port |
||||
self.uri_created_callback(str(ep.getHost().onion_uri)) |
||||
|
||||
def start_tor(self): |
||||
""" This function executes the workflow |
||||
of starting the hidden service. |
||||
""" |
||||
if not self.local_port: |
||||
control_host = jm_single().config.get("PAYJOIN", "tor_control_host") |
||||
control_port = int(jm_single().config.get("PAYJOIN", "tor_control_port")) |
||||
if str(control_host).startswith('unix:'): |
||||
control_endpoint = UNIXClientEndpoint(reactor, control_host[5:]) |
||||
else: |
||||
control_endpoint = TCP4ClientEndpoint(reactor, control_host, control_port) |
||||
d = txtorcon.connect(reactor, control_endpoint) |
||||
d.addCallback(self.create_onion_ep) |
||||
d.addErrback(self.setup_failed) |
||||
else: |
||||
d = Deferred() |
||||
d.callback(None) |
||||
d.addCallback(self.create_onion_ep) |
||||
# TODO: add errbacks to the next two calls in |
||||
# the chain: |
||||
d.addCallback(self.onion_listen) |
||||
d.addCallback(self.print_host) |
||||
|
||||
def shutdown(self): |
||||
self.tor_connection.protocol.transport.loseConnection() |
||||
process_shutdown(self.mode) |
||||
self.info_callback("Hidden service shutdown complete") |
||||
if self.shutdown_callback: |
||||
self.shutdown_callback() |
||||
|
||||
def snicker_server_start(port, local_port=None, hsdir=None): |
||||
ssm = SNICKERServerManager(port, local_port=local_port, hsdir=hsdir) |
||||
ssm.start_snicker_server_and_tor() |
||||
|
||||
if __name__ == "__main__": |
||||
load_program_config(bs="no-blockchain") |
||||
# in testing, we can optionally use ephemeral; |
||||
# in testing or prod we can use persistent: |
||||
if len(sys.argv) < 2: |
||||
snicker_server_start(80) |
||||
else: |
||||
port = int(sys.argv[1]) |
||||
local_port = int(sys.argv[2]) |
||||
hsdir = sys.argv[3] |
||||
snicker_server_start(port, local_port, hsdir) |
||||
reactor.run() |
||||
Loading…
Reference in new issue