Browse Source

Merge #872: Announce and use fidelity bonds

c70b12f For timelock addrs use new pubkey foreach locktime (chris-belcher)
3dc8d86 Fix importprivkey on fidelity bond wallets (chris-belcher)
bbd3d1b Print privacy warning when showing timelocked addr (chris-belcher)
1ea62a7 Fix bug with timelocked addrs in receive payjoin (chris-belcher)
4868343 Fix showutxos wallettool method for fidelity bonds (chris-belcher)
81bade7 Update ygrunner to use fidelity bonds (Adam Gibson)
a3b775b Increase default fee by 4x (chris-belcher)
e970a01 Create release notes section for fidelity bonds (chris-belcher)
6cf4162 Add fidelity bond protocol tests (chris-belcher)
199b571 Consider fidelity bonds when choosing makers (chris-belcher)
97b8b3b Show fidelity bonds on orderbook watch html page (chris-belcher)
7a50c76 Announce yieldgenerator's fidelity bond (chris-belcher)
6b6fc4a Handle fidelity bonds in client-server protocol (chris-belcher)
662f097 Write and update fidelity bond docs (chris-belcher)
eb0a738 Add interest rate option to config file (chris-belcher)
d4b3f70 Enable creation of fidelity bond wallets on cli (chris-belcher)
b9eab6e Increase max locktime of fidelity bond wallets (chris-belcher)
a3b3cd4 Make fidelity bond wallets be native segwit (chris-belcher)
9a372fd Add getblockhash RPC method (chris-belcher)
e6c0847 Add calculate fidelity bond value function + tests (chris-belcher)

Tree-SHA512: d2ffde2b752f66c5fd0d3100670e69d0357fc27bd3c50f9438cef9b261a17b0554c14ee56567ca1980937f889ac8be75ecc96517b1a77c7370ed33f8b5650cc7
master
chris-belcher 4 years ago
parent
commit
d6a6fc3094
No known key found for this signature in database
GPG Key ID: EF734EA677F31129
  1. 268
      docs/fidelity-bonds.md
  2. 19
      docs/release-notes/release-notes-fidelity-bonds.md
  3. 19
      jmbase/jmbase/commands.py
  4. 14
      jmbase/test/test_commands.py
  5. 4
      jmclient/jmclient/__init__.py
  6. 7
      jmclient/jmclient/blockchaininterface.py
  7. 13
      jmclient/jmclient/cli_options.py
  8. 22
      jmclient/jmclient/client_protocol.py
  9. 27
      jmclient/jmclient/configure.py
  10. 11
      jmclient/jmclient/cryptoengine.py
  11. 136
      jmclient/jmclient/fidelity_bond.py
  12. 10
      jmclient/jmclient/maker.py
  13. 21
      jmclient/jmclient/support.py
  14. 50
      jmclient/jmclient/taker.py
  15. 94
      jmclient/jmclient/wallet.py
  16. 25
      jmclient/jmclient/wallet_service.py
  17. 66
      jmclient/jmclient/wallet_utils.py
  18. 51
      jmclient/jmclient/yieldgenerator.py
  19. 11
      jmclient/test/test_client_protocol.py
  20. 2
      jmclient/test/test_coinjoin.py
  21. 7
      jmclient/test/test_support.py
  22. 16
      jmclient/test/test_taker.py
  23. 111
      jmclient/test/test_wallet.py
  24. 1
      jmdaemon/jmdaemon/__init__.py
  25. 40
      jmdaemon/jmdaemon/daemon_protocol.py
  26. 12
      jmdaemon/jmdaemon/fidelity_bond_sanity_check.py
  27. 37
      jmdaemon/jmdaemon/message_channel.py
  28. 25
      jmdaemon/jmdaemon/orderbookwatch.py
  29. 2
      jmdaemon/jmdaemon/protocol.py
  30. 5
      jmdaemon/test/test_daemon_protocol.py
  31. 8
      jmdaemon/test/test_message_channel.py
  32. 238
      jmdaemon/test/test_orderbookwatch.py
  33. 368
      scripts/obwatch/ob-watcher.py
  34. 2
      scripts/obwatch/orderbook.html
  35. 81
      scripts/obwatch/sybil_attack_calculations.py
  36. 19
      test/common.py
  37. 69
      test/ygrunner.py

268
docs/fidelity-bonds.md

@ -3,8 +3,6 @@
Fidelity bonds are a feature of JoinMarket which improves the resistance to
sybil attacks, and therefore improves the privacy of the system.
## This feature is incomplete and so is disabled for now
A fidelity bond is a mechanism where bitcoin value is deliberately sacrificed
to make a cryptographic identity expensive to obtain. The sacrifice is done in
a way that can be proven to a third party. Takers in JoinMarket will
@ -22,12 +20,8 @@ long-term holder (or hodler) of bitcoins can buy time-locked fidelity bonds
essentially for free, assuming they never intended to transact with their coins
anyway.
Another way to create fidelity bonds is to destroy coins by sending them to a
[OP_RETURN](https://en.bitcoin.it/wiki/Script#Provably_Unspendable.2FPrunable_Outputs)
output.
The private keys to fidelity bonds can be kept in [cold storage](https://en.bitcoin.it/wiki/Cold_storage)
for added security.
for added security. (Note: not implemented in JoinMarket)
For a more detailed explanation of how fidelity bonds work see these documents:
@ -36,11 +30,35 @@ bonds](https://gist.github.com/chris-belcher/18ea0e6acdb885a2bfbdee43dcd6b5af/)
* [Financial mathematics of JoinMarket fidelity bonds](https://gist.github.com/chris-belcher/87ebbcbb639686057a389acb9ab3e25b)
#### Note on privacy
## How to use fidelity bonds as a taker
In JoinMarket version v0.9 or later takers will by default use fidelity bonds
without any input needed from the user.
The orderbook watcher script now displays information about any fidelity bonds
advertised by makers, as well as calculating how strong the system is against
hypothetical sybil attackers.
Some makers with high-valued fidelity bonds may choose to ask for a high coinjoin fee, so
for the strongest protection from sybil attacks make sure to set your maximum coinjoin fee
high enough (or if you think the sybil protection is too expensive then set the max fee
lower, as always its your choice as a taker in the market).
Takers will still choose makers equally (i.e. without taking into account fidelity bonds) with a
small probability. By default this probability is 12.5%, so approximately 1-in-8 makers. This can
be changed in the config file with the option `bondless_makers_allowance`.
Bitcoin outputs which create fidelity bonds will be published to the entire
world, so before and after creating them make sure the outputs are not linked
to your identity in any way. Perhaps mix with JoinMarket before and after.
The previous algorithm for choosing makers without regards to fidelity bonds can still be used by
passing the relevant CLI option when running a script (for example
`python3 sendpayment.py -R wallet.jmdat <amount> <address>`). As always use `--help` to get a full
list of options.
## How to use fidelity bonds as a yield-generator
You need to create a fidelity bond wallet and run the yield-generator script on it. The yield
generator will automatically announce the most valuable fidelity bond in its wallet. Fidelity bonds
are only supported for native segwit wallets.
### Creating a JoinMarket wallet which supports fidelity bonds
@ -56,7 +74,7 @@ will offer an option to make the wallet support fidelity bonds.
Would you like this wallet to support fidelity bonds? write 'n' if you don't know what this is (y/n): y
Write down this wallet recovery mnemonic
evidence initial knee image inspire plug dad midnight blast awkward clean between
use amateur twelve unfair weekend file canal frog cotton play renew illegal
Generated wallet OK
@ -65,6 +83,27 @@ as a backup. It is also recommended to write down the name of the creating walle
"JoinMarket" and that the fidelity bond option was enabled. Writing the wallet
creation date is also useful as it can help with rescanning.
#### Adding fidelity bonds to an existing wallet
On any **native segwit** wallet this can be done by using the `recover` method:
(jmvenv) $ python3 wallet-tool.py recover
And then choosing `yes` to create a fidelity bond wallet.
#### Note on privacy
Bitcoin transactions which create fidelity bonds will be published to the entire world, so before
creating them make sure the coins are not linked to any of your privacy-relevant information.
Perhaps mix with JoinMarket. Also, use a sweep transaction which does not create a change output
when funding the timelocked address. Change addresses can also leak privacy information and the
best way to avoid that is to not create change outputs at all i.e. use only sweep transactions.
Once the timelocked addresses expire and become spendable, make sure you don't leak any information
then either, mix afterwards as well. If your timelocked address expires and you want to send the
coins to another timelocked address then you don't need to mix in between, because no
privacy-relevant information linked to you has been leaked.
### Obtaining time-locked addresses
The `wallet-tool.py` script supports a new method `gettimelockaddress` used for
@ -72,13 +111,14 @@ obtaining time-locked addresses. If coins are sent to these addresses they will
be locked up until the timelock date passes. Only mixdepth zero can have
fidelity bonds in it.
This example creates an address which locks any coins sent to it until April 2020.
This example creates an address which locks any coins sent to it until January 2025.
(jmvenv) $ python3 wallet-tool.py testfidelity.jmdat gettimelockaddress 2020-4
(jmvenv) $ python3 wallet-tool.py testfidelity.jmdat gettimelockaddress 2025-1
Enter wallet decryption passphrase:
path = m/49'/1'/0'/2/3:1585699200
Coins sent to this address will be not be spendable until April 2020. Full date: 2020-04-01 00:00:00
bcrt1qrc2qu3m2l2spayu5kr0k0rnn9xgjz46zsxmruh87a3h3f5zmnkaqlfx7v5
path = m/84'/1'/0'/2/0:1748736000
Coins sent to this address will be not be spendable until June 2025. Full date: 2025-06-01 00:00:00
bcrt1qvcjcggpcw2rzk4sks94r3nxj5xhgkqm4p9h54c7mtr695se27efqqxnu0k
If coins are sent to these addresses they will appear in the usual `wallet-tool.py`
display:
@ -86,21 +126,24 @@ display:
(jmvenv) $ python3 wallet-tool.py -m 0 testfidelity.jmdat
Enter wallet decryption passphrase:
JM wallet
mixdepth 0 fbonds-mpk-tpubDDCbCPdf5wJVGYWB4mZr3E3Lys4NBcEKysrrUrLfhG6sekmrvs6KZNe4i5p5z3FyfwRmKMqB9NWEcEUiTS4LwqfrKPQzhKj6aLihu2EejaU
external addresses m/49'/1'/0'/0 tpubDEGdmPwmQRcZmGKhaudjch9Fgw4J5yP4bYw5B8LoSDkMdmhBxM4ndEQXHK4r1TPexGjLidxdpeEzsJcdXEe7khWToxCZuN6JiLzvUoHAki2
m/49'/1'/0'/0/0 2N8jHuQaApgFtQ8UKxKbREAvNxKn4BGX4x2 0.00000000 new
m/49'/1'/0'/0/1 2Mx5CwDoNcuCT38EDmgenQxv9skHbZfXFdo 0.00000000 new
m/49'/1'/0'/0/2 2N1tNTTwNyucGGmfDWNVk3AUi3i5S8jVKqn 0.00000000 new
m/49'/1'/0'/0/3 2N8eBEU5wpWb6kS1gvbRgewtxsmXsMkShV6 0.00000000 new
m/49'/1'/0'/0/4 2MuHgeSgMsvkcn6aGNW2uk2UXP3xVVnkfh2 0.00000000 new
m/49'/1'/0'/0/5 2NA8d8um5KmBNNR8dadhbEDYGiTJPFCdjMB 0.00000000 new
mixdepth 0 fbonds-mpk-tpubDCv7SSisuV4AqNgRqHcXFAeCjV9Z5SPZgSVFjzydEZrS5Jg1uCkv4wGfMZc3wEaiC2hkEfbD753u4R6Shpgj8bR1kuKnEciB522kSpQ3j1v
external addresses m/84'/1'/0'/0 tpubDEdbDAFbNCXXN54M2qgzHBJYxkHK9hoeisyRsC2gC3WZuxziU5RkcJWgpw7nJqugPx26Ui9c2AqCy9gwZpgTtL1GW1TuPtKRX2SdrrjBY2W
m/84'/1'/0'/0/0 bcrt1qwmj5yht2xxr3juxczt453uqjltc3xdyklkjnjt 2.00000000 used
m/84'/1'/0'/0/1 bcrt1q99nzc6s8fh37rjju8gfws4dcfhrpcfz0jst829 0.00000000 new
m/84'/1'/0'/0/2 bcrt1ql3fxzq2ueyhm8kwy7du6nsv7fgpvujupd2emms 0.00000000 new
m/84'/1'/0'/0/3 bcrt1qgxr5mh8v4dj8kuqv98ckjzqdmzd4xjcn539nc6 0.00000000 new
m/84'/1'/0'/0/4 bcrt1qhyhwzkh60p26pk2v008ejqhcl8g70h5vuw2fn6 0.00000000 new
m/84'/1'/0'/0/5 bcrt1q9xuzrqql028wpj3933zyny6geg2u75rhaygv6z 0.00000000 new
m/84'/1'/0'/0/6 bcrt1q8w7ewzl4q8mwxx7erf7pjq36z5g088jxzqjdcn 0.00000000 new
Balance: 2.00000000
internal addresses m/84'/1'/0'/1
Balance: 0.00000000
internal addresses m/49'/1'/0'/1
internal addresses m/84'/1'/0'/2 tpubDEdbDAFbNCXXSFfUKS5QAaxN9toxv8pFSn3TxRdhEij46wj88RCch7ZBA2fgqsocD7MZqowVAdm6LyYumKuKZbzT4V2CfudwDicrMnqqbjC
m/84'/1'/0'/2/0:1748736000 bcrt1qvcjcggpcw2rzk4sks94r3nxj5xhgkqm4p9h54c7mtr695se27efqqxnu0k 2.00000000 2025-06-01 [LOCKED]
Balance: 2.00000000
internal addresses m/84'/1'/0'/3 tpubDEdbDAFbNCXXUpShWMdtPMDcAogMyZJVAzMMn3wM9rC364sdeUuFMS7ZmdjoMbkf14jeK56uy95UBR2SA9AFFeoVv4j4CqMeaq1tcuBVkZe
Balance: 0.00000000
internal addresses m/49'/1'/0'/2 tpubDEGdmPwmQRcZrzjRmUFqXXyLdRedwxCWQviAFqDe6sXJeZzRNTwmwqMfxN6Ka3v7hEebstrU5kqUNoHsFKaA3RoB2vopL6kLHVo1EQq6USw
m/49'/1'/0'/2/0:1585699200 bcrt1qrc2qu3m2l2spayu5kr0k0rnn9xgjz46zsxmruh87a3h3f5zmnkaqlfx7v5 0.15000000 2020-04-01 [LOCKED]
Balance: 0.15000000
Balance for mixdepth 0: 0.15000000
Balance for mixdepth 0: 4.00000000
#### Spending time-locked coins
@ -111,10 +154,117 @@ JoinMarket's coin control feature, so before spending you need to unfreeze the
coins using `python3 wallet-tool.py <walletname> -m 0 freeze`.
Once unfrozen and untimelocked the coins can be spent normally with the scripts
`sendpayment.py`, `tumber.py`, or yield generator.
`sendpayment.py`, `tumber.py`, or yield generator. NB You cannot export the private keys (which is
always disadvised, anyway) of timelocked addresses to any other wallets, as they use custom scripts.
You must spend them from JoinMarket itself.
### How many coins to lock up and for how long?
Fidelity bonds UTXOs are valuable as soon as they confirmed. The simplified formula for a fidelity
bond value with locked coins is:
bond_value = (locked_coins * exp(interest_rate * locktime))^2
A few important things to notice:
* The bond value goes as the _square_ of sacrificed value. For example if your sacrificed value is
5 BTC then the fidelity bond value is 25 (because 5 x 5 = 25). If instead you sacrificed 6 BTC the
value is 36 (because 6 x 6 = 36). The point of this is to create an incentive for makers to lump
all their coins into just one bot rather than spreading it over many bots. It makes a sybil attack
much more expensive.
* The longer you lock for the greater the value. The value increases as the `interest_rate`, which
is configurable in the config file with the option `interest_rate`. By default it is 1.5% per
annum and because of tyranny-of-the-default takers are unlikely to change it. This value is probably
not too far from the "real" interest rate, and the system still works fine even if the real rate
is something like 3% or 0.1%.
* The above formula would suggest that if you lock 3 BTC for 10000 years you get a fidelity
bond worth `1.7481837557171304e+131` (17 followed by 130 zeros). This does not happen because the
sacrificed value is capped at the value of the burned coins. So in this example the fidelity bond
value would be just 9 (equal to 3x3 or 3 squared). This feature is not included in the above
simplified equation.
* After the locktime expires and the coins are free to move, the fidelity bond will continue to be
valuable, but its value will exponentially drop following the interest rate. So it would be good
for you as a yield generator to create a transaction with the UTXO spending it to another
time-locked address, but it's not a huge rush (specifically, there's likely no need to pay massive
miner fees, you can probably wait until fees are low).
The full details on valuing a time-locked fidelity bond are [found in the relevant section of the
"Financial mathematics of fidelity bonds" document](https://gist.github.com/chris-belcher/87ebbcbb639686057a389acb9ab3e25b#time-locked-fidelity-bonds).
At any time you can use the orderbook watcher script to see your own fidelity bond value.
Consider also the [warning on the bitcoin wiki page on timelocks](https://en.bitcoin.it/wiki/Timelock#Far-future_locks).
I would recommend locking as many coins as you are comfortable with for a period of between 6
months and 2 years. Perhaps at the very start lock for only 1 month or 2 months(?) It's a
marketplace and the rules are known to all, so ultimately you'll have to make your own decision.
### Can I add coins to a fidelity bond that already exists?
No. Sending more coins to the same timelocked address will not add to the existing fidelity bond,
but instead create a new one with the same locktime.
### Creating watch-only fidelity bond wallets
#### Note: Fidelity bond in cold storage cannot be advertised to takers right now. You can create watch-only fidelity bond wallets but cant advertise them yet. This feature is pretty easy to add though, and can be done without changing the JoinMarket protocol.
Fidelity bonds can be held on an offline computer in
[cold storage](https://en.bitcoin.it/wiki/Cold_storage). To do this we create
a watch-only fidelity bond wallet.
When fidelity bonds are displayed in `wallet-tool.py`, their master public key
is highlighted with a prefix `fbonds-mpk-`.
This master public key can be used to create a watch-only wallet using
`wallet-tool.py`.
$ python3 wallet-tool.py createwatchonly fbonds-mpk-tpubDDCbCPdf5wJVGYWB4mZr3E3Lys4NBcEKysrrUrLfhG6sekmrvs6KZNe4i5p5z3FyfwRmKMqB9NWEcEUiTS4LwqfrKPQzhKj6aLihu2EejaU
Input wallet file name (default: watchonly.jmdat): watchfidelity.jmdat
Enter wallet file encryption passphrase:
Reenter wallet file encryption passphrase:
Done
Then the wallet can be displayed like a regular wallet, although only the zeroth
mixdepth will be shown.
$ python3 wallet-tool.py watchfidelity.jmdat
User data location: .
Enter wallet decryption passphrase:
JM wallet
mixdepth 0 fbonds-mpk-tpubDCv7SSisuV4AqNgRqHcXFAeCjV9Z5SPZgSVFjzydEZrS5Jg1uCkv4wGfMZc3wEaiC2hkEfbD753u4R6Shpgj8bR1kuKnEciB522kSpQ3j1v
external addresses m/84'/1'/0'/0 tpubDEdbDAFbNCXXN54M2qgzHBJYxkHK9hoeisyRsC2gC3WZuxziU5RkcJWgpw7nJqugPx26Ui9c2AqCy9gwZpgTtL1GW1TuPtKRX2SdrrjBY2W
m/84'/1'/0'/0/0 bcrt1qwmj5yht2xxr3juxczt453uqjltc3xdyklkjnjt 2.00000000 used
m/84'/1'/0'/0/1 bcrt1q99nzc6s8fh37rjju8gfws4dcfhrpcfz0jst829 0.00000000 new
m/84'/1'/0'/0/2 bcrt1ql3fxzq2ueyhm8kwy7du6nsv7fgpvujupd2emms 0.00000000 new
m/84'/1'/0'/0/3 bcrt1qgxr5mh8v4dj8kuqv98ckjzqdmzd4xjcn539nc6 0.00000000 new
m/84'/1'/0'/0/4 bcrt1qhyhwzkh60p26pk2v008ejqhcl8g70h5vuw2fn6 0.00000000 new
m/84'/1'/0'/0/5 bcrt1q9xuzrqql028wpj3933zyny6geg2u75rhaygv6z 0.00000000 new
m/84'/1'/0'/0/6 bcrt1q8w7ewzl4q8mwxx7erf7pjq36z5g088jxzqjdcn 0.00000000 new
Balance: 2.00000000
internal addresses m/84'/1'/0'/1
Balance: 0.00000000
internal addresses m/84'/1'/0'/2 tpubDEdbDAFbNCXXSFfUKS5QAaxN9toxv8pFSn3TxRdhEij46wj88RCch7ZBA2fgqsocD7MZqowVAdm6LyYumKuKZbzT4V2CfudwDicrMnqqbjC
m/84'/1'/0'/2/0:1748736000 bcrt1qvcjcggpcw2rzk4sks94r3nxj5xhgkqm4p9h54c7mtr695se27efqqxnu0k 2.00000000 2025-06-01 [LOCKED]
Balance: 2.00000000
internal addresses m/84'/1'/0'/3 tpubDEdbDAFbNCXXUpShWMdtPMDcAogMyZJVAzMMn3wM9rC364sdeUuFMS7ZmdjoMbkf14jeK56uy95UBR2SA9AFFeoVv4j4CqMeaq1tcuBVkZe
Balance: 0.00000000
Balance for mixdepth 0: 4.00000000
### BIP32 Paths
Fidelity bond wallets extend the BIP32 path format to include the locktime
values. In this example we've got `m/49'/1'/0'/2/0:1583020800` where the
number after the colon is the locktime value in Unix time.
This path can be passed to certain wallet methods like `dumpprivkey`.
$ python3 wallet-tool.py -H "m/49'/1'/0'/2/0:1583020800" testfidelity.jmdat dumpprivkey
Enter wallet decryption passphrase:
cNEuE5ypNTxVFCyC5iH7u5AQTrddamcUHRPNweiLvmHUWd6XXDkz
### Burning coins
#### Note: There is no point using this feature. Fidelity bonds in JoinMarket cannot be created by burning coins right now. This feature is here only for historical reasons.
Coins can be burned with a special method of the `sendpayment.py` script. Set
the destination to be `BURN`. Transactions which burn coins must only have one
input and one output, so use coin control to freeze all coins in the zeroth
@ -210,59 +360,3 @@ The `-H` flag must point to the path containing the burn output.
Then synchronizing the wallet won't output the no-merkle-proof warning.
### Creating watch-only fidelity bond wallets
Fidelity bonds can be held on an offline computer in
[cold storage](https://en.bitcoin.it/wiki/Cold_storage). To do this we create
a watch-only fidelity bond wallet.
When fidelity bonds are displayed in `wallet-tool.py`, their master public key
is highlighted with a prefix `fbonds-mpk-`.
This master public key can be used to create a watch-only wallet using
`wallet-tool.py`.
$ python3 wallet-tool.py createwatchonly fbonds-mpk-tpubDDCbCPdf5wJVGYWB4mZr3E3Lys4NBcEKysrrUrLfhG6sekmrvs6KZNe4i5p5z3FyfwRmKMqB9NWEcEUiTS4LwqfrKPQzhKj6aLihu2EejaU
Input wallet file name (default: watchonly.jmdat): watchfidelity.jmdat
Enter wallet file encryption passphrase:
Reenter wallet file encryption passphrase:
Done
Then the wallet can be displayed like a regular wallet, although only the zeroth
mixdepth will be shown.
$ python3 wallet-tool.py watchfidelity.jmdat
User data location: .
Enter wallet decryption passphrase:
JM wallet
mixdepth 0 fbonds-mpk-tpubDDCbCPdf5wJVGYWB4mZr3E3Lys4NBcEKysrrUrLfhG6sekmrvs6KZNe4i5p5z3FyfwRmKMqB9NWEcEUiTS4LwqfrKPQzhKj6aLihu2EejaU
external addresses m/49'/1'/0'/0 tpubDEGdmPwmQRcZmGKhaudjch9Fgw4J5yP4bYw5B8LoSDkMdmhBxM4ndEQXHK4r1TPexGjLidxdpeEzsJcdXEe7khWToxCZuN6JiLzvUoHAki2
m/49'/1'/0'/0/0 2N8jHuQaApgFtQ8UKxKbREAvNxKn4BGX4x2 0.00000000 used
m/49'/1'/0'/0/1 2Mx5CwDoNcuCT38EDmgenQxv9skHbZfXFdo 0.00000000 new
m/49'/1'/0'/0/2 2N1tNTTwNyucGGmfDWNVk3AUi3i5S8jVKqn 0.00000000 new
m/49'/1'/0'/0/3 2N8eBEU5wpWb6kS1gvbRgewtxsmXsMkShV6 0.00000000 new
m/49'/1'/0'/0/4 2MuHgeSgMsvkcn6aGNW2uk2UXP3xVVnkfh2 0.00000000 new
m/49'/1'/0'/0/5 2NA8d8um5KmBNNR8dadhbEDYGiTJPFCdjMB 0.00000000 new
m/49'/1'/0'/0/6 2NG76BAHPccfyy6sH68EHrB9QJBycx3FKb6 0.00000000 new
Balance: 0.25000000
internal addresses m/49'/1'/0'/1
Balance: 0.00000000
internal addresses m/49'/1'/0'/2 tpubDEGdmPwmQRcZrzjRmUFqXXyLdRedwxCWQviAFqDe6sXJeZzRNTwmwqMfxN6Ka3v7hEebstrU5kqUNoHsFKaA3RoB2vopL6kLHVo1EQq6USw
m/49'/1'/0'/0/3:1585699200 bcrt1qrc2qu3m2l2spayu5kr0k0rnn9xgjz46zsxmruh87a3h3f5zmnkaqlfx7v5 0.15000000 2020-04-01 [UNLOCKED]
Balance: 0.15000000
internal addresses m/49'/1'/0'/3 tpubDEGdmPwmQRcZuX3uNrCouu5bRgp2GJcoQTvhkFAJMTA3yxhKmQyeGwecbnkms4DYmBhCJn2fGTuejTe3g8oyJW3qKcfB4b3Swj2hDk1h4Y2
Balance: 0.00000000
Balance for mixdepth 0: 0.15000000
### BIP32 Paths
Fidelity bond wallets extend the BIP32 path format to include the locktime
values. In this example we've got `m/49'/1'/0'/2/0:1583020800` where the
number after the colon is the locktime value in Unix time.
This path can be passed to certain wallet methods like `dumpprivkey`.
$ python3 wallet-tool.py -H "m/49'/1'/0'/2/0:1583020800" testfidelity.jmdat dumpprivkey
Enter wallet decryption passphrase:
cNEuE5ypNTxVFCyC5iH7u5AQTrddamcUHRPNweiLvmHUWd6XXDkz

19
docs/release-notes/release-notes-fidelity-bonds.md

@ -0,0 +1,19 @@
Notable changes
===============
### Fidelity bond for improving sybil attack resistance
From the very beginning of JoinMarket it was possible to attack the system by creating many many maker bots all controlled by the same person. If an unlucky taker came along and created a coinjoin only with those fake maker bots then their coinjoins could be easily unmixed. This is called a sybil attack and until now it was relatively cheap to do against JoinMarket. Some yield generators were already doing this by running multiple bots, because they could earn higher coinjoin fees from their multiple makers.
Fidelity bonds are a new feature intended to make this sybil attack a lot more expensive. It works by allowing JoinMarket makers to lock up bitcoins into time locked addresses. Takers will still choose makers to coinjoin with randomly but they have a greater chance of choosing makers who have advertised more valuable fidelity bonds. Any sybil attacker then has to lock up many many bitcoins into time locked addresses.
For full details of the scheme see: [Design for improving JoinMarket's resistance to sybil attacks using fidelity bonds](https://gist.github.com/chris-belcher/18ea0e6acdb885a2bfbdee43dcd6b5af/)
This release implements all the features needed to add fidelity bonds to JoinMarket. Takers (via scripts such as `sendpayment.py` or `tumbler.py` or the Joinmarket-Qt app) will automatically give preference to makers who advertise fidelity bonds. Makers can optionally update their wallets to fidelity bond wallets. When a fidelity bond wallet is used with a yield generator script, it will automatically announce its fidelity bond publicly. Makers who don't create fidelity bonds by locking up bitcoins will still be chosen for coinjoins occasionally, but probably much less often than before.
For full user documentation see the file `/docs/`fidelity-bonds.md` in the repository.
With realistic assumptions we have calculated that an adversary would need to lock up around 50000 bitcoins for 6 months in order to sybil attack the JoinMarket system with 95% success rate. Now that fidelity bonds are being added to JoinMarket for real we can see how the system behaves in practice.
Fidelity bond coins cannot be yet be held in cold storage, but this is easy to add later because the JoinMarket protocol is set up in a way that the change would be backward-compatible.

19
jmbase/jmbase/commands.py

@ -6,6 +6,7 @@ Used for AMP asynchronous messages.
from twisted.protocols.amp import Boolean, Command, Integer, Unicode
from .bigstring import BigUnicode
class DaemonNotReady(Exception):
pass
@ -45,7 +46,8 @@ class JMSetup(JMCommand):
role, passes initial offers for announcement (for TAKER, this data is "none")
"""
arguments = [(b'role', Unicode()),
(b'initdata', Unicode())]
(b'offers', Unicode()),
(b'use_fidelity_bond', Boolean())]
class JMMsgSignature(JMCommand):
"""A response to a request for a bitcoin signature
@ -107,6 +109,11 @@ class JMAnnounceOffers(JMCommand):
(b'to_cancel', Unicode()),
(b'offerlist', Unicode())]
class JMFidelityBondProof(JMCommand):
"""Send requested fidelity bond proof message"""
arguments = [(b'nick', Unicode()),
(b'proof', Unicode())]
class JMIOAuth(JMCommand):
"""Send contents of !ioauth message after
verifying Taker's auth message
@ -182,7 +189,8 @@ class JMOffers(JMCommand):
"""Return the entire contents of the
orderbook to TAKER, as a json-ified dict.
"""
arguments = [(b'orderbook', BigUnicode())]
arguments = [(b'orderbook', BigUnicode()),
(b'fidelitybonds', BigUnicode())]
class JMFillResponse(JMCommand):
"""Returns ioauth data from MAKER if successful.
@ -200,6 +208,11 @@ class JMSigReceived(JMCommand):
"""MAKER-specific commands
"""
class JMFidelityBondProofRequest(JMCommand):
"""MAKER wants to announce a fidelity bond proof message"""
arguments = [(b'takernick', Unicode()),
(b'makernick', Unicode())]
class JMAuthReceived(JMCommand):
"""Return the commitment and revelation
provided in !fill, !auth by the TAKER,
@ -376,4 +389,4 @@ class BIP78InfoMsg(JMCommand):
from the daemon about current status at
network level.
"""
arguments = [(b'infomsg', Unicode())]
arguments = [(b'infomsg', Unicode())]

14
jmbase/test/test_commands.py

@ -63,8 +63,8 @@ class JMTestServerProtocol(JMBaseProtocol):
return {'accepted': True}
@JMSetup.responder
def on_JM_SETUP(self, role, initdata):
show_receipt("JMSETUP", role, initdata)
def on_JM_SETUP(self, role, offers, use_fidelity_bond):
show_receipt("JMSETUP", role, offers, use_fidelity_bond)
d = self.callRemote(JMSetupDone)
self.defaultCallbacks(d)
return {'accepted': True}
@ -75,7 +75,8 @@ class JMTestServerProtocol(JMBaseProtocol):
#build a huge orderbook to test BigString Argument
orderbook = ["aaaa" for _ in range(2**15)]
d = self.callRemote(JMOffers,
orderbook=json.dumps(orderbook))
orderbook=json.dumps(orderbook),
fidelitybonds="dummyfidelitybonds")
self.defaultCallbacks(d)
return {'accepted': True}
@ -156,7 +157,8 @@ class JMTestClientProtocol(JMBaseProtocol):
show_receipt("JMUP")
d = self.callRemote(JMSetup,
role="TAKER",
initdata="none")
offers="{}",
use_fidelity_bond=False)
self.defaultCallbacks(d)
return {'accepted': True}
@ -177,8 +179,8 @@ class JMTestClientProtocol(JMBaseProtocol):
return {'accepted': True}
@JMOffers.responder
def on_JM_OFFERS(self, orderbook):
show_receipt("JMOFFERS", orderbook)
def on_JM_OFFERS(self, orderbook, fidelitybonds):
show_receipt("JMOFFERS", orderbook, fidelitybonds)
d = self.callRemote(JMFill,
amount=100,
commitment="dummycommitment",

4
jmclient/jmclient/__init__.py

@ -14,7 +14,7 @@ from .taker import Taker
from .wallet import (Mnemonic, estimate_tx_fee, WalletError, BaseWallet, ImportWalletMixin,
BIP39WalletMixin, BIP32Wallet, BIP49Wallet, LegacyWallet,
SegwitWallet, SegwitLegacyWallet, FidelityBondMixin,
FidelityBondWatchonlyWallet, SegwitLegacyWalletFidelityBonds,
FidelityBondWatchonlyWallet, SegwitWalletFidelityBonds,
UTXOManager, WALLET_IMPLEMENTATIONS, compute_tx_locktime)
from .storage import (Argon2Hash, Storage, StorageError, RetryableStorageError,
StoragePasswordError, VolatileStorage)
@ -24,7 +24,7 @@ from .configure import (load_test_config, process_shutdown,
load_program_config, jm_single, get_network, update_persist_config,
validate_address, is_burn_destination, get_irc_mchannels,
get_blockchain_interface_instance, set_config, is_segwit_mode,
is_native_segwit_mode, JMPluginService)
is_native_segwit_mode, JMPluginService, get_interest_rate, get_bondless_makers_allowance)
from .blockchaininterface import (BlockchainInterface,
RegtestBitcoinCoreInterface, BitcoinCoreInterface)
from .snicker_receiver import SNICKERError, SNICKERReceiver

7
jmclient/jmclient/blockchaininterface.py

@ -190,7 +190,7 @@ class BitcoinCoreInterface(BlockchainInterface):
def get_block(self, blockheight):
"""Returns full serialized block at a given height.
"""
block_hash = self._rpc('getblockhash', [blockheight])
block_hash = self.get_block_hash(blockheight)
block = self._rpc('getblock', [block_hash, False])
if not block:
return False
@ -503,6 +503,9 @@ class BitcoinCoreInterface(BlockchainInterface):
def get_block_time(self, blockhash):
return self._get_block_header_data(blockhash, 'time')
def get_block_hash(self, height):
return self._rpc("getblockhash", [height])
def get_tx_merkle_branch(self, txid, blockhash=None):
if not blockhash:
tx = self._rpc("gettransaction", [txid])
@ -522,7 +525,7 @@ class BitcoinCoreInterface(BlockchainInterface):
return core_proof[80:]
def verify_tx_merkle_branch(self, txid, block_height, merkle_branch):
block_hash = self._rpc("getblockhash", [block_height])
block_hash = self.get_block_hash(block_height)
core_proof = self._rpc("getblockheader", [block_hash, False]) + \
binascii.hexlify(merkle_branch).decode()
ret = self._rpc("verifytxoutproof", [core_proof])

13
jmclient/jmclient/cli_options.py

@ -16,7 +16,8 @@ options which are common to more than one script in a base class.
order_choose_algorithms = {
'random_under_max_order_choose': '-R',
'cheapest_order_choose': '-C',
'weighted_order_choose': '-W'
'weighted_order_choose': '-W',
'fidelity_bond_weighted_order_choose': '-F'
}
def add_base_options(parser):
@ -90,12 +91,12 @@ def add_common_options(parser):
'--order-choose-algorithm',
action='callback',
type='string',
default=jmclient.support.random_under_max_order_choose,
default=jmclient.support.fidelity_bond_weighted_order_choose,
callback=get_order_choose_algorithm,
help="Set the algorithm to use for selecting orders from the order book.\n"
"Default: {}\n"
"Available options: {}"
.format('random_under_max_order_choose',
.format('fidelity_bond_weighted_order_choose',
', '.join(order_choose_algorithms.keys())),
dest='order_choose_fn')
add_order_choose_short_options(parser)
@ -130,9 +131,9 @@ def get_order_choose_algorithm(option, opt_str, value, parser, value_kw=None):
The following defaults are maintained as accessed via functions for
flexibility.
TODO This should be moved from this module."""
MAX_DEFAULT_REL_FEE = 0.001
MIN_MAX_DEFAULT_ABS_FEE = 1000
MAX_MAX_DEFAULT_ABS_FEE = 10000
MAX_DEFAULT_REL_FEE = 0.004
MIN_MAX_DEFAULT_ABS_FEE = 4000
MAX_MAX_DEFAULT_ABS_FEE = 40000
def get_default_max_relative_fee():
return MAX_DEFAULT_REL_FEE

22
jmclient/jmclient/client_protocol.py

@ -394,7 +394,8 @@ class JMMakerClientProtocol(JMClientProtocol):
self.offers_ready_loop.stop()
d = self.callRemote(commands.JMSetup,
role="MAKER",
initdata=json.dumps(self.client.offerlist))
offers=json.dumps(self.client.offerlist),
use_fidelity_bond=(self.client.fidelity_bond is not None))
self.defaultCallbacks(d)
@commands.JMSetupDone.responder
@ -427,6 +428,17 @@ class JMMakerClientProtocol(JMClientProtocol):
maker_timeout_sec=maker_timeout_sec)
self.defaultCallbacks(d)
@commands.JMFidelityBondProofRequest.responder
def on_JM_FIDELITY_BOND_PROOF_REQUEST(self, takernick, makernick):
proof_msg = (self.client.fidelity_bond
.create_proof(makernick, takernick)
.create_proof_msg(self.client.fidelity_bond.cert_privkey))
d = self.callRemote(commands.JMFidelityBondProof,
nick=takernick,
proof=proof_msg)
self.defaultCallbacks(d)
return {"accepted": True}
@commands.JMAuthReceived.responder
def on_JM_AUTH_RECEIVED(self, nick, offer, commitment, revelation, amount,
kphex):
@ -627,7 +639,8 @@ class JMTakerClientProtocol(JMClientProtocol):
def on_JM_UP(self):
d = self.callRemote(commands.JMSetup,
role="TAKER",
initdata="none")
offers="{}",
use_fidelity_bond=False)
self.defaultCallbacks(d)
return {'accepted': True}
@ -676,11 +689,12 @@ class JMTakerClientProtocol(JMClientProtocol):
return {'accepted': True}
@commands.JMOffers.responder
def on_JM_OFFERS(self, orderbook):
def on_JM_OFFERS(self, orderbook, fidelitybonds):
self.orderbook = json.loads(orderbook)
fidelity_bonds_list = json.loads(fidelitybonds)
#Removed for now, as judged too large, even for DEBUG:
#jlog.debug("Got the orderbook: " + str(self.orderbook))
retval = self.client.initialize(self.orderbook)
retval = self.client.initialize(self.orderbook, fidelity_bonds_list)
#format of retval is:
#True, self.cjamount, commitment, revelation, self.filtered_orderbook)
if not retval[0]:

27
jmclient/jmclient/configure.py

@ -88,6 +88,10 @@ required_options = {'BLOCKCHAIN': ['blockchain_source', 'network'],
'POLICY': ['absurd_fee_per_kb', 'taker_utxo_retries',
'taker_utxo_age', 'taker_utxo_amtpercent']}
_DEFAULT_INTEREST_RATE = "0.015"
_DEFAULT_BONDLESS_MAKERS_ALLOWANCE = "0.125"
defaultconfig = \
"""
[DAEMON]
@ -291,6 +295,21 @@ minimum_makers = 4
# whatever the value of it, and this is set with the value -1.
max_sats_freeze_reuse = -1
# Interest rate used when calculating the value of fidelity bonds created
# by locking bitcoins in timelocked addresses
# See also:
# https://gist.github.com/chris-belcher/87ebbcbb639686057a389acb9ab3e25b#determining-interest-rate-r
# Set as a real number, i.e. 1 = 100% and 0.01 = 1%
interest_rate = """ + _DEFAULT_INTEREST_RATE + """
# Some makers run their bots to mix their funds not just to earn money
# So to improve privacy very slightly takers dont always choose a maker based
# on his fidelity bond but allow a certain small percentage to be chosen completely
# randomly without taking into account fidelity bonds
# This parameter sets how many makers on average will be chosen regardless of bonds
# A real number, i.e. 1 = 100%, 0.125 = 1/8 = 1 in every 8 makers on average will be bondless
bondless_makers_allowance = """ + _DEFAULT_BONDLESS_MAKERS_ALLOWANCE + """
##############################
#THE FOLLOWING SETTINGS ARE REQUIRED TO DEFEND AGAINST SNOOPERS.
#DON'T ALTER THEM UNLESS YOU UNDERSTAND THE IMPLICATIONS.
@ -538,6 +557,14 @@ _BURN_DESTINATION = "BURN"
def is_burn_destination(destination):
return destination == _BURN_DESTINATION
def get_interest_rate():
return float(global_singleton.config.get('POLICY', 'interest_rate',
fallback=_DEFAULT_INTEREST_RATE))
def get_bondless_makers_allowance():
return float(global_singleton.config.get('POLICY', 'bondless_makers_allowance',
fallback=_DEFAULT_BONDLESS_MAKERS_ALLOWANCE))
def remove_unwanted_default_settings(config):
for section in config.sections():
if section.startswith('MESSAGING:'):

11
jmclient/jmclient/cryptoengine.py

@ -12,8 +12,8 @@ from .configure import get_network, jm_single
#with fidelity bond wallets and watchonly fidelity bond wallet, the wallet class
# can have two engines, one for single-sig addresses and the other for timelocked addresses
TYPE_P2PKH, TYPE_P2SH_P2WPKH, TYPE_P2WPKH, TYPE_P2SH_M_N, TYPE_TIMELOCK_P2WSH, \
TYPE_SEGWIT_LEGACY_WALLET_FIDELITY_BONDS, TYPE_WATCHONLY_FIDELITY_BONDS, \
TYPE_WATCHONLY_TIMELOCK_P2WSH, TYPE_WATCHONLY_P2SH_P2WPKH = range(9)
TYPE_SEGWIT_WALLET_FIDELITY_BONDS, TYPE_WATCHONLY_FIDELITY_BONDS, \
TYPE_WATCHONLY_TIMELOCK_P2WSH, TYPE_WATCHONLY_P2WPKH = range(9)
NET_MAINNET, NET_TESTNET, NET_SIGNET = range(3)
NET_MAP = {'mainnet': NET_MAINNET, 'testnet': NET_TESTNET,
'signet': NET_SIGNET}
@ -407,7 +407,7 @@ class BTC_Watchonly_Timelocked_P2WSH(BTC_Timelocked_P2WSH):
hashcode=btc.SIGHASH_ALL, **kwargs):
raise RuntimeError("Cannot spend from watch-only wallets")
class BTC_Watchonly_P2SH_P2WPKH(BTC_P2SH_P2WPKH):
class BTC_Watchonly_P2WPKH(BTC_P2WPKH):
@classmethod
def derive_bip32_privkey(cls, master_key, path):
@ -426,7 +426,7 @@ class BTC_Watchonly_P2SH_P2WPKH(BTC_P2SH_P2WPKH):
@classmethod
def derive_bip32_pub_export(cls, master_key, path):
return super(BTC_Watchonly_P2SH_P2WPKH, cls).derive_bip32_pub_export(
return super(BTC_Watchonly_P2WPKH, cls).derive_bip32_pub_export(
master_key, BTC_Watchonly_Timelocked_P2WSH.get_watchonly_path(path))
@classmethod
@ -440,5 +440,6 @@ ENGINES = {
TYPE_P2WPKH: BTC_P2WPKH,
TYPE_TIMELOCK_P2WSH: BTC_Timelocked_P2WSH,
TYPE_WATCHONLY_TIMELOCK_P2WSH: BTC_Watchonly_Timelocked_P2WSH,
TYPE_WATCHONLY_P2SH_P2WPKH: BTC_Watchonly_P2SH_P2WPKH
TYPE_WATCHONLY_P2WPKH: BTC_Watchonly_P2WPKH,
TYPE_SEGWIT_WALLET_FIDELITY_BONDS: BTC_P2WPKH
}

136
jmclient/jmclient/fidelity_bond.py

@ -0,0 +1,136 @@
import struct
import base64
import json
from jmbitcoin import ecdsa_sign, ecdsa_verify
from jmdaemon import fidelity_bond_sanity_check
def assert_is_utxo(utxo):
assert len(utxo) == 2
assert isinstance(utxo[0], bytes)
assert len(utxo[0]) == 32
assert isinstance(utxo[1], int)
assert utxo[1] >= 0
def get_cert_msg(cert_pub, cert_expiry):
return b'fidelity-bond-cert|' + cert_pub + b'|' + str(cert_expiry).encode('ascii')
class FidelityBond:
def __init__(self, utxo, utxo_pubkey, locktime, cert_expiry,
cert_privkey, cert_pubkey, cert_signature):
assert_is_utxo(utxo)
assert isinstance(utxo_pubkey, bytes)
assert isinstance(locktime, int)
assert isinstance(cert_expiry, int)
assert isinstance(cert_privkey, bytes)
assert isinstance(cert_pubkey, bytes)
assert isinstance(cert_signature, bytes)
self.utxo = utxo
self.utxo_pubkey = utxo_pubkey
self.locktime = locktime
self.cert_expiry = cert_expiry
self.cert_privkey = cert_privkey
self.cert_pubkey = cert_pubkey
self.cert_signature = cert_signature
def create_proof(self, maker_nick, taker_nick):
return FidelityBondProof(
maker_nick, taker_nick, self.cert_pubkey, self.cert_expiry,
self.cert_signature, self.utxo, self.utxo_pubkey, self.locktime)
def serialize(self):
return json.dumps([
self.utxo,
self.utxo_pubkey,
self.locktime,
self.cert_expiry,
self.cert_privkey,
self.cert_pubkey,
self.cert_signature,
])
@classmethod
def deserialize(cls, data):
return cls(*json.loads(data))
class FidelityBondProof:
# nick_sig + cert_sig + cert_pubkey + cert_expiry + utxo_pubkey + txid + vout + timelock
# 72 + 72 + 33 + 2 + 33 + 32 + 4 + 4 = 252 bytes
SER_STUCT_FMT = '<72s72s33sH33s32sII'
def __init__(self, maker_nick, taker_nick, cert_pub, cert_expiry,
cert_sig, utxo, utxo_pub, locktime):
assert isinstance(maker_nick, str)
assert isinstance(taker_nick, str)
assert isinstance(cert_pub, bytes)
assert isinstance(cert_sig, bytes)
assert isinstance(utxo_pub, bytes)
assert isinstance(locktime, int)
assert_is_utxo(utxo)
self.maker_nick = maker_nick
self.taker_nick = taker_nick
self.cert_pub = cert_pub
self.cert_expiry = cert_expiry
self.cert_sig = cert_sig
self.utxo = utxo
self.utxo_pub = utxo_pub
self.locktime = locktime
@property
def nick_msg(self):
return (self.taker_nick + '|' + self.maker_nick).encode('ascii')
def create_proof_msg(self, cert_priv):
nick_sig = ecdsa_sign(self.nick_msg, cert_priv)
# FIXME: remove stupid base64
nick_sig = base64.b64decode(nick_sig)
return self._serialize_proof_msg(nick_sig)
def _serialize_proof_msg(self, msg_signature):
msg_signature = msg_signature.rjust(72, b'\xff')
cert_sig = self.cert_sig.rjust(72, b'\xff')
fidelity_bond_data = struct.pack(
self.SER_STUCT_FMT,
msg_signature,
cert_sig,
self.cert_pub,
self.cert_expiry,
self.utxo_pub,
self.utxo[0],
self.utxo[1],
self.locktime
)
return base64.b64encode(fidelity_bond_data).decode('ascii')
@staticmethod
def _verify_signature(message, signature, pubkey):
# FIXME: remove stupid base64
return ecdsa_verify(message, base64.b64encode(signature), pubkey)
@classmethod
def parse_and_verify_proof_msg(cls, maker_nick, taker_nick, data):
if not fidelity_bond_sanity_check.fidelity_bond_sanity_check(data):
raise ValueError("sanity check failed")
decoded_data = base64.b64decode(data)
unpacked_data = struct.unpack(cls.SER_STUCT_FMT, decoded_data)
try:
signature = unpacked_data[0][unpacked_data[0].index(b'\x30'):]
cert_sig = unpacked_data[1][unpacked_data[1].index(b'\x30'):]
except ValueError:
#raised if index() doesnt find the position
raise ValueError("der signature header not found")
proof = cls(maker_nick, taker_nick, unpacked_data[2], unpacked_data[3],
cert_sig, (unpacked_data[5], unpacked_data[6]),
unpacked_data[4], unpacked_data[7])
cert_msg = get_cert_msg(proof.cert_pub, proof.cert_expiry)
if not cls._verify_signature(proof.nick_msg, signature, proof.cert_pub):
raise ValueError("nick sig does not verify")
if not cls._verify_signature(cert_msg, proof.cert_sig, proof.utxo_pub):
raise ValueError("cert sig does not verify")
return proof

10
jmclient/jmclient/maker.py

@ -21,6 +21,7 @@ class Maker(object):
self.wallet_service = wallet_service
self.nextoid = -1
self.offerlist = None
self.fidelity_bond = None
self.sync_wait_loop = task.LoopingCall(self.try_to_create_my_orders)
# don't fire on the first tick since reactor is still starting up
# and may not shutdown appropriately if we immediately recognize
@ -38,6 +39,7 @@ class Maker(object):
if not self.wallet_service.synced:
return
self.offerlist = self.create_my_orders()
self.fidelity_bond = self.get_fidelity_bond_template()
self.sync_wait_loop.stop()
if not self.offerlist:
jlog.info("Failed to create offers, giving up.")
@ -277,3 +279,11 @@ class Maker(object):
"""Performs actions on receipt of 1st confirmation of
a transaction into a block (e.g. announce orders)
"""
def get_fidelity_bond_template(self):
"""
Generates information about a fidelity bond which will be announced
By default returns no fidelity bond
Does not contain nick signature which has to be calculated individually
"""
return None

21
jmclient/jmclient/support.py

@ -2,6 +2,7 @@ from functools import reduce
import random
from jmbase.support import get_log
from decimal import Decimal
from .configure import get_bondless_makers_allowance
from math import exp
@ -218,6 +219,26 @@ def cheapest_order_choose(orders, n):
"""
return orders[0]
def fidelity_bond_weighted_order_choose(orders, n):
"""
choose orders based on fidelity bond for improved sybil resistance
* with probability `bondless_makers_allowance`: will revert to previous default
order choose (random_under_max_order_choose)
* with probability `1 - bondless_makers_allowance`: if there are no bond offerings, revert
to previous default as above. If there are, choose randomly from those, with weighting
being the fidelity bond values.
"""
if random.random() < get_bondless_makers_allowance():
return random_under_max_order_choose(orders, n)
#remove orders without fidelity bonds
filtered_orders = list(filter(lambda x: x[0]["fidelity_bond_value"] != 0, orders))
if len(filtered_orders) == 0:
return random_under_max_order_choose(orders, n)
weights = list(map(lambda x: x[0]["fidelity_bond_value"], filtered_orders))
weights = [x / sum(weights) for x in weights]
return filtered_orders[rand_weighted_choice(len(filtered_orders), weights)]
def _get_is_within_max_limits(max_fee_rel, max_fee_abs, cjvalue):
def check_max_fee(fee):

50
jmclient/jmclient/taker.py

@ -7,13 +7,14 @@ from typing import Any, NamedTuple
from twisted.internet import reactor, task
import jmbitcoin as btc
from jmclient.configure import jm_single, validate_address
from jmclient.configure import jm_single, validate_address, get_interest_rate
from jmbase import get_log, bintohex, hexbin
from jmclient.support import (calc_cj_fee, weighted_order_choose, choose_orders,
choose_sweep_orders)
from jmclient.wallet import estimate_tx_fee, compute_tx_locktime
from jmclient.wallet import estimate_tx_fee, compute_tx_locktime, FidelityBondMixin
from jmclient.podle import generate_podle, get_podle_commitments
from jmclient.wallet_service import WalletService
from jmclient.fidelity_bond import FidelityBondProof
from .output import generate_podle_error_string
from .cryptoengine import EngineError
from .schedule import NO_ROUNDING
@ -166,7 +167,7 @@ class Taker(object):
return
self.honest_only = truefalse
def initialize(self, orderbook):
def initialize(self, orderbook, fidelity_bonds_info):
"""Once the daemon is active and has returned the current orderbook,
select offers, re-initialize variables and prepare a commitment,
then send it to the protocol to fill offers.
@ -227,6 +228,11 @@ class Taker(object):
self.latest_tx = None
self.txid = None
fidelity_bond_values = calculate_fidelity_bond_values(fidelity_bonds_info)
for offer in orderbook:
#having no fidelity bond is like having a zero value fidelity bond
offer["fidelity_bond_value"] = fidelity_bond_values.get(offer["counterparty"], 0)
sweep = True if self.cjamount == 0 else False
if not self.filter_orderbook(orderbook, sweep):
return (False,)
@ -987,3 +993,41 @@ def round_to_significant_figures(d, sf):
sigfiged = int(round(d/power10*sf_power10)*power10/sf_power10)
return sigfiged
raise RuntimeError()
def calculate_fidelity_bond_values(fidelity_bonds_info):
if len(fidelity_bonds_info) == 0:
return {}
interest_rate = get_interest_rate()
blocks = jm_single().bc_interface.get_current_block_height()
mediantime = jm_single().bc_interface.get_best_block_median_time()
validated_bonds = {}
for bond_data in fidelity_bonds_info:
try:
fb_proof = FidelityBondProof.parse_and_verify_proof_msg(
bond_data["counterparty"], bond_data["takernick"], bond_data["proof"])
except ValueError:
continue
if fb_proof.utxo in validated_bonds:
continue
utxo_data = FidelityBondMixin.get_validated_timelocked_fidelity_bond_utxo(
fb_proof.utxo, fb_proof.utxo_pub, fb_proof.locktime,
fb_proof.cert_expiry, blocks)
if utxo_data is not None:
validated_bonds[fb_proof.utxo] = (fb_proof, utxo_data)
fidelity_bond_values = {
bond_data.maker_nick:
FidelityBondMixin.calculate_timelocked_fidelity_bond_value(
utxo_data["value"],
jm_single().bc_interface.get_block_time(
jm_single().bc_interface.get_block_hash(
blocks - utxo_data["confirms"] + 1
)
),
bond_data.locktime,
mediantime,
interest_rate)
for bond_data, utxo_data in validated_bonds.values()
}
return fidelity_bond_values

94
jmclient/jmclient/wallet.py

@ -17,6 +17,7 @@ from hashlib import sha256
from itertools import chain
from decimal import Decimal
from numbers import Integral
from math import exp
from .configure import jm_single
@ -24,8 +25,8 @@ from .blockchaininterface import INF_HEIGHT
from .support import select_gradual, select_greedy, select_greediest, \
select
from .cryptoengine import TYPE_P2PKH, TYPE_P2SH_P2WPKH,\
TYPE_P2WPKH, TYPE_TIMELOCK_P2WSH, TYPE_SEGWIT_LEGACY_WALLET_FIDELITY_BONDS,\
TYPE_WATCHONLY_FIDELITY_BONDS, TYPE_WATCHONLY_TIMELOCK_P2WSH, TYPE_WATCHONLY_P2SH_P2WPKH,\
TYPE_P2WPKH, TYPE_TIMELOCK_P2WSH, TYPE_SEGWIT_WALLET_FIDELITY_BONDS,\
TYPE_WATCHONLY_FIDELITY_BONDS, TYPE_WATCHONLY_TIMELOCK_P2WSH, TYPE_WATCHONLY_P2WPKH,\
ENGINES
from .support import get_random_bytes
from . import mn_encode, mn_decode
@ -404,10 +405,10 @@ class BaseWallet(object):
"""
if self.TYPE == TYPE_P2PKH:
return 'p2pkh'
elif self.TYPE in (TYPE_P2SH_P2WPKH,
TYPE_SEGWIT_LEGACY_WALLET_FIDELITY_BONDS):
elif self.TYPE == TYPE_P2SH_P2WPKH:
return 'p2sh-p2wpkh'
elif self.TYPE == TYPE_P2WPKH:
elif self.TYPE in (TYPE_P2WPKH,
TYPE_SEGWIT_WALLET_FIDELITY_BONDS):
return 'p2wpkh'
assert False
@ -1249,7 +1250,11 @@ class PSBTWalletMixin(object):
privkeys = []
for k, v in self._utxos._utxo.items():
for k2, v2 in v.items():
privkeys.append(self._get_key_from_path(v2[0]))
key = self._get_key_from_path(v2[0])
if FidelityBondMixin.is_timelocked_path(v2[0]) and len(key[0]) == 2:
#key is ((privkey, locktime), engine) for timelocked addrs
key = (key[0][0], key[1])
privkeys.append(key)
jmckeys = list(btc.JMCKey(x[0][:-1]) for x in privkeys)
new_keystore = btc.KeyStore.from_iterable(jmckeys)
@ -2177,7 +2182,7 @@ class FidelityBondMixin(object):
For example, if TIMENUMBER_UNIT = 2 (i.e. every time number is two months)
then there are 6 timelocks per year so just 600 possible
addresses per century per pubkey. Easily searchable when recovering a
addresses per century. Easily searchable when recovering a
wallet from seed phrase. Therefore the user doesn't need to store any
dates, the seed phrase is sufficent for recovery.
"""
@ -2190,16 +2195,8 @@ class FidelityBondMixin(object):
TIMELOCK_EPOCH_MONTH = 1 #january
MONTHS_IN_YEAR = 12
TIMELOCK_ERA_YEARS = 30
TIMENUMBERS_PER_PUBKEY = TIMELOCK_ERA_YEARS * MONTHS_IN_YEAR // TIMENUMBER_UNIT
"""
As each pubkey corresponds to hundreds of addresses, to reduce load the
given gap limit will be reduced by this factor. Also these timelocked
addresses are never handed out to takers so there wont be a problem of
having many used addresses with no transactions on them.
"""
TIMELOCK_GAP_LIMIT_REDUCTION_FACTOR = 6
TIMELOCK_ERA_YEARS = 80
TIMENUMBER_COUNT = TIMELOCK_ERA_YEARS * MONTHS_IN_YEAR // TIMENUMBER_UNIT
_TIMELOCK_ENGINE = ENGINES[TYPE_TIMELOCK_P2WSH]
@ -2217,7 +2214,7 @@ class FidelityBondMixin(object):
"""
converts a time number to a unix timestamp
"""
if not 0 <= timenumber < cls.TIMENUMBERS_PER_PUBKEY:
if not 0 <= timenumber < cls.TIMENUMBER_COUNT:
raise ValueError()
year = cls.TIMELOCK_EPOCH_YEAR + (timenumber*cls.TIMENUMBER_UNIT) // cls.MONTHS_IN_YEAR
month = cls.TIMELOCK_EPOCH_MONTH + (timenumber*cls.TIMENUMBER_UNIT) % cls.MONTHS_IN_YEAR
@ -2238,7 +2235,7 @@ class FidelityBondMixin(object):
raise ValueError()
timenumber = (dt.year - cls.TIMELOCK_EPOCH_YEAR)*(cls.MONTHS_IN_YEAR //
cls.TIMENUMBER_UNIT) + ((dt.month - cls.TIMELOCK_EPOCH_MONTH) // cls.TIMENUMBER_UNIT)
if timenumber < 0 or timenumber > cls.TIMENUMBERS_PER_PUBKEY:
if timenumber < 0 or timenumber > cls.TIMENUMBER_COUNT:
raise ValueError("datetime out of range")
return timenumber
@ -2261,13 +2258,12 @@ class FidelityBondMixin(object):
def _populate_script_map(self):
super()._populate_script_map()
for md in self._index_cache:
address_type = self.BIP32_TIMELOCK_ID
for i in range(self._index_cache[md][address_type]):
for timenumber in range(self.TIMENUMBERS_PER_PUBKEY):
path = self.get_path(md, address_type, i, timenumber)
script = self.get_script_from_path(path)
self._script_map[script] = path
md = self.FIDELITY_BOND_MIXDEPTH
address_type = self.BIP32_TIMELOCK_ID
for timenumber in range(self.TIMENUMBER_COUNT):
path = self.get_path(md, address_type, timenumber, timenumber)
script = self.get_script_from_path(path)
self._script_map[script] = path
def add_utxo(self, txid, index, script, value, height=None):
super().add_utxo(txid, index, script, value, height)
@ -2380,6 +2376,42 @@ class FidelityBondMixin(object):
self._storage.data[self._BURNER_OUTPUT_STORAGE_KEY][path][2] = \
merkle_branch
@classmethod
def calculate_timelocked_fidelity_bond_value(cls, utxo_value, confirmation_time, locktime,
current_time, interest_rate):
"""
utxo_value is in satoshi
interest rate is per year
all times are seconds
"""
YEAR = 60 * 60 * 24 * 365.2425 #gregorian calender year length
r = interest_rate
T = (locktime - confirmation_time) / YEAR
L = locktime / YEAR
t = current_time / YEAR
a = max(0, min(1, exp(r*T) - 1) - min(1, exp(r*max(0, t-L)) - 1))
return utxo_value*utxo_value*a*a
@classmethod
def get_validated_timelocked_fidelity_bond_utxo(cls, utxo, utxo_pubkey, locktime,
cert_expiry, current_block_height):
utxo_data = jm_single().bc_interface.query_utxo_set(utxo, includeconf=True)
if utxo_data[0] == None:
return None
if utxo_data[0]["confirms"] <= 0:
return None
RETARGET_INTERVAL = 2016
if current_block_height > cert_expiry*RETARGET_INTERVAL:
return None
implied_spk = btc.redeem_script_to_p2wsh_script(btc.mk_freeze_script(utxo_pubkey, locktime))
if utxo_data[0]["script"] != implied_spk:
return None
return utxo_data[0]
class BIP49Wallet(BIP32PurposedWallet):
_PURPOSE = 2**31 + 49
_ENGINE = ENGINES[TYPE_P2SH_P2WPKH]
@ -2394,13 +2426,13 @@ class SegwitLegacyWallet(ImportWalletMixin, BIP39WalletMixin, PSBTWalletMixin, S
class SegwitWallet(ImportWalletMixin, BIP39WalletMixin, PSBTWalletMixin, SNICKERWalletMixin, BIP84Wallet):
TYPE = TYPE_P2WPKH
class SegwitLegacyWalletFidelityBonds(FidelityBondMixin, SegwitLegacyWallet):
TYPE = TYPE_SEGWIT_LEGACY_WALLET_FIDELITY_BONDS
class SegwitWalletFidelityBonds(FidelityBondMixin, SegwitWallet):
TYPE = TYPE_SEGWIT_WALLET_FIDELITY_BONDS
class FidelityBondWatchonlyWallet(FidelityBondMixin, BIP49Wallet):
class FidelityBondWatchonlyWallet(FidelityBondMixin, BIP84Wallet):
TYPE = TYPE_WATCHONLY_FIDELITY_BONDS
_ENGINE = ENGINES[TYPE_WATCHONLY_P2SH_P2WPKH]
_ENGINE = ENGINES[TYPE_WATCHONLY_P2WPKH]
_TIMELOCK_ENGINE = ENGINES[TYPE_WATCHONLY_TIMELOCK_P2WSH]
@classmethod
@ -2420,6 +2452,6 @@ WALLET_IMPLEMENTATIONS = {
LegacyWallet.TYPE: LegacyWallet,
SegwitLegacyWallet.TYPE: SegwitLegacyWallet,
SegwitWallet.TYPE: SegwitWallet,
SegwitLegacyWalletFidelityBonds.TYPE: SegwitLegacyWalletFidelityBonds,
SegwitWalletFidelityBonds.TYPE: SegwitWalletFidelityBonds,
FidelityBondWatchonlyWallet.TYPE: FidelityBondWatchonlyWallet
}

25
jmclient/jmclient/wallet_service.py

@ -923,18 +923,8 @@ class WalletService(Service):
if isinstance(self.wallet, FidelityBondMixin):
md = FidelityBondMixin.FIDELITY_BOND_MIXDEPTH
address_type = FidelityBondMixin.BIP32_TIMELOCK_ID
saved_indices[md] += [0]
next_unused = self.get_next_unused_index(md, address_type)
for index in range(next_unused):
for timenumber in range(FidelityBondMixin.TIMENUMBERS_PER_PUBKEY):
addresses.add(self.get_addr(md, address_type, index, timenumber))
for index in range(self.gap_limit // FidelityBondMixin.TIMELOCK_GAP_LIMIT_REDUCTION_FACTOR):
index += next_unused
assert self.wallet.get_index_cache_and_increment(md, address_type) == index
for timenumber in range(FidelityBondMixin.TIMENUMBERS_PER_PUBKEY):
self.wallet.get_script_and_update_map(md, address_type, index, timenumber)
addresses.add(self.get_addr(md, address_type, index, timenumber))
self.wallet.set_next_index(md, address_type, next_unused)
for timenumber in range(FidelityBondMixin.TIMENUMBER_COUNT):
addresses.add(self.get_addr(md, address_type, timenumber, timenumber))
return addresses, saved_indices
@ -950,17 +940,6 @@ class WalletService(Service):
addresses.add(self.get_new_addr(md, address_type))
self.set_next_index(md, address_type, old_next)
if isinstance(self.wallet, FidelityBondMixin):
md = FidelityBondMixin.FIDELITY_BOND_MIXDEPTH
address_type = FidelityBondMixin.BIP32_TIMELOCK_ID
old_next = self.get_next_unused_index(md, address_type)
for ii in range(gap_limit // FidelityBondMixin.TIMELOCK_GAP_LIMIT_REDUCTION_FACTOR):
index = self.wallet.get_index_cache_and_increment(md, address_type)
for timenumber in range(FidelityBondMixin.TIMENUMBERS_PER_PUBKEY):
self.wallet.get_script_and_update_map(md, address_type, index, timenumber)
addresses.add(self.get_addr(md, address_type, index, timenumber))
self.set_next_index(md, address_type, old_next)
return addresses
def get_external_addr(self, mixdepth):

66
jmclient/jmclient/wallet_utils.py

@ -21,7 +21,7 @@ from jmbase.support import (get_password, jmprint, EXIT_FAILURE,
IndentedHelpFormatterWithNL)
from .cryptoengine import TYPE_P2PKH, TYPE_P2SH_P2WPKH, TYPE_P2WPKH, \
TYPE_SEGWIT_LEGACY_WALLET_FIDELITY_BONDS
TYPE_SEGWIT_WALLET_FIDELITY_BONDS
from .output import fmt_utxo
import jmbitcoin as btc
@ -246,7 +246,8 @@ class WalletViewAccount(WalletViewBase):
self.account_name = account_name
self.xpub = xpub
if branches:
assert len(branches) in [2, 3, 4] #3 if imported keys, 4 if fidelity bonds
assert len(branches) in [2, 3, 4, 5] #3 if imported keys, 4 if fidelity bonds
#5 if all those plus imported
assert all([isinstance(x, WalletViewBranch) for x in branches])
self.branches = branches
@ -360,7 +361,11 @@ def wallet_showutxos(wallet_service, showprivkey):
for u, av in utxos[md].items():
success, us = utxo_to_utxostr(u)
assert success
key = wallet_service.get_key_from_addr(av['address'])
key = wallet_service._get_key_from_path(av["path"])[0]
if FidelityBondMixin.is_timelocked_path(av["path"]):
key, locktime = key
else:
locktime = None
tries = podle.get_podle_tries(u, key, max_tries)
tries_remaining = max(0, max_tries - tries)
mixdepth = wallet_service.wallet.get_details(av['path'])[0]
@ -372,6 +377,8 @@ def wallet_showutxos(wallet_service, showprivkey):
'frozen': True if u in utxo_d else False}
if showprivkey:
unsp[us]['privkey'] = wallet_service.get_wif_path(av['path'])
if locktime:
unsp[us]["locktime"] = str(datetime.utcfromtimestamp(locktime))
used_commitments, external_commitments = podle.get_podle_commitments()
for u, ec in external_commitments.items():
@ -466,27 +473,23 @@ def wallet_display(wallet_service, showprivkey, displayall=False,
if m == FidelityBondMixin.FIDELITY_BOND_MIXDEPTH and \
isinstance(wallet_service.wallet, FidelityBondMixin):
address_type = FidelityBondMixin.BIP32_TIMELOCK_ID
unused_index = wallet_service.get_next_unused_index(m, address_type)
timelocked_gaplimit = (wallet_service.wallet.gap_limit
// FidelityBondMixin.TIMELOCK_GAP_LIMIT_REDUCTION_FACTOR)
entrylist = []
for k in range(unused_index + timelocked_gaplimit):
for timenumber in range(FidelityBondMixin.TIMENUMBERS_PER_PUBKEY):
path = wallet_service.get_path(m, address_type, k, timenumber)
addr = wallet_service.get_address_from_path(path)
timelock = datetime.utcfromtimestamp(path[-1])
balance = sum([utxodata["value"] for utxo, utxodata in
utxos[m].items() if path == utxodata["path"]])
status = timelock.strftime("%Y-%m-%d") + " [" + (
"LOCKED" if datetime.now() < timelock else "UNLOCKED") + "]"
privkey = ""
if showprivkey:
privkey = wallet_service.get_wif_path(path)
if displayall or balance > 0:
entrylist.append(WalletViewEntry(
wallet_service.get_path_repr(path), m, address_type, k,
addr, [balance, balance], priv=privkey, used=status))
for timenumber in range(FidelityBondMixin.TIMENUMBER_COUNT):
path = wallet_service.get_path(m, address_type, timenumber, timenumber)
addr = wallet_service.get_address_from_path(path)
timelock = datetime.utcfromtimestamp(path[-1])
balance = sum([utxodata["value"] for utxo, utxodata in
utxos[m].items() if path == utxodata["path"]])
status = timelock.strftime("%Y-%m-%d") + " [" + (
"LOCKED" if datetime.now() < timelock else "UNLOCKED") + "]"
privkey = ""
if showprivkey:
privkey = wallet_service.get_wif_path(path)
if displayall or balance > 0:
entrylist.append(WalletViewEntry(
wallet_service.get_path_repr(path), m, address_type, k,
addr, [balance, balance], priv=privkey, used=status))
xpub_key = wallet_service.get_bip32_pub_export(m, address_type)
path = wallet_service.get_path_repr(wallet_service.get_path(m, address_type))
branchlist.append(WalletViewBranch(path, m, address_type, entrylist,
@ -633,13 +636,8 @@ def wallet_generate_recover_bip39(method, walletspath, default_wallet_name,
if not wallet_name:
wallet_name = default_wallet_name
wallet_path = os.path.join(walletspath, wallet_name)
# disable creating fidelity bond wallets for now until the
# rest of the fidelity bond feature is created
#support_fidelity_bonds = enter_do_support_fidelity_bonds()
support_fidelity_bonds = False
support_fidelity_bonds = enter_do_support_fidelity_bonds()
wallet_cls = get_wallet_cls(get_configured_wallet_type(support_fidelity_bonds))
wallet = create_wallet(wallet_path, password, mixdepth, wallet_cls,
entropy=entropy,
entropy_extension=mnemonic_extension)
@ -1227,16 +1225,20 @@ def wallet_gettimelockaddress(wallet, locktime_string):
m = FidelityBondMixin.FIDELITY_BOND_MIXDEPTH
address_type = FidelityBondMixin.BIP32_TIMELOCK_ID
index = wallet.get_next_unused_index(m, address_type)
lock_datetime = datetime.strptime(locktime_string, "%Y-%m")
timenumber = FidelityBondMixin.timestamp_to_time_number(timegm(
lock_datetime.timetuple()))
index = timenumber
path = wallet.get_path(m, address_type, index, timenumber)
jmprint("path = " + wallet.get_path_repr(path), "info")
jmprint("Coins sent to this address will be not be spendable until "
+ lock_datetime.strftime("%B %Y") + ". Full date: "
+ str(lock_datetime))
jmprint("WARNING: Only send coins here which are from coinjoins or otherwise"
+ " not linked to your identity. Also, use a sweep transaction when funding the"
+ " timelocked address, i.e. Don't create a change address. See the privacy warnings in"
+ " fidelity-bonds.md")
addr = wallet.get_address_from_path(path)
return addr
@ -1296,8 +1298,8 @@ def get_configured_wallet_type(support_fidelity_bonds):
if not support_fidelity_bonds:
return configured_type
if configured_type == TYPE_P2SH_P2WPKH:
return TYPE_SEGWIT_LEGACY_WALLET_FIDELITY_BONDS
if configured_type == TYPE_P2WPKH:
return TYPE_SEGWIT_WALLET_FIDELITY_BONDS
else:
raise ValueError("Fidelity bonds not supported with the configured "
"options of segwit and native. Edit joinmarket.cfg")

51
jmclient/jmclient/yieldgenerator.py

@ -4,15 +4,19 @@ import datetime
import os
import time
import abc
import base64
from twisted.python.log import startLogging
from optparse import OptionParser
from jmbase import get_log
from jmclient import (Maker, jm_single, load_program_config,
JMClientProtocolFactory, start_reactor, calc_cj_fee,
WalletService, add_base_options, SNICKERReceiver,
SNICKERClientProtocolFactory)
SNICKERClientProtocolFactory, FidelityBondMixin,
get_interest_rate, fmt_utxo)
from .wallet_utils import open_test_wallet_maybe, get_wallet_path
from jmbase.support import EXIT_ARGERROR, EXIT_FAILURE, get_jm_version_str
import jmbitcoin as btc
from jmclient.fidelity_bond import FidelityBond
jlog = get_log()
@ -66,7 +70,6 @@ class YieldGenerator(Maker):
return to_cancel, to_announce
class YieldGeneratorBasic(YieldGenerator):
"""A simplest possible instantiation of a yieldgenerator.
It will often (but not always) reannounce orders after transactions,
@ -112,6 +115,50 @@ class YieldGeneratorBasic(YieldGenerator):
return [order]
def get_fidelity_bond_template(self):
if not isinstance(self.wallet_service.wallet, FidelityBondMixin):
jlog.info("Not a fidelity bond wallet, not announcing fidelity bond")
return None
blocks = jm_single().bc_interface.get_current_block_height()
mediantime = jm_single().bc_interface.get_best_block_median_time()
BLOCK_COUNT_SAFETY = 2 #use this safety number to reduce chances of the proof expiring
#before the taker gets a chance to verify it
RETARGET_INTERVAL = 2016
CERT_MAX_VALIDITY_TIME = 1
cert_expiry = ((blocks + BLOCK_COUNT_SAFETY) // RETARGET_INTERVAL) + CERT_MAX_VALIDITY_TIME
utxos = self.wallet_service.wallet.get_utxos_by_mixdepth(include_disabled=True,
includeheight=True)[FidelityBondMixin.FIDELITY_BOND_MIXDEPTH]
timelocked_utxos = [(outpoint, info) for outpoint, info in utxos.items()
if FidelityBondMixin.is_timelocked_path(info["path"])]
if len(timelocked_utxos) == 0:
jlog.info("No timelocked coins in wallet, not announcing fidelity bond")
return
timelocked_utxos_with_confirmation_time = [(outpoint, info,
jm_single().bc_interface.get_block_time(
jm_single().bc_interface.get_block_hash(info["height"])
))
for (outpoint, info) in timelocked_utxos]
interest_rate = get_interest_rate()
max_valued_bond = max(timelocked_utxos_with_confirmation_time, key=lambda x:
FidelityBondMixin.calculate_timelocked_fidelity_bond_value(x[1]["value"], x[2],
x[1]["path"][-1], mediantime, interest_rate)
)
(utxo_priv, locktime), engine = self.wallet_service.wallet._get_key_from_path(
max_valued_bond[1]["path"])
utxo_pub = engine.privkey_to_pubkey(utxo_priv)
cert_priv = os.urandom(32) + b"\x01"
cert_pub = btc.privkey_to_pubkey(cert_priv)
cert_msg = b"fidelity-bond-cert|" + cert_pub + b"|" + str(cert_expiry).encode("ascii")
cert_sig = base64.b64decode(btc.ecdsa_sign(cert_msg, utxo_priv))
utxo = (max_valued_bond[0][0], max_valued_bond[0][1])
fidelity_bond = FidelityBond(utxo, utxo_pub, locktime, cert_expiry,
cert_priv, cert_pub, cert_sig)
jlog.info("Announcing fidelity bond coin {}".format(fmt_utxo(utxo)))
return fidelity_bond
def oid_to_order(self, offer, amount):
total_amount = amount + offer["txfee"]
real_cjfee = calc_cj_fee(offer["ordertype"], offer["cjfee"], amount)

11
jmclient/test/test_client_protocol.py

@ -43,7 +43,7 @@ class DummyTaker(Taker):
def default_taker_info_callback(self, infotype, msg):
jlog.debug(infotype + ":" + msg)
def initialize(self, orderbook):
def initialize(self, orderbook, fidelity_bonds_info):
"""Once the daemon is active and has returned the current orderbook,
select offers, re-initialize variables and prepare a commitment,
then send it to the protocol to fill offers.
@ -90,6 +90,7 @@ class DummyMaker(Maker):
self.aborted = False
self.wallet_service = WalletService(DummyWallet())
self.offerlist = self.create_my_orders()
self.fidelity_bond = None
def try_to_create_my_orders(self):
pass
@ -184,8 +185,8 @@ class JMTestServerProtocol(JMBaseProtocol):
return {'accepted': True}
@JMSetup.responder
def on_JM_SETUP(self, role, initdata):
show_receipt("JMSETUP", role, initdata)
def on_JM_SETUP(self, role, offers, use_fidelity_bond):
show_receipt("JMSETUP", role, offers, use_fidelity_bond)
d = self.callRemote(JMSetupDone)
self.defaultCallbacks(d)
return {'accepted': True}
@ -195,8 +196,10 @@ class JMTestServerProtocol(JMBaseProtocol):
show_receipt("JMREQUESTOFFERS")
#build a huge orderbook to test BigString Argument
orderbook = ["aaaa" for _ in range(15)]
fidelitybonds = ["bbbb" for _ in range(15)]
d = self.callRemote(JMOffers,
orderbook=json.dumps(orderbook))
orderbook=json.dumps(orderbook),
fidelitybonds=json.dumps(fidelitybonds))
self.defaultCallbacks(d)
return {'accepted': True}

2
jmclient/test/test_coinjoin.py

@ -80,7 +80,7 @@ def create_orders(makers):
maker.try_to_create_my_orders()
def init_coinjoin(taker, makers, orderbook, cj_amount):
init_data = taker.initialize(orderbook)
init_data = taker.initialize(orderbook, [])
assert init_data[0], "taker.initialize error"
active_orders = init_data[4]
maker_data = {}

7
jmclient/test/test_support.py

@ -6,7 +6,7 @@ from jmclient import (select, select_gradual, select_greedy, select_greediest,
choose_orders, choose_sweep_orders, weighted_order_choose)
from jmclient.support import (calc_cj_fee, rand_exp_array,
rand_norm_array, rand_weighted_choice,
cheapest_order_choose)
cheapest_order_choose, fidelity_bond_weighted_order_choose)
from taker_test_data import t_orderbook
import copy
@ -73,6 +73,11 @@ def test_choose_orders():
#test the hated 'cheapest'
orders_fees = choose_orders(orderbook, 100000000, 3, cheapest_order_choose)
assert len(orders_fees[0]) == 3
#test the fidelity bond one
for i, o in enumerate(orderbook):
o["fidelity_bond_value"] = i+1
orders_fees = choose_orders(orderbook, 100000000, 3, fidelity_bond_weighted_order_choose)
assert len(orders_fees[0]) == 3
#test sweep
result, cjamount, total_fee = choose_sweep_orders(orderbook, 50000000,
30000,

16
jmclient/test/test_taker.py

@ -165,11 +165,11 @@ def test_filter_rejection(setup_taker):
return False
taker = get_taker(filter_orders=filter_orders_reject)
taker.schedule = [[0, 20000000, 3, "mnsquzxrHXpFsZeL42qwbKdCP2y1esN3qw", 0, NO_ROUNDING]]
res = taker.initialize(t_orderbook)
res = taker.initialize(t_orderbook, [])
assert not res[0]
taker = get_taker(filter_orders=filter_orders_reject)
taker.schedule = [[0, 0, 3, "mnsquzxrHXpFsZeL42qwbKdCP2y1esN3qw", 0, NO_ROUNDING]]
res = taker.initialize(t_orderbook)
res = taker.initialize(t_orderbook, [])
assert not res[0]
@pytest.mark.parametrize(
@ -258,7 +258,7 @@ def test_make_commitment(setup_taker, mixdepth, cjamt, failquery, external,
def test_not_found_maker_utxos(setup_taker):
taker = get_taker([(0, 20000000, 3, "mnsquzxrHXpFsZeL42qwbKdCP2y1esN3qw", 0, NO_ROUNDING)])
orderbook = copy.deepcopy(t_orderbook)
res = taker.initialize(orderbook)
res = taker.initialize(orderbook, [])
taker.orderbook = copy.deepcopy(t_chosen_orders) #total_cjfee unaffected, all same
maker_response = copy.deepcopy(t_maker_response)
jm_single().bc_interface.setQUSFail(True)
@ -270,7 +270,7 @@ def test_not_found_maker_utxos(setup_taker):
def test_auth_pub_not_found(setup_taker):
taker = get_taker([(0, 20000000, 3, "mnsquzxrHXpFsZeL42qwbKdCP2y1esN3qw", 0, NO_ROUNDING)])
orderbook = copy.deepcopy(t_orderbook)
res = taker.initialize(orderbook)
res = taker.initialize(orderbook, [])
taker.orderbook = copy.deepcopy(t_chosen_orders) #total_cjfee unaffected, all same
maker_response = copy.deepcopy(t_maker_response)
utxos = [utxostr_to_utxo(x)[1] for x in [
@ -351,11 +351,11 @@ def test_taker_init(setup_taker, schedule, highfee, toomuchcoins, minmakers,
if schedule[0][1] == 0.2:
#triggers calc-ing amount based on a fraction
jm_single().mincjamount = 50000000 #bigger than 40m = 0.2 * 200m
res = taker.initialize(orderbook)
res = taker.initialize(orderbook, [])
assert res[0]
assert res[1] == jm_single().mincjamount
return clean_up()
res = taker.initialize(orderbook)
res = taker.initialize(orderbook, [])
if toomuchcoins or ignored:
assert not res[0]
return clean_up()
@ -427,7 +427,7 @@ def test_taker_init(setup_taker, schedule, highfee, toomuchcoins, minmakers,
assert res[0]
#re-calling will trigger "finished" code, since schedule is "complete".
res = taker.initialize(orderbook)
res = taker.initialize(orderbook, [])
assert not res[0]
#some exception cases: no coinjoin address, no change address:
@ -454,7 +454,7 @@ def test_custom_change(setup_taker):
for script, addr in zip(scripts, addrs):
taker = get_taker(schedule, custom_change=addr)
orderbook = copy.deepcopy(t_orderbook)
res = taker.initialize(orderbook)
res = taker.initialize(orderbook, [])
taker.orderbook = copy.deepcopy(t_chosen_orders)
maker_response = copy.deepcopy(t_maker_response)
res = taker.receive_utxos(maker_response)

111
jmclient/test/test_wallet.py

@ -11,7 +11,7 @@ from jmbase import get_log, hextobin
from jmclient import load_test_config, jm_single, BaseWallet, \
SegwitLegacyWallet,BIP32Wallet, BIP49Wallet, LegacyWallet,\
VolatileStorage, get_network, cryptoengine, WalletError,\
SegwitWallet, WalletService, SegwitLegacyWalletFidelityBonds,\
SegwitWallet, WalletService, SegwitWalletFidelityBonds,\
create_wallet, open_test_wallet_maybe, \
FidelityBondMixin, FidelityBondWatchonlyWallet, wallet_gettimelockaddress
from test_blockchaininterface import sync_test_wallet
@ -243,19 +243,19 @@ def test_bip32_addresses_p2sh_p2wpkh(setup_wallet, mixdepth, internal, index, ad
assert address == wallet.get_addr(mixdepth, internal, index)
@pytest.mark.parametrize('index,timenumber,address,wif', [
[0, 0, 'bcrt1qndcqwedwa4lu77ryqpvp738d6p034a2fv8mufw3pw5smfcn39sgqpesn76', 'cST4g5R3mKp44K4J8PRVyys4XJu6EFavZyssq67PJKCnbhjdEdBY'],
[0, 50, 'bcrt1q73zhrfcu0ttkk4er9esrmvnpl6wpzhny5aly97jj9nw52agf8ncqjv8rda', 'cST4g5R3mKp44K4J8PRVyys4XJu6EFavZyssq67PJKCnbhjdEdBY'],
[5, 0, 'bcrt1qz5208jdm6399ja309ra28d0a34qlt0859u77uxc94v5mgk7auhtssau4pw', 'cRnUaBYTmyZURPe72YCrtvgxpBMvLKPZaCoXvKuWRPMryeJeAZx2'],
[9, 1, 'bcrt1qa7pd6qnadpmlm29vtvqnykalc34tr33eclaz7eeqal59n4gwr28qwnka2r', 'cQCxEPCWMwXVB16zCikDBTXMUccx6ioHQipPhYEp1euihkJUafyD']
[0, 0, 'bcrt1qgysu2eynn6klarz200ctgev7gqhhp7hwsdaaec3c7h0ltmc3r68q87c2d3', 'cVASAS6bpC5yctGmnsKaDz7D8CxEwccUtpjSNBQzeV2fw8ox8RR9'],
[0, 50, 'bcrt1qyrdhyqzj87vq20e853x7gzhx9lp8ta6cd8mwp8haqex8r4vrg2wsf7rcxm', 'cVASAS6bpC5yctGmnsKaDz7D8CxEwccUtpjSNBQzeV2fw8ox8RR9'],
[5, 0, 'bcrt1quunmmsudhpsuksa2ke8m6aj7757mst966mqq50nckx9wdrs4y6fs9gjuww', 'cUgT5jRjYi6i8Fc7TirJrrvhbs7ceSqJ6USKboVrLYghJKDzEQHQ'],
[9, 1, 'bcrt1qvpgmrn5a7yc0h2j6fp8jhtwzd8eetlt7hsu3cn098qftzp4t2h6sp5p35p', 'cW7H2pv6Rr5NWaTAnDC6r7bviHwDsAwyqh4XdZTE4xf2H2DB2hmb']
])
def test_bip32_timelocked_addresses(setup_wallet, index, timenumber, address, wif):
jm_single().config.set('BLOCKCHAIN', 'network', 'testnet')
entropy = unhexlify('2e0339ba89b4a1272cdf78b27ee62669ee01992a59e836e2807051be128ca817')
storage = VolatileStorage()
SegwitLegacyWalletFidelityBonds.initialize(
SegwitWalletFidelityBonds.initialize(
storage, get_network(), entropy=entropy, max_mixdepth=1)
wallet = SegwitLegacyWalletFidelityBonds(storage)
wallet = SegwitWalletFidelityBonds(storage)
mixdepth = FidelityBondMixin.FIDELITY_BOND_MIXDEPTH
address_type = FidelityBondMixin.BIP32_TIMELOCK_ID
@ -274,12 +274,12 @@ def test_bip32_timelocked_addresses(setup_wallet, index, timenumber, address, wi
])
def test_gettimelockaddress_method(setup_wallet, timenumber, locktime_string):
storage = VolatileStorage()
SegwitLegacyWalletFidelityBonds.initialize(storage, get_network())
wallet = SegwitLegacyWalletFidelityBonds(storage)
SegwitWalletFidelityBonds.initialize(storage, get_network())
wallet = SegwitWalletFidelityBonds(storage)
m = FidelityBondMixin.FIDELITY_BOND_MIXDEPTH
address_type = FidelityBondMixin.BIP32_TIMELOCK_ID
index = wallet.get_next_unused_index(m, address_type)
index = timenumber
script = wallet.get_script_and_update_map(m, address_type, index,
timenumber)
addr = wallet.script_to_addr(script)
@ -289,18 +289,18 @@ def test_gettimelockaddress_method(setup_wallet, timenumber, locktime_string):
assert addr == addr_from_method
@pytest.mark.parametrize('index,wif', [
[0, 'cMg9eH3fW2JDSyggvXucjmECRwiheCMDo2Qik8y1keeYaxynzrYa'],
[9, 'cURA1Qgxhd7QnhhwxCnCHD4pZddVrJdu2BkTdzNaTp9owRSkUvPy'],
[50, 'cRTaHZ1eezb8s6xsT2V7EAevYToQMi7cxQD9vgFZzaJZDfhMhf3c']
[0, 'cVQbz7DB5JQ1TGsg9Dbm32VtJbXBHaj39Yc9QLkaGpRgXcibHTDH'],
[9, 'cULqe2sYZ4z8jZTGybr2Bzf4EyiT5Ts6wAE3mvCUofRuTVsofR8N'],
[50, 'cQNp7cQbrwjWuxmbkZF8ax9ogmTuWp3Ykb9LEpainhRTJXYc8Deu']
])
def test_bip32_burn_keys(setup_wallet, index, wif):
jm_single().config.set('BLOCKCHAIN', 'network', 'testnet')
entropy = unhexlify('2e0339ba89b4a1272cdf78b27ee62669ee01992a59e836e2807051be128ca817')
storage = VolatileStorage()
SegwitLegacyWalletFidelityBonds.initialize(
SegwitWalletFidelityBonds.initialize(
storage, get_network(), entropy=entropy, max_mixdepth=1)
wallet = SegwitLegacyWalletFidelityBonds(storage)
wallet = SegwitWalletFidelityBonds(storage)
mixdepth = FidelityBondMixin.FIDELITY_BOND_MIXDEPTH
address_type = FidelityBondMixin.BIP32_BURN_ID
@ -402,8 +402,8 @@ def test_timelocked_output_signing(setup_wallet):
jm_single().config.set('BLOCKCHAIN', 'network', 'testnet')
ensure_bip65_activated()
storage = VolatileStorage()
SegwitLegacyWalletFidelityBonds.initialize(storage, get_network())
wallet = SegwitLegacyWalletFidelityBonds(storage)
SegwitWalletFidelityBonds.initialize(storage, get_network())
wallet = SegwitWalletFidelityBonds(storage)
index = 0
timenumber = 0
@ -629,7 +629,7 @@ def test_path_repr_imported(setup_wallet):
[0, 1577836800],
[50, 1709251200],
[300, 2366841600],
[400, None], #too far in the future
[1000, None], #too far in the future
[-1, None] #before epoch
])
def test_timenumber_to_timestamp(setup_wallet, timenumber, timestamp):
@ -646,7 +646,7 @@ def test_timenumber_to_timestamp(setup_wallet, timenumber, timestamp):
[1709251200, 50],
[2366841600, 300],
[1577836801, None], #not exactly midnight on first of month
[2629670400, None], #too far in future
[4133980800, None], #too far in future
[1575158400, None] #before epoch
])
def test_timestamp_to_timenumber(setup_wallet, timestamp, timenumber):
@ -787,14 +787,14 @@ def test_wallet_mixdepth_decrease(setup_wallet):
def test_watchonly_wallet(setup_wallet):
jm_single().config.set('BLOCKCHAIN', 'network', 'testnet')
storage = VolatileStorage()
SegwitLegacyWalletFidelityBonds.initialize(storage, get_network())
wallet = SegwitLegacyWalletFidelityBonds(storage)
SegwitWalletFidelityBonds.initialize(storage, get_network())
wallet = SegwitWalletFidelityBonds(storage)
paths = [
"m/49'/1'/0'/0/0",
"m/49'/1'/0'/1/0",
"m/49'/1'/0'/2/0:1577836800",
"m/49'/1'/0'/2/0:2314051200"
"m/84'/1'/0'/0/0",
"m/84'/1'/0'/1/0",
"m/84'/1'/0'/2/0:1577836800",
"m/84'/1'/0'/2/0:2314051200"
]
burn_path = "m/49'/1'/0'/3/0"
@ -821,6 +821,67 @@ def test_watchonly_wallet(setup_wallet):
assert script == watchonly_script
assert burn_pubkey == watchonly_burn_pubkey
def test_calculate_timelocked_fidelity_bond_value(setup_wallet):
EPSILON = 0.000001
YEAR = 60*60*24*356.25
#the function should be flat anywhere before the locktime ends
values = [FidelityBondMixin.calculate_timelocked_fidelity_bond_value(
utxo_value=100000000,
confirmation_time=0,
locktime=6*YEAR,
current_time=y*YEAR,
interest_rate=0.01
)
for y in range(4)
]
value_diff = [values[i] - values[i+1] for i in range(len(values)-1)]
for vd in value_diff:
assert abs(vd) < EPSILON
#after locktime, the value should go down
values = [FidelityBondMixin.calculate_timelocked_fidelity_bond_value(
utxo_value=100000000,
confirmation_time=0,
locktime=6*YEAR,
current_time=(6+y)*YEAR,
interest_rate=0.01
)
for y in range(5)
]
value_diff = [values[i+1] - values[i] for i in range(len(values)-1)]
for vrd in value_diff:
assert vrd < 0
#value of a bond goes up as the locktime goes up
values = [FidelityBondMixin.calculate_timelocked_fidelity_bond_value(
utxo_value=100000000,
confirmation_time=0,
locktime=y*YEAR,
current_time=0,
interest_rate=0.01
)
for y in range(5)
]
value_ratio = [values[i] / values[i+1] for i in range(len(values)-1)]
value_ratio_diff = [value_ratio[i] - value_ratio[i+1] for i in range(len(value_ratio)-1)]
for vrd in value_ratio_diff:
assert vrd < 0
#value of a bond locked into the far future is constant, clamped at the value of burned coins
values = [FidelityBondMixin.calculate_timelocked_fidelity_bond_value(
utxo_value=100000000,
confirmation_time=0,
locktime=(200+y)*YEAR,
current_time=0,
interest_rate=0.01
)
for y in range(5)
]
value_diff = [values[i] - values[i+1] for i in range(len(values)-1)]
for vd in value_diff:
assert abs(vd) < EPSILON
@pytest.mark.parametrize('password, wallet_cls', [
["hunter2", SegwitLegacyWallet],
["hunter2", SegwitWallet],

1
jmdaemon/jmdaemon/__init__.py

@ -14,6 +14,7 @@ from .daemon_protocol import (JMDaemonServerProtocolFactory, JMDaemonServerProto
from .protocol import (COMMAND_PREFIX, ORDER_KEYS, NICK_HASH_LENGTH,
NICK_MAX_ENCODED, JM_VERSION, JOINMARKET_NICK_HEADER)
from .message_channel import MessageChannelCollection
# Set default logging handler to avoid "No handler found" warnings.
try:
from logging import NullHandler

40
jmdaemon/jmdaemon/daemon_protocol.py

@ -474,6 +474,7 @@ class JMDaemonServerProtocol(amp.AMP, OrderbookWatch):
self.crypto_boxes = {}
self.sig_lock = threading.Lock()
self.active_orders = {}
self.use_fidelity_bond = False
def checkClientResponse(self, response):
"""A generic check of client acceptance; any failure
@ -551,7 +552,7 @@ class JMDaemonServerProtocol(amp.AMP, OrderbookWatch):
return {'accepted': True}
@JMSetup.responder
def on_JM_SETUP(self, role, initdata):
def on_JM_SETUP(self, role, offers, use_fidelity_bond):
assert self.jm_state == 0
self.role = role
self.crypto_boxes = {}
@ -565,8 +566,9 @@ class JMDaemonServerProtocol(amp.AMP, OrderbookWatch):
if self.role == "TAKER":
self.mcc.pubmsg(COMMAND_PREFIX + "orderbook")
elif self.role == "MAKER":
self.offerlist = json.loads(initdata)
self.mcc.announce_orders(self.offerlist)
self.offerlist = json.loads(offers)
self.use_fidelity_bond = use_fidelity_bond
self.mcc.announce_orders(self.offerlist, None, None, None)
self.jm_state = 1
return {'accepted': True}
@ -592,10 +594,16 @@ class JMDaemonServerProtocol(amp.AMP, OrderbookWatch):
This call is stateless."""
rows = self.db.execute('SELECT * FROM orderbook;').fetchall()
self.orderbook = [dict([(k, o[k]) for k in ORDER_KEYS]) for o in rows]
log.msg("About to send orderbook of size: " + str(len(self.orderbook)))
string_orderbook = json.dumps(self.orderbook)
d = self.callRemote(JMOffers,
orderbook=string_orderbook)
fbond_rows = self.db.execute("SELECT * FROM fidelitybonds;").fetchall()
fidelitybonds = [fb for fb in fbond_rows]
string_fidelitybonds = json.dumps(fidelitybonds)
log.msg("About to send orderbook (size=" + str(len(self.orderbook))
+ " with fidelity bonds (size=" + str(len(fidelitybonds)))
d = self.callRemote(JMOffers, orderbook=string_orderbook,
fidelitybonds=string_fidelitybonds)
self.defaultCallbacks(d)
return {'accepted': True}
@ -655,7 +663,13 @@ class JMDaemonServerProtocol(amp.AMP, OrderbookWatch):
if len(to_cancel) > 0:
self.mcc.cancel_orders(to_cancel)
if len(to_announce) > 0:
self.mcc.announce_orders(to_announce, None, None)
self.mcc.announce_orders(to_announce, None, None, None)
return {"accepted": True}
@JMFidelityBondProof.responder
def on_JM_FIDELITY_BOND_PROOF(self, nick, proof):
"""Called by maker client as a reply to a request of a fidelity bond proof"""
self.mcc.announce_orders(self.offerlist, nick, proof, new_mc=None)
return {"accepted": True}
@JMIOAuth.responder
@ -711,7 +725,16 @@ class JMDaemonServerProtocol(amp.AMP, OrderbookWatch):
def on_orderbook_requested(self, nick, mc=None):
"""Dealt with by daemon, assuming offerlist is up to date
"""
self.mcc.announce_orders(self.offerlist, nick, mc)
if self.use_fidelity_bond:
taker_nick = nick
maker_nick = self.mcc.nick
d = self.callRemote(JMFidelityBondProofRequest,
takernick=taker_nick,
makernick=maker_nick)
self.defaultCallbacks(d)
else:
self.mcc.announce_orders(self.offerlist, nick, fidelity_bond_proof_msg=None,
new_mc=mc)
@maker_only
def on_order_fill(self, nick, oid, amount, taker_pk, commit):
@ -1015,6 +1038,7 @@ class JMDaemonServerProtocol(amp.AMP, OrderbookWatch):
if self.mcc:
self.mcc.shutdown()
class JMDaemonServerProtocolFactory(ServerFactory):
protocol = JMDaemonServerProtocol

12
jmdaemon/jmdaemon/fidelity_bond_sanity_check.py

@ -0,0 +1,12 @@
import base64
def fidelity_bond_sanity_check(proof):
try:
decoded_data = base64.b64decode(proof, validate=True)
if len(decoded_data) != 252:
return False
except Exception:
return False
return True

37
jmdaemon/jmdaemon/message_channel.py

@ -6,7 +6,8 @@ import threading
from twisted.internet import reactor
from jmdaemon import encrypt_encode, decode_decrypt, COMMAND_PREFIX,\
NICK_HASH_LENGTH, NICK_MAX_ENCODED, plaintext_commands,\
encrypted_commands, commitment_broadcast_list, offername_list
encrypted_commands, commitment_broadcast_list, offername_list,\
fidelity_bond_cmd_list
from jmbase.support import get_log
from functools import wraps
@ -284,12 +285,13 @@ class MessageChannelCollection(object):
"; cannot find on any message channel.")
return
def announce_orders(self, orderlist, nick=None, new_mc=None):
def announce_orders(self, orderlist, nick, fidelity_bond_proof_msg, new_mc):
"""Send orders defined in list orderlist either
to the shared public channel (pit), on all
message channels, if nick=None,
or to an individual counterparty nick, as
privmsg, on a specific mc.
Fidelity bonds can only be announced over privmsg, nick must be nonNone
"""
order_keys = ['oid', 'minsize', 'maxsize', 'txfee', 'cjfee']
orderlines = []
@ -301,6 +303,7 @@ class MessageChannelCollection(object):
"Tried to announce orders on an unavailable message channel.")
return
if nick is None:
assert fidelity_bond_proof_msg is None
for mc in self.available_channels():
mc.announce_orders(orderlines)
else:
@ -310,6 +313,9 @@ class MessageChannelCollection(object):
cmd = orderlist[0]['ordertype']
msg = ' '.join(orderlines[0].split(' ')[1:])
msg += ''.join(orderlines[1:])
if fidelity_bond_proof_msg:
msg += (COMMAND_PREFIX + fidelity_bond_cmd_list[0] + " " +
fidelity_bond_proof_msg)
if new_mc:
self.prepare_privmsg(nick, cmd, msg, mc=new_mc)
else:
@ -563,7 +569,8 @@ class MessageChannelCollection(object):
# orderbook watcher commands
def register_orderbookwatch_callbacks(self,
on_order_seen=None,
on_order_cancel=None):
on_order_cancel=None,
on_fidelity_bond_seen=None):
"""Special cases:
on_order_seen: use it as a trigger for presence of nick.
on_order_cancel: what happens if cancel/modify in one place
@ -572,7 +579,7 @@ class MessageChannelCollection(object):
self.on_order_seen = on_order_seen
for mc in self.mchannels:
mc.register_orderbookwatch_callbacks(self.on_order_seen_trigger,
on_order_cancel)
on_order_cancel, on_fidelity_bond_seen)
def on_orderbook_requested_trigger(self, nick, mc):
"""Update nicks_seen state to reflect presence of
@ -647,6 +654,7 @@ class MessageChannel(object):
# orderbook watch functions
self.on_order_seen = None
self.on_order_cancel = None
self.on_fidelity_bond_seen = None
# taker functions
self.on_error = None
self.on_pubkey = None
@ -730,9 +738,11 @@ class MessageChannel(object):
# orderbook watcher commands
def register_orderbookwatch_callbacks(self,
on_order_seen=None,
on_order_cancel=None):
on_order_cancel=None,
on_fidelity_bond_seen=None):
self.on_order_seen = on_order_seen
self.on_order_cancel = on_order_cancel
self.on_fidelity_bond_seen = on_fidelity_bond_seen
# taker commands
def register_taker_callbacks(self,
@ -813,6 +823,21 @@ class MessageChannel(object):
return True
return False
def check_for_fidelity_bond(self, nick, _chunks):
if _chunks[0] in fidelity_bond_cmd_list:
try:
fidelity_bond_proof_msg = _chunks[1]
if self.on_fidelity_bond_seen:
self.on_fidelity_bond_seen(nick, _chunks[0], fidelity_bond_proof_msg)
except IndexError as e:
log.debug(e)
log.debug('index error parsing chunks, possibly malformed '
'offer by other party. No user action required. '
'Triggered by: ' + str(nick))
finally:
return True
return False
def cancel_orders(self, oid_list):
clines = [COMMAND_PREFIX + 'cancel ' + str(oid) for oid in oid_list]
self.pubmsg(''.join(clines))
@ -952,6 +977,8 @@ class MessageChannel(object):
# orderbook watch commands
if self.check_for_orders(nick, _chunks):
pass
elif self.check_for_fidelity_bond(nick, _chunks):
pass
# taker commands
elif _chunks[0] == 'error':
error = " ".join(_chunks[1:])

25
jmdaemon/jmdaemon/orderbookwatch.py

@ -7,6 +7,7 @@ from decimal import InvalidOperation, Decimal
from numbers import Integral
from jmdaemon.protocol import JM_VERSION
from jmdaemon import fidelity_bond_sanity_check
from jmbase.support import get_log, joinmarket_alert, DUST_THRESHOLD
log = get_log()
@ -21,13 +22,12 @@ def dict_factory(cursor, row):
class JMTakerError(Exception):
pass
class OrderbookWatch(object):
def set_msgchan(self, msgchan):
self.msgchan = msgchan
self.msgchan.register_orderbookwatch_callbacks(self.on_order_seen,
self.on_order_cancel)
self.on_order_cancel, self.on_fidelity_bond_seen)
self.msgchan.register_channel_callbacks(
self.on_welcome, self.on_set_topic, None, self.on_disconnect,
self.on_nick_leave, None)
@ -41,6 +41,8 @@ class OrderbookWatch(object):
self.db.execute("CREATE TABLE orderbook(counterparty TEXT, "
"oid INTEGER, ordertype TEXT, minsize INTEGER, "
"maxsize INTEGER, txfee INTEGER, cjfee TEXT);")
self.db.execute("CREATE TABLE fidelitybonds(counterparty TEXT, "
"takernick TEXT, proof TEXT);");
finally:
self.dblock.release()
@ -134,11 +136,29 @@ class OrderbookWatch(object):
finally:
self.dblock.release()
def on_fidelity_bond_seen(self, nick, bond_type, fidelity_bond_proof_msg):
taker_nick = self.msgchan.nick
maker_nick = nick
if not fidelity_bond_sanity_check.fidelity_bond_sanity_check(fidelity_bond_proof_msg):
log.debug("Failed to verify fidelity bond for {}, skipping."
.format(maker_nick))
return
try:
self.dblock.acquire(True)
self.db.execute("DELETE FROM fidelitybonds WHERE counterparty=?;",
(nick, ))
self.db.execute("INSERT INTO fidelitybonds VALUES(?, ?, ?);",
(nick, taker_nick, fidelity_bond_proof_msg))
finally:
self.dblock.release()
def on_nick_leave(self, nick):
try:
self.dblock.acquire(True)
self.db.execute('DELETE FROM orderbook WHERE counterparty=?;',
(nick,))
self.db.execute('DELETE FROM fidelitybonds WHERE counterparty=?;',
(nick,))
finally:
self.dblock.release()
@ -146,5 +166,6 @@ class OrderbookWatch(object):
try:
self.dblock.acquire(True)
self.db.execute('DELETE FROM orderbook;')
self.db.execute('DELETE FROM fidelitybonds;')
finally:
self.dblock.release()

2
jmdaemon/jmdaemon/protocol.py

@ -20,6 +20,8 @@ offertypes = {"reloffer": [(int, "oid"), (int, "minsize"), (int, "maxsize"),
offername_list = list(offertypes.keys())
fidelity_bond_cmd_list = ["tbond"]
ORDER_KEYS = ['counterparty', 'oid', 'ordertype', 'minsize', 'maxsize', 'txfee',
'cjfee']

5
jmdaemon/test/test_daemon_protocol.py

@ -84,7 +84,8 @@ class JMTestClientProtocol(JMBaseProtocol):
show_receipt("JMUP")
d = self.callRemote(JMSetup,
role="TAKER",
initdata="none")
offers="{}",
use_fidelity_bond=False)
self.defaultCallbacks(d)
return {'accepted': True}
@ -110,7 +111,7 @@ class JMTestClientProtocol(JMBaseProtocol):
self.defaultCallbacks(d)
@JMOffers.responder
def on_JM_OFFERS(self, orderbook):
def on_JM_OFFERS(self, orderbook, fidelitybonds):
if end_early:
return {'accepted': True}
jlog.debug("JMOFFERS" + str(orderbook))

8
jmdaemon/test/test_message_channel.py

@ -194,13 +194,13 @@ def test_setup_mc():
del mcc.active_channels[cp1]
mcc.prepare_privmsg(cp1, "auth", "a b c")
#try announcing orders; first public
mcc.announce_orders(t_orderbook)
mcc.announce_orders(t_orderbook, nick=None, fidelity_bond_proof_msg=None, new_mc=None)
#try on fake mc
mcc.announce_orders(t_orderbook, new_mc="fakemc")
mcc.announce_orders(t_orderbook, nick=None, fidelity_bond_proof_msg=None, new_mc="fakemc")
#direct to one cp
mcc.announce_orders(t_orderbook, nick=cp1)
mcc.announce_orders(t_orderbook, nick=cp1, fidelity_bond_proof_msg=None, new_mc=None)
#direct to one cp on one mc
mcc.announce_orders(t_orderbook, nick=cp1, new_mc=dmcs[0])
mcc.announce_orders(t_orderbook, nick=cp1, fidelity_bond_proof_msg=None, new_mc=dmcs[0])
#Next, set up 6 counterparties and fill their offers,
#send txs to them
cps = [make_valid_nick(i) for i in range(1, 7)]

238
jmdaemon/test/test_orderbookwatch.py

@ -3,9 +3,12 @@
import pytest
from jmdaemon.orderbookwatch import OrderbookWatch
from jmdaemon import IRCMessageChannel
from jmdaemon import IRCMessageChannel, fidelity_bond_cmd_list
from jmclient import get_irc_mchannels, load_test_config
from jmdaemon.protocol import JM_VERSION, ORDER_KEYS
from jmbase.support import hextobin
from jmclient.fidelity_bond import FidelityBondProof
class DummyDaemon(object):
def request_signature_verify(self, a, b, c, d, e,
f, g, h):
@ -122,7 +125,234 @@ def test_disconnect_leave():
rows = ob.db.execute('SELECT * FROM orderbook;').fetchall()
orderbook = [dict([(k, o[k]) for k in ORDER_KEYS]) for o in rows]
assert len(orderbook) == 0
@pytest.mark.parametrize(
"valid, fidelity_bond_proof, maker_nick, taker_nick",
[
(
True,
{
#nicksig len = 71, certsig len = 71
"nick-signature": (b'0E\x02!\x00\xdbb\x15\x96\xa0\x87\xb8\x1d\xe05\xddV\xa1\x1bn\x8f'
+ b'q\x90&\x8cG@\x89"2\xb2\x81\x9b\xc00\xa5\xb6\x02 \x03\x14l\xd7BR\xba\x8c:\x88('
+ b'\x8e3l\xac\xf5`T\x87\xfa\xf5\xa9\x1f\x19\xc0\xb6\xe9\xbb\xdc\xc7y\x99'),
"certificate-signature": ("3045022100eb512af938113badb4d7b29e0c22061c51dadb113a9395e"
+ "9ed81a46103391213022029170de414964f07228c4f0d404b1386272bae337f0133f1329d948a"
+ "252fa2a0"),
"certificate-pubkey": "0258efb077960d6848f001904857f062fa453de26c1ad8736f55497254f56e8a74",
"certificate-expiry": 1,
"utxo-pubkey": "02f54f027377e84171296453828aa863c23fc4489453025f49bd3addfb3a359b3d",
"txid": "84c88fafe0bb75f507fe3bfb29a93d10b2e80c15a63b2943c1a5fecb5a55cba2",
"vout": 0,
"locktime": 1640995200
},
"J5A4k9ecQzRRDfBx",
"J55VZ6U6ZyFDNeuv"
),
(
True,
{
#nicksig len = 71, certsig len = 70
"nick-signature": (b'0E\x02!\x00\x80\xc6$\x0c\xa1\x15YS\xacHB\xb33\xfa~\x9f\xb9`\xb3'
+ b'\xfe\xed0\xadHq\xc1~\x03.B\xbb#\x02 #y~]\xd9\xbbX2\xc0\x1b\xe57\xf4\x0f\x1f'
+ b'\xd6$\x01\xf9\x15Z\xc9X\xa5\x18\xbe\x83\x1a&4Y\xd4'),
"certificate-signature": ("304402205669ea394f7381e9abf0b3c013fac2b79d24c02feb86ff153"
+ "cff83c658d7cf7402200b295ace655687f80738f3733c1dc5f1e2b8f351c017a05b8bd31983dd"
+ "4d723f"),
"certificate-pubkey": "031d1c006a6310dbdf57341efc19c3a43c402379d7ccd2480416cadc7579f973f7",
"certificate-expiry": 1,
"utxo-pubkey": "02616c56412eb738a9eacfb0550b43a5a2e77e5d5205ea9e2ca8dfac34e50c9754",
"txid": "84c88fafe0bb75f507fe3bfb29a93d10b2e80c15a63b2943c1a5fecb5a55cba2",
"vout": 1,
"locktime": 1893456000
},
"J54LS6YyJPoseqFS",
"J55VZ6U6ZyFDNeuv"
),
(
True,
{ #nicksig len = 70, certsig len = 71
"nick-signature": (b'0D\x02 K)\xe9\x17d\x0b\xc0\x82(\xd1\xa2*l\xd8\x0eJ\xc7\x01NV\xbf'
+ b'\xcb\x02O]\xc0\x11\x01\x01B"\xed\x02 ob\xa1\xf8>\x80U)\xc8\x96\x86\x1b \x0e'
+ b'\x00.\xf8\x86}\xcd\xf8\x82T\xa2\xb5\x8a4\xdb4\xbe\xf3{'),
"certificate-signature": ("3045022100d3beb5660bef33d095f92a3023bbbab15ece48ab2f211fa"
+ "935b62fe8b764c8c002204892deffb4c9aa0d734aa3f55cc8e2baae4a03fc5a9e571b4f671493"
+ "f1254df9"),
"certificate-pubkey": "03a2d1d15290d6d21204d1153c062970b4ff757a675e47a451fd0ba5c084127807",
"certificate-expiry": 1,
"utxo-pubkey": "03b9c12c9c31286772349b986653d07232327b284bd0787ad5829a04ac68f59b89",
"txid": "70c2995b283db086813d97817264f10b8823b870298d30ab09cb43c6bf2670cf",
"vout": 0,
"locktime": 1735689600
},
"J59PRzM6ZsdA5uyJ",
"J55VZ6U6ZyFDNeuv"
),
(
False,
{ #nick signature with no DER header
"nick-signature": (b'ZD\x02 K)\xe9\x17d\x0b\xc0\x82(\xd1\xa2*l\xd8\x0eJ\xc7\x01NV\xbf'
+ b'\xcb\x02O]\xc0\x11\x01\x01B"\xed\x02 ob\xa1\xf8>\x80U)\xc8\x96\x86\x1b \x0e'
+ b'\x00.\xf8\x86}\xcd\xf8\x82T\xa2\xb5\x8a4\xdb4\xbe\xf3{'),
"certificate-signature": ("3045022100d3beb5660bef33d095f92a3023bbbab15ece48ab2f211fa"
+ "935b62fe8b764c8c002204892deffb4c9aa0d734aa3f55cc8e2baae4a03fc5a9e571b4f671493"
+ "f1254df9"),
"certificate-pubkey": "03a2d1d15290d6d21204d1153c062970b4ff757a675e47a451fd0ba5c084127807",
"certificate-expiry": 1,
"utxo-pubkey": "03b9c12c9c31286772349b986653d07232327b284bd0787ad5829a04ac68f59b89",
"txid": "70c2995b283db086813d97817264f10b8823b870298d30ab09cb43c6bf2670cf",
"vout": 0,
"locktime": 1735689600
},
"J59PRzM6ZsdA5uyJ",
"J55VZ6U6ZyFDNeuv"
),
(
False,
{ #nick signature which fails ecdsa_verify
"nick-signature": (b'0E\x02 K)\xe9\x17d\x0b\xc0\x82(\xd1\xa2*l\xd8\x0eJ\xc7\x01NV\xbf'
+ b'\xcb\x02O]\xc0\x11\x01\x01B"\xed\x02 ob\xa1\xf8>\x80U)\xc8\x96\x86\x1b \x0e'
+ b'\x00.\xf8\x86}\xcd\xf8\x82T\xa2\xb5\x8a4\xdb4\xbe\xf3{'),
"certificate-signature": ("3045022100d3beb5660bef33d095f92a3023bbbab15ece48ab2f211fa"
+ "935b62fe8b764c8c002204892deffb4c9aa0d734aa3f55cc8e2baae4a03fc5a9e571b4f671493"
+ "f1254df9"),
"certificate-pubkey": "03a2d1d15290d6d21204d1153c062970b4ff757a675e47a451fd0ba5c084127807",
"certificate-expiry": 1,
"utxo-pubkey": "03b9c12c9c31286772349b986653d07232327b284bd0787ad5829a04ac68f59b89",
"txid": "70c2995b283db086813d97817264f10b8823b870298d30ab09cb43c6bf2670cf",
"vout": 0,
"locktime": 1735689600
},
"J59PRzM6ZsdA5uyJ",
"J55VZ6U6ZyFDNeuv"
),
(
False,
{ #cert signature which fails ecdsa_verify
"nick-signature": (b'0D\x02 K)\xe9\x17d\x0b\xc0\x82(\xd1\xa2*l\xd8\x0eJ\xc7\x01NV\xbf'
+ b'\xcb\x02O]\xc0\x11\x01\x01B"\xed\x02 ob\xa1\xf8>\x80U)\xc8\x96\x86\x1b \x0e'
+ b'\x00.\xf8\x86}\xcd\xf8\x82T\xa2\xb5\x8a4\xdb4\xbe\xf3{'),
"certificate-signature": ("3055022100d3beb5660bef33d095f92a3023bbbab15ece48ab2f211fa"
+ "935b62fe8b764c8c002204892deffb4c9aa0d734aa3f55cc8e2baae4a03fc5a9e571b4f671493"
+ "f1254df9"),
"certificate-pubkey": "03a2d1d15290d6d21204d1153c062970b4ff757a675e47a451fd0ba5c084127807",
"certificate-expiry": 1,
"utxo-pubkey": "03b9c12c9c31286772349b986653d07232327b284bd0787ad5829a04ac68f59b89",
"txid": "70c2995b283db086813d97817264f10b8823b870298d30ab09cb43c6bf2670cf",
"vout": 0,
"locktime": 1735689600
},
"J59PRzM6ZsdA5uyJ",
"J55VZ6U6ZyFDNeuv"
)
])
def test_fidelity_bond_seen(valid, fidelity_bond_proof, maker_nick, taker_nick):
proof = FidelityBondProof(
maker_nick, taker_nick, hextobin(fidelity_bond_proof['certificate-pubkey']),
fidelity_bond_proof['certificate-expiry'],
hextobin(fidelity_bond_proof['certificate-signature']),
(hextobin(fidelity_bond_proof['txid']), fidelity_bond_proof['vout']),
hextobin(fidelity_bond_proof['utxo-pubkey']), fidelity_bond_proof['locktime']
)
serialized = proof._serialize_proof_msg(fidelity_bond_proof['nick-signature'])
ob = get_ob()
ob.msgchan.nick = taker_nick
ob.on_fidelity_bond_seen(maker_nick, fidelity_bond_cmd_list[0], serialized)
rows = ob.db.execute("SELECT * FROM fidelitybonds;").fetchall()
assert len(rows) == 1
assert rows[0]["counterparty"] == maker_nick
assert rows[0]["takernick"] == taker_nick
try:
parsed_proof = FidelityBondProof.parse_and_verify_proof_msg(rows[0]["counterparty"],
rows[0]["takernick"], rows[0]["proof"])
except ValueError:
parsed_proof = None
if valid:
assert parsed_proof is not None
assert parsed_proof.utxo[0] == hextobin(fidelity_bond_proof["txid"])
assert parsed_proof.utxo[1] == fidelity_bond_proof["vout"]
assert parsed_proof.locktime == fidelity_bond_proof["locktime"]
assert parsed_proof.cert_expiry == fidelity_bond_proof["certificate-expiry"]
assert parsed_proof.utxo_pub == hextobin(fidelity_bond_proof["utxo-pubkey"])
else:
assert parsed_proof is None
def test_duplicate_fidelity_bond_rejected():
fidelity_bond_info = (
(
{
"nick-signature": (b'0E\x02!\x00\xdbb\x15\x96\xa0\x87\xb8\x1d\xe05\xddV\xa1\x1bn\x8f'
+ b'q\x90&\x8cG@\x89"2\xb2\x81\x9b\xc00\xa5\xb6\x02 \x03\x14l\xd7BR\xba\x8c:\x88('
+ b'\x8e3l\xac\xf5`T\x87\xfa\xf5\xa9\x1f\x19\xc0\xb6\xe9\xbb\xdc\xc7y\x99'),
"certificate-signature": ("3045022100eb512af938113badb4d7b29e0c22061c51dadb113a9395e"
+ "9ed81a46103391213022029170de414964f07228c4f0d404b1386272bae337f0133f1329d948a"
+ "252fa2a0"),
"certificate-pubkey": "0258efb077960d6848f001904857f062fa453de26c1ad8736f55497254f56e8a74",
"certificate-expiry": 1,
"utxo-pubkey": "02f54f027377e84171296453828aa863c23fc4489453025f49bd3addfb3a359b3d",
"txid": "84c88fafe0bb75f507fe3bfb29a93d10b2e80c15a63b2943c1a5fecb5a55cba2",
"vout": 0,
"locktime": 1640995200
},
"J5A4k9ecQzRRDfBx",
"J55VZ6U6ZyFDNeuv"
),
(
{
"nick-signature": (b'0E\x02!\x00\x80\xc6$\x0c\xa1\x15YS\xacHB\xb33\xfa~\x9f\xb9`\xb3'
+ b'\xfe\xed0\xadHq\xc1~\x03.B\xbb#\x02 #y~]\xd9\xbbX2\xc0\x1b\xe57\xf4\x0f\x1f'
+ b'\xd6$\x01\xf9\x15Z\xc9X\xa5\x18\xbe\x83\x1a&4Y\xd4'),
"certificate-signature": ("304402205669ea394f7381e9abf0b3c013fac2b79d24c02feb86ff153"
+ "cff83c658d7cf7402200b295ace655687f80738f3733c1dc5f1e2b8f351c017a05b8bd31983dd"
+ "4d723f"),
"certificate-pubkey": "031d1c006a6310dbdf57341efc19c3a43c402379d7ccd2480416cadc7579f973f7",
"certificate-expiry": 1,
"utxo-pubkey": "02616c56412eb738a9eacfb0550b43a5a2e77e5d5205ea9e2ca8dfac34e50c9754",
"txid": "84c88fafe0bb75f507fe3bfb29a93d10b2e80c15a63b2943c1a5fecb5a55cba2",
"vout": 1,
"locktime": 1893456000
},
"J54LS6YyJPoseqFS",
"J55VZ6U6ZyFDNeuv"
)
)
ob = get_ob()
fidelity_bond_proof1, maker_nick1, taker_nick1 = fidelity_bond_info[0]
proof = FidelityBondProof(
maker_nick1, taker_nick1, hextobin(fidelity_bond_proof1['certificate-pubkey']),
fidelity_bond_proof1['certificate-expiry'],
hextobin(fidelity_bond_proof1['certificate-signature']),
(hextobin(fidelity_bond_proof1['txid']), fidelity_bond_proof1['vout']),
hextobin(fidelity_bond_proof1['utxo-pubkey']), fidelity_bond_proof1['locktime']
)
serialized1 = proof._serialize_proof_msg(fidelity_bond_proof1['nick-signature'])
ob.msgchan.nick = taker_nick1
ob.on_fidelity_bond_seen(maker_nick1, fidelity_bond_cmd_list[0], serialized1)
rows = ob.db.execute("SELECT * FROM fidelitybonds;").fetchall()
assert len(rows) == 1
#show the same fidelity bond message again, check it gets rejected as duplicate
ob.on_fidelity_bond_seen(maker_nick1, fidelity_bond_cmd_list[0], serialized1)
rows = ob.db.execute("SELECT * FROM fidelitybonds;").fetchall()
assert len(rows) == 1
#show a different fidelity bond and check it does get accepted
fidelity_bond_proof2, maker_nick2, taker_nick2 = fidelity_bond_info[1]
proof2 = FidelityBondProof(
maker_nick1, taker_nick1, hextobin(fidelity_bond_proof2['certificate-pubkey']),
fidelity_bond_proof2['certificate-expiry'],
hextobin(fidelity_bond_proof2['certificate-signature']),
(hextobin(fidelity_bond_proof2['txid']), fidelity_bond_proof2['vout']),
hextobin(fidelity_bond_proof2['utxo-pubkey']), fidelity_bond_proof2['locktime']
)
serialized2 = proof2._serialize_proof_msg(fidelity_bond_proof2['nick-signature'])
ob.msgchan.nick = taker_nick2
ob.on_fidelity_bond_seen(maker_nick2, fidelity_bond_cmd_list[0], serialized2)
rows = ob.db.execute("SELECT * FROM fidelitybonds;").fetchall()
assert len(rows) == 2

368
scripts/obwatch/ob-watcher.py

@ -16,12 +16,18 @@ from future.moves.urllib.parse import parse_qs
from decimal import Decimal
from optparse import OptionParser
from twisted.internet import reactor
from datetime import datetime
if sys.version_info < (3, 7):
print("ERROR: this script requires at least python 3.7")
exit(1)
from jmbase.support import EXIT_FAILURE
from jmbase import bintohex
from jmclient import FidelityBondMixin, get_interest_rate
from jmclient.fidelity_bond import FidelityBondProof
import sybil_attack_calculations as sybil
from jmbase import get_log
log = get_log()
@ -86,31 +92,45 @@ def cjfee_display(cjfee, order, btc_unit, rel_unit):
return str(Decimal(cjfee) * Decimal(rel_unit_to_factor[rel_unit])) + rel_unit
def satoshi_to_unit(sat, order, btc_unit, rel_unit):
power = unit_to_power[btc_unit]
def satoshi_to_unit_power(sat, power):
return ("%." + str(power) + "f") % float(
Decimal(sat) / Decimal(10 ** power))
def satoshi_to_unit(sat, order, btc_unit, rel_unit):
return satoshi_to_unit_power(sat, unit_to_power[btc_unit])
def order_str(s, order, btc_unit, rel_unit):
return str(s)
def create_table_heading(btc_unit, rel_unit):
def create_offerbook_table_heading(btc_unit, rel_unit):
col = ' <th>{1}</th>\n' # .format(field,label)
tableheading = '<table class="tftable sortable" border="1">\n <tr>' + ''.join(
[
col.format('ordertype', 'Type'), col.format(
'counterparty', 'Counterparty'),
col.format('ordertype', 'Type'),
col.format('counterparty', 'Counterparty'),
col.format('oid', 'Order ID'),
col.format('cjfee', 'Fee'), col.format(
'txfee', 'Miner Fee Contribution / ' + btc_unit),
col.format(
'minsize', 'Minimum Size / ' + btc_unit), col.format(
'maxsize', 'Maximum Size / ' + btc_unit)
col.format('cjfee', 'Fee'),
col.format('txfee', 'Miner Fee Contribution / ' + btc_unit),
col.format('minsize', 'Minimum Size / ' + btc_unit),
col.format('maxsize', 'Maximum Size / ' + btc_unit),
col.format('bondvalue', 'Bond value / ' + btc_unit + '&#xb2;')
]) + ' </tr>'
return tableheading
def create_bonds_table_heading(btc_unit):
tableheading = ('<table class="tftable sortable" border="1"><tr>'
+ '<th>Counterparty</th>'
+ '<th>UTXO</th>'
+ '<th>Bond value / ' + btc_unit + '&#xb2;</th>'
+ '<th>Locktime</th>'
+ '<th>Locked coins / ' + btc_unit + '</th>'
+ '<th>Confirmation time</th>'
+ '<th>Signature expiry height</th>'
+ '<th>Redeem script</th>'
+ '</tr>'
)
return tableheading
def create_choose_units_form(selected_btc, selected_rel):
choose_units_form = (
@ -128,6 +148,52 @@ def create_choose_units_form(selected_btc, selected_rel):
'<option selected="selected">' + selected_rel)
return choose_units_form
def get_fidelity_bond_data(taker):
with taker.dblock:
fbonds = taker.db.execute("SELECT * FROM fidelitybonds;").fetchall()
blocks = jm_single().bc_interface.get_current_block_height()
mediantime = jm_single().bc_interface.get_best_block_median_time()
interest_rate = get_interest_rate()
bond_utxo_set = set()
fidelity_bond_data = []
bond_outpoint_conf_times = []
fidelity_bond_values = []
for fb in fbonds:
try:
parsed_bond = FidelityBondProof.parse_and_verify_proof_msg(fb["counterparty"],
fb["takernick"], fb["proof"])
except ValueError:
continue
bond_utxo_data = FidelityBondMixin.get_validated_timelocked_fidelity_bond_utxo(
parsed_bond.utxo, parsed_bond.utxo_pub, parsed_bond.locktime, parsed_bond.cert_expiry,
blocks)
if bond_utxo_data == None:
continue
#check for duplicated utxos i.e. two or more makers using the same UTXO
# which is obviously not allowed, a fidelity bond must only be usable by one maker nick
utxo_str = parsed_bond.utxo[0] + b":" + str(parsed_bond.utxo[1]).encode("ascii")
if utxo_str in bond_utxo_set:
continue
bond_utxo_set.add(utxo_str)
fidelity_bond_data.append((parsed_bond, bond_utxo_data))
conf_time = jm_single().bc_interface.get_block_time(
jm_single().bc_interface.get_block_hash(
blocks - bond_utxo_data["confirms"] + 1
)
)
bond_outpoint_conf_times.append(conf_time)
bond_value = FidelityBondMixin.calculate_timelocked_fidelity_bond_value(
bond_utxo_data["value"],
conf_time,
parsed_bond.locktime,
mediantime,
interest_rate)
fidelity_bond_values.append(bond_value)
return (fidelity_bond_data, fidelity_bond_values, bond_outpoint_conf_times)
class OrderbookPageRequestHeader(http.server.SimpleHTTPRequestHandler):
def __init__(self, request, client_address, base_server):
@ -138,15 +204,21 @@ class OrderbookPageRequestHeader(http.server.SimpleHTTPRequestHandler):
directory=os.path.dirname(os.path.realpath(__file__)))
def create_orderbook_obj(self):
try:
self.taker.dblock.acquire(True)
with self.taker.dblock:
rows = self.taker.db.execute('SELECT * FROM orderbook;').fetchall()
finally:
self.taker.dblock.release()
if not rows:
fbonds = self.taker.db.execute("SELECT * FROM fidelitybonds;").fetchall()
if not rows or not fbonds:
return []
result = []
if jm_single().bc_interface != None:
(fidelity_bond_data, fidelity_bond_values, bond_outpoint_conf_times) =\
get_fidelity_bond_data(self.taker)
fidelity_bond_values_dict = dict([(bond_data["counterparty"], bond_value)
for (bond_data, _), bond_value in zip(fidelity_bond_data, fidelity_bond_values)])
else:
fidelity_bond_values_dict = {}
offers = []
for row in rows:
o = dict(row)
if 'cjfee' in o:
@ -155,8 +227,19 @@ class OrderbookPageRequestHeader(http.server.SimpleHTTPRequestHandler):
o['cjfee'] = int(o['cjfee'])
else:
o['cjfee'] = str(Decimal(o['cjfee']))
result.append(o)
return result
o["fidelity_bond_value"] = fidelity_bond_values_dict.get(o["counterparty"], 0)
offers.append(o)
BIN_KEYS = ["txid", "utxopubkey"]
fidelitybonds = []
for fbond in fbonds:
o = dict(fbond)
for k in BIN_KEYS:
o[k] = bintohex(o[k])
o["fidelity_bond_value"] = fidelity_bond_values_dict.get(o["counterparty"], 0)
fidelitybonds.append(o)
return {"offers": offers, "fidelitybonds": fidelitybonds}
def create_depth_chart(self, cj_amount, args=None):
if 'matplotlib' not in sys.modules:
@ -233,6 +316,166 @@ class OrderbookPageRequestHeader(http.server.SimpleHTTPRequestHandler):
return get_graph_html(fig) + ("<br/><a href='?scale=log'>log scale</a>" if
bins == 30 else "<br/><a href='?'>linear</a>")
def create_fidelity_bond_table(self, btc_unit):
if jm_single().bc_interface == None:
with self.taker.dblock:
fbonds = self.taker.db.execute("SELECT * FROM fidelitybonds;").fetchall()
fidelity_bond_data = []
for fb in fbonds:
try:
proof = FidelityBondProof.parse_and_verify_proof_msg(
fb["counterparty"],
fb["takernick"],
fb["proof"])
except ValueError:
proof = None
fidelity_bond_data.append((proof, None))
fidelity_bond_values = [-1]*len(fidelity_bond_data) #-1 means no data
bond_outpoint_conf_times = [-1]*len(fidelity_bond_data)
total_btc_committed_str = "unknown"
else:
(fidelity_bond_data, fidelity_bond_values, bond_outpoint_conf_times) =\
get_fidelity_bond_data(self.taker)
total_btc_committed_str = satoshi_to_unit(
sum([utxo_data["value"] for _, utxo_data in fidelity_bond_data]),
None, btc_unit, 0)
RETARGET_INTERVAL = 2016
elem = lambda e: "<td>" + e + "</td>"
bondtable = ""
for (bond_data, utxo_data), bond_value, conf_time in zip(
fidelity_bond_data, fidelity_bond_values, bond_outpoint_conf_times):
if bond_value == -1 or conf_time == -1 or utxo_data == None:
bond_value_str = "No data"
conf_time_str = "No data"
utxo_value_str = "No data"
else:
bond_value_str = satoshi_to_unit_power(bond_value, 2*unit_to_power[btc_unit])
conf_time_str = str(datetime.utcfromtimestamp(conf_time))
utxo_value_str = satoshi_to_unit(utxo_data["value"], None, btc_unit, 0)
bondtable += ("<tr>"
+ elem(bond_data.maker_nick)
+ elem(bintohex(bond_data.utxo[0]) + ":" + str(bond_data.utxo[1]))
+ elem(bond_value_str)
+ elem(datetime.utcfromtimestamp(bond_data.locktime).strftime("%Y-%m-%d"))
+ elem(utxo_value_str)
+ elem(conf_time_str)
+ elem(str(bond_data.cert_expiry*RETARGET_INTERVAL))
+ elem(bintohex(btc.mk_freeze_script(bond_data.utxo_pub,
bond_data.locktime)))
+ "</tr>"
)
heading2 = (str(len(fidelity_bond_data)) + " fidelity bonds found with "
+ total_btc_committed_str + " " + btc_unit
+ " total locked up")
choose_units_form = (
'<form method="get" action="">' +
'<select name="btcunit" onchange="this.form.submit();">' +
''.join(('<option>' + u + ' </option>' for u in sorted_units)) +
'</select></form>')
choose_units_form = choose_units_form.replace(
'<option>' + btc_unit,
'<option selected="selected">' + btc_unit)
decodescript_tip = ("<br/>Tip: try running the RPC <code>decodescript "
+ "&lt;redeemscript&gt;</code> as proof that the fidelity bond address matches the "
+ "locktime.<br/>Also run <code>gettxout &lt;utxo_txid&gt; &lt;utxo_vout&gt;</code> "
+ "as proof that the fidelity bond UTXO is real.")
return (heading2,
choose_units_form + create_bonds_table_heading(btc_unit) + bondtable + "</table>"
+ decodescript_tip)
def create_sybil_resistance_page(self, btc_unit):
if jm_single().bc_interface == None:
return "", "Calculations unavailable, requires configured bitcoin node."
(fidelity_bond_data, fidelity_bond_values, bond_outpoint_conf_times) =\
get_fidelity_bond_data(self.taker)
choose_units_form = (
'<form method="get" action="">' +
'<select name="btcunit" onchange="this.form.submit();">' +
''.join(('<option>' + u + ' </option>' for u in sorted_units)) +
'</select></form>')
choose_units_form = choose_units_form.replace(
'<option>' + btc_unit,
'<option selected="selected">' + btc_unit)
mainbody = choose_units_form
honest_weight = sum(fidelity_bond_values)
mainbody += ("Assuming the makers in the offerbook right now are not sybil attackers, "
+ "how much would a sybil attacker starting now have to sacrifice to succeed in their"
+ " attack with 95% probability. Honest weight="
+ satoshi_to_unit_power(honest_weight, 2*unit_to_power[btc_unit]) + " " + btc_unit
+ "&#xb2;<br/>Also assumes that takers are not price-sensitive and that their max "
+ "coinjoin fee is configured high enough that they dont exclude any makers.")
heading2 = "Sybil attacks from external enemies."
mainbody += ('<table class="tftable" border="1"><tr>'
+ '<th>Maker count</th>'
+ '<th>6month locked coins / ' + btc_unit + '</th>'
+ '<th>1y locked coins / ' + btc_unit + '</th>'
+ '<th>2y locked coins / ' + btc_unit + '</th>'
+ '<th>5y locked coins / ' + btc_unit + '</th>'
+ '<th>10y locked coins / ' + btc_unit + '</th>'
+ '<th>Required burned coins / ' + btc_unit + '</th>'
+ '</tr>'
)
timelocks = [0.5, 1.0, 2.0, 5.0, 10.0, None]
interest_rate = get_interest_rate()
for makercount, unit_success_sybil_weight in sybil.successful_attack_95pc_sybil_weight.items():
success_sybil_weight = unit_success_sybil_weight * honest_weight
row = "<tr><td>" + str(makercount) + "</td>"
for timelock in timelocks:
if timelock != None:
coins_per_sybil = sybil.weight_to_locked_coins(success_sybil_weight,
interest_rate, timelock)
else:
coins_per_sybil = sybil.weight_to_burned_coins(success_sybil_weight)
row += ("<td>" + satoshi_to_unit(coins_per_sybil*makercount, None, btc_unit, 0)
+ "</td>")
row += "</tr>"
mainbody += row
mainbody += "</table>"
mainbody += ("<h2>Sybil attacks from enemies within</h2>Assume a sybil attack is ongoing"
+ " right now and that the counterparties with the most valuable fidelity bonds are "
+ " actually controlled by the same entity. Then, what is the probability of a "
+ " successful sybil attack for a given makercount, and what is the fidelity bond "
+ " value being foregone by not putting all bitcoins into just one maker.")
mainbody += ('<table class="tftable" border="1"><tr>'
+ '<th>Maker count</th>'
+ '<th>Success probability</th>'
+ '<th>Foregone value / ' + btc_unit + '&#xb2;</th>'
+ '</tr>'
)
#limited because calculation is slow, so this avoids server being too slow to respond
MAX_MAKER_COUNT_INTERNAL = 10
weights = sorted(fidelity_bond_values)[::-1]
for makercount in range(1, MAX_MAKER_COUNT_INTERNAL+1):
makercount_str = (str(makercount) + " - " + str(MAX_MAKER_COUNT_INTERNAL)
if makercount == len(fidelity_bond_data) and len(fidelity_bond_data) !=
MAX_MAKER_COUNT_INTERNAL else str(makercount))
success_prob = sybil.calculate_top_makers_sybil_attack_success_probability(weights,
makercount)
total_sybil_weight = sum(weights[:makercount])
sacrificed_values = [sybil.weight_to_burned_coins(w) for w in weights[:makercount]]
foregone_value = (sybil.coins_burned_to_weight(sum(sacrificed_values))
- total_sybil_weight)
mainbody += ("<tr><td>" + makercount_str + "</td><td>" + str(round(success_prob*100.0, 5))
+ "%</td><td>" + satoshi_to_unit_power(foregone_value, 2*unit_to_power[btc_unit])
+ "</td></tr>")
if makercount == len(weights):
break
mainbody += "</table>"
return heading2, mainbody
def create_orderbook_table(self, btc_unit, rel_unit):
result = ''
try:
@ -242,13 +485,59 @@ class OrderbookPageRequestHeader(http.server.SimpleHTTPRequestHandler):
self.taker.dblock.release()
if not rows:
return 0, result
#print("len rows before filter: " + str(len(rows)))
rows = [o for o in rows if o["ordertype"] in filtered_offername_list]
if jm_single().bc_interface == None:
for row in rows:
row["bondvalue"] = "No data"
else:
blocks = jm_single().bc_interface.get_current_block_height()
mediantime = jm_single().bc_interface.get_best_block_median_time()
interest_rate = get_interest_rate()
for row in rows:
with self.taker.dblock:
fbond_data = self.taker.db.execute(
"SELECT * FROM fidelitybonds WHERE counterparty=?;", (row["counterparty"],)
).fetchall()
if len(fbond_data) == 0:
row["bondvalue"] = "0"
continue
else:
try:
parsed_bond = FidelityBondProof.parse_and_verify_proof_msg(
fbond_data[0]["counterparty"],
fbond_data[0]["takernick"],
fbond_data[0]["proof"]
)
except ValueError:
row["bondvalue"] = "0"
continue
utxo_data = FidelityBondMixin.get_validated_timelocked_fidelity_bond_utxo(
parsed_bond.utxo, parsed_bond.utxo_pub, parsed_bond.locktime,
parsed_bond.cert_expiry, blocks)
if utxo_data == None:
row["bondvalue"] = "0"
continue
bond_value = FidelityBondMixin.calculate_timelocked_fidelity_bond_value(
utxo_data["value"],
jm_single().bc_interface.get_block_time(
jm_single().bc_interface.get_block_hash(
blocks - utxo_data["confirms"] + 1
)
),
parsed_bond.locktime,
mediantime,
interest_rate)
row["bondvalue"] = satoshi_to_unit_power(bond_value, 2*unit_to_power[btc_unit])
order_keys_display = (('ordertype', ordertype_display),
('counterparty', do_nothing), ('oid', order_str),
('cjfee', cjfee_display), ('txfee', satoshi_to_unit),
('counterparty', do_nothing),
('oid', order_str),
('cjfee', cjfee_display),
('txfee', satoshi_to_unit),
('minsize', satoshi_to_unit),
('maxsize', satoshi_to_unit))
('maxsize', satoshi_to_unit),
('bondvalue', do_nothing))
# somewhat complex sorting to sort by cjfee but with swabsoffers on top
@ -278,16 +567,15 @@ class OrderbookPageRequestHeader(http.server.SimpleHTTPRequestHandler):
def do_GET(self):
# http.server.SimpleHTTPRequestHandler.do_GET(self)
# print 'httpd received ' + self.path + ' request'
# print('httpd received ' + self.path + ' request')
self.path, query = self.path.split('?', 1) if '?' in self.path else (
self.path, '')
args = parse_qs(query)
pages = ['/', '/ordersize', '/depth', '/orderbook.json']
pages = ['/', '/fidelitybonds', '/ordersize', '/depth', '/sybilresistance',
'/orderbook.json']
static_files = {'/vendor/sorttable.js', '/vendor/bootstrap.min.css', '/vendor/jquery-3.5.1.slim.min.js'}
if self.path in static_files:
if self.path in static_files or self.path not in pages:
return super().do_GET()
elif self.path not in pages:
return
fd = open(os.path.join(os.path.dirname(os.path.realpath(__file__)),
'orderbook.html'), 'r')
orderbook_fmt = fd.read()
@ -308,7 +596,7 @@ class OrderbookPageRequestHeader(http.server.SimpleHTTPRequestHandler):
ordercount, ordertable = self.create_orderbook_table(
btc_unit, rel_unit)
choose_units_form = create_choose_units_form(btc_unit, rel_unit)
table_heading = create_table_heading(btc_unit, rel_unit)
table_heading = create_offerbook_table_heading(btc_unit, rel_unit)
replacements = {
'PAGETITLE': 'JoinMarket Browser Interface',
'MAINHEADING': 'JoinMarket Orderbook',
@ -319,6 +607,18 @@ class OrderbookPageRequestHeader(http.server.SimpleHTTPRequestHandler):
rotateObform + refresh_orderbook_form + choose_units_form +
table_heading + ordertable + '</table>\n')
}
elif self.path == '/fidelitybonds':
btc_unit = args['btcunit'][0] if 'btcunit' in args else sorted_units[0]
if btc_unit not in sorted_units:
btc_unit = sorted_units[0]
heading2, mainbody = self.create_fidelity_bond_table(btc_unit)
replacements = {
'PAGETITLE': 'JoinMarket Browser Interface',
'MAINHEADING': 'Fidelity Bonds',
'SECONDHEADING': heading2,
'MAINBODY': mainbody
}
elif self.path == '/ordersize':
replacements = {
'PAGETITLE': 'JoinMarket Browser Interface',
@ -340,6 +640,17 @@ class OrderbookPageRequestHeader(http.server.SimpleHTTPRequestHandler):
'SECONDHEADING': 'Orderbook Depth' + alert_msg,
'MAINBODY': '<br />'.join(mainbody)
}
elif self.path == '/sybilresistance':
btc_unit = args['btcunit'][0] if 'btcunit' in args else sorted_units[0]
if btc_unit not in sorted_units:
btc_unit = sorted_units[0]
heading2, mainbody = self.create_sybil_resistance_page(btc_unit)
replacements = {
'PAGETITLE': 'JoinMarket Browser Interface',
'MAINHEADING': 'Resistance to Sybil Attacks from Fidelity Bonds',
'SECONDHEADING': heading2,
'MAINBODY': mainbody
}
elif self.path == '/orderbook.json':
replacements = {}
orderbook_fmt = json.dumps(self.create_orderbook_obj())
@ -437,6 +748,7 @@ class ObIRCMessageChannel(IRCMessageChannel):
_chunks = command.split(" ")
try:
self.check_for_orders(nick, _chunks)
self.check_for_fidelity_bond(nick, _chunks)
except:
pass

2
scripts/obwatch/orderbook.html

@ -84,8 +84,10 @@
<div id="navbar" class="collapse navbar-collapse">
<ul class="nav navbar-nav">
<li class="active"><a href=".">Orders</a></li>
<li><a href="fidelitybonds">Fidelity Bonds</a></li>
<li><a href="ordersize">Size Distribution</a></li>
<li><a href="depth">Depth</a></li>
<li><a href="sybilresistance">Sybil resistance</a></li>
<li><a href="orderbook.json">Export orders</a></li>
<li><a target="_blank" href="https://github.com/JoinMarket-Org/joinmarket-clientserver/releases">New segwit version</a></li>
</ul>

81
scripts/obwatch/sybil_attack_calculations.py

@ -0,0 +1,81 @@
##this file calculates the success probability of a sybil attack on the
# orderbook with fidelity bonds used in joinmarket
# see https://gist.github.com/chris-belcher/87ebbcbb639686057a389acb9ab3e25b
#precomputed
#what sybil weight is required per-maker to sybil attack joinmarket with 95% success rate
#this is for when the honest weight (i.e. value of all fidelity bonds added up) equals 1
#however it is linear, so to calculate for another honest_weight just multiply
#see
#https://gist.github.com/chris-belcher/87ebbcbb639686057a389acb9ab3e25b#appendix-1---fit-to-unit-honest-weight-sybil-attack
successful_attack_95pc_sybil_weight = {
1: 19.2125,
2: 28.829523311823312,
3: 35.37299702466422,
4: 40.27618399827166,
5: 44.19631358837695,
6: 47.46160578701477,
7: 50.25944623742167,
8: 52.706868994753286,
9: 54.881852860047836,
10: 56.8389576639515,
11: 58.61784778500215,
12: 60.248261563672784,
13: 61.75306801,
14: 62.97189476,
15: 64.28155594,
16: 65.21832112385313,
17: 66.29765063354174,
18: 67.315269563541,
19: 68.27785449480159,
20: 69.19105386203657,
21: 70.05968878944397,
22: 70.88790716279642,
23: 71.67930342495613,
24: 72.43701285697972,
25: 73.16378660022
}
def descend_probability_tree(weights, remaining_descents, branch_probability):
if remaining_descents == 0:
return branch_probability
else:
total_weight = sum(weights)
result = 0
for i, w in enumerate(weights):
#honest makers are at index 0
if i == 0:
#an honest maker being chosen means the sybil attack failed
#so this branch contributes zero to the attack success prob
continue
if w == 0:
continue
weight_cache = weights[i]
weights[i] = 0
result += descend_probability_tree(weights,
remaining_descents-1, branch_probability*w/total_weight)
weights[i] = weight_cache
return result
def calculate_top_makers_sybil_attack_success_probability(weights, taker_peer_count):
honest_weight = sum(weights[taker_peer_count:])
weights = [honest_weight] + weights[:taker_peer_count]
return descend_probability_tree(weights, taker_peer_count, 1.0)
def weight_to_burned_coins(w):
#calculates how many coins need to be burned to produce a certain bond
return w**0.5
def weight_to_locked_coins(w, r, locktime_months):
#calculates how many coins need to be locked to produce a certain bond
return w**0.5 / r / locktime_months * 12
def coins_locked_to_weight(c, r, locktime_months):
return (c*r*locktime_months/12.0)**2
def coins_burned_to_weight(c):
return c*c

19
test/common.py

@ -12,7 +12,8 @@ sys.path.insert(0, os.path.join(data_dir))
from jmbase import get_log
from jmclient import open_test_wallet_maybe, BIP32Wallet, SegwitWallet, \
estimate_tx_fee, jm_single, WalletService, BaseWallet
estimate_tx_fee, jm_single, WalletService, BaseWallet, WALLET_IMPLEMENTATIONS
from jmclient.wallet_utils import get_configured_wallet_type
import jmbitcoin as btc
from jmbase import chunks
@ -65,13 +66,18 @@ def make_wallets(n,
test_wallet=False,
passwords=None,
walletclass=SegwitWallet,
mixdepths=5):
mixdepths=5,
fb_indices=[]):
'''n: number of wallets to be created
wallet_structure: array of n arrays , each subarray
specifying the number of addresses to be populated with coins
at each depth (for now, this will only populate coins into 'receive' addresses)
mean_amt: the number of coins (in btc units) in each address as above
sdev_amt: if randomness in amouts is desired, specify here.
fb_indices: a list of integers in range(n), and for each of those we will
use the fidelity bond wallet type (note: to actually *use* the FB feature
the calling code will have to create the FB utxo, itself). Only supported
if walletclass=SegwitWallet.
Returns: a dict of dicts of form {0:{'seed':seed,'wallet':Wallet object},1:..,}
Default Wallet constructor is joinmarket.Wallet, else use TestWallet,
which takes a password parameter as in the list passwords.
@ -93,9 +99,14 @@ def make_wallets(n,
pwd = passwords[i]
else:
pwd = None
if i in fb_indices:
assert walletclass == SegwitWallet, "Cannot use FB except for native segwit."
wc = WALLET_IMPLEMENTATIONS[get_configured_wallet_type(True)]
print("for index: {}, we got wallet type: {}".format(i, wc))
else:
wc = walletclass
w = open_test_wallet_maybe(seeds[i], seeds[i], mixdepths - 1,
test_wallet_cls=walletclass)
test_wallet_cls=wc)
wallet_service = WalletService(w)
wallets[i + start_index] = {'seed': seeds[i].decode('ascii'),

69
test/ygrunner.py

@ -11,14 +11,25 @@
--btcpwd=123456abcdef --btcconf=/blah/bitcoin.conf \
--nirc=2 -s test/ygrunner.py
'''
from twisted.internet import task, reactor
from common import make_wallets
import pytest
import random
from datetime import datetime
from jmbase import jmprint
from jmclient import YieldGeneratorBasic, load_test_config, jm_single,\
JMClientProtocolFactory, start_reactor, SegwitWallet,\
JMClientProtocolFactory, start_reactor, SegwitWallet, WalletService,\
SegwitLegacyWallet, cryptoengine, SNICKERClientProtocolFactory, SNICKERReceiver
from jmclient.wallet_utils import wallet_gettimelockaddress
# For quicker testing, restrict the range of timelock
# addresses to avoid slow load of multiple bots.
# Note: no need to revert this change as ygrunner runs
# in isolation.
from jmclient import FidelityBondMixin
FidelityBondMixin.TIMELOCK_ERA_YEARS = 2
FidelityBondMixin.TIMELOCK_EPOCH_YEAR = datetime.now().year
FidelityBondMixin.TIMENUMBERS_PER_PUBKEY = 12
class MaliciousYieldGenerator(YieldGeneratorBasic):
"""Overrides, randomly, some maker functions
@ -80,18 +91,19 @@ class DeterministicMaliciousYieldGenerator(YieldGeneratorBasic):
return (False, "malicious tx rejection")
return super().on_tx_received(nick, txhex, offerinfo)
@pytest.mark.parametrize(
"num_ygs, wallet_structures, mean_amt, malicious, deterministic",
"num_ygs, wallet_structures, fb_indices, mean_amt, malicious, deterministic",
[
# 1sp 3yg, honest makers
(3, [[1, 3, 0, 0, 0]] * 4, 2, 0, False),
# 1sp 3yg, honest makers, one maker has FB:
(3, [[1, 3, 0, 0, 0]] * 4, [1, 2], 2, 0, False),
# 1sp 3yg, malicious makers reject on auth and on tx 30% of time
#(3, [[1, 3, 0, 0, 0]] * 4, 2, 30, False),
# 1 sp 9 ygs, deterministically malicious 50% of time
#(9, [[1, 3, 0, 0, 0]] * 10, 2, 50, True),
])
def test_start_ygs(setup_ygrunner, num_ygs, wallet_structures, mean_amt,
malicious, deterministic):
def test_start_ygs(setup_ygrunner, num_ygs, wallet_structures, fb_indices,
mean_amt, malicious, deterministic):
"""Set up some wallets, for the ygs and 1 sp.
Then start the ygs in background and publish
the seed of the sp wallet for easy import into -qt
@ -105,7 +117,8 @@ def test_start_ygs(setup_ygrunner, num_ygs, wallet_structures, mean_amt,
wallet_services = make_wallets(num_ygs + 1,
wallet_structures=wallet_structures,
mean_amt=mean_amt,
walletclass=walletclass)
walletclass=walletclass,
fb_indices=fb_indices)
#the sendpayment bot uses the last wallet in the list
wallet_service = wallet_services[num_ygs]['wallet']
jmprint("\n\nTaker wallet seed : " + wallet_services[num_ygs]['seed'])
@ -160,12 +173,19 @@ def test_start_ygs(setup_ygrunner, num_ygs, wallet_structures, mean_amt,
else:
ygclass = MaliciousYieldGenerator
for i in range(num_ygs):
cfg = [txfee, cjfee_a, cjfee_r, ordertype, minsize, txfee_factor,
cjfee_factor, size_factor]
wallet_service_yg = wallet_services[i]["wallet"]
wallet_service_yg.startService()
yg = ygclass(wallet_service_yg, cfg)
if i in fb_indices:
# create a timelocked address and fund it;
# must be done after sync, so deferred:
wallet_service_yg.timelock_funded = False
sync_wait_loop = task.LoopingCall(get_addr_and_fund, yg)
sync_wait_loop.start(1.0, now=False)
if malicious:
yg.set_maliciousness(malicious, mtype="tx")
clientfactory = JMClientProtocolFactory(yg, proto_type="MAKER")
@ -183,6 +203,39 @@ def test_start_ygs(setup_ygrunner, num_ygs, wallet_structures, mean_amt,
clientfactory, snickerfactory=snicker_factory,
daemon=daemon, rs=rs)
def get_addr_and_fund(yg):
""" This function allows us to create
and publish a fidelity bond for a particular
yield generator object after the wallet has reached
a synced state and is therefore ready to serve up
timelock addresses. We create the TL address, fund it,
refresh the wallet and then republish our offers, which
will also publish the new FB.
"""
if not yg.wallet_service.synced:
return
if yg.wallet_service.timelock_funded:
return
addr = wallet_gettimelockaddress(yg.wallet_service.wallet, "2021-11")
print("Got timelockaddress: {}".format(addr))
# pay into it; amount is randomized for now.
# Note that grab_coins already mines 1 block.
fb_amt = random.randint(1, 5)
jm_single().bc_interface.grab_coins(addr, fb_amt)
# we no longer have to run this loop (TODO kill with nonlocal)
yg.wallet_service.timelock_funded = True
# force wallet to check for the new coins so the new
# yg offers will include them:
yg.wallet_service.transaction_monitor()
# publish a new offer:
yg.offerlist = yg.create_my_orders()
yg.fidelity_bond = yg.get_fidelity_bond_template()
jmprint('updated offerlist={}'.format(yg.offerlist))
@pytest.fixture(scope="module")
def setup_ygrunner():
load_test_config()

Loading…
Cancel
Save