diff --git a/docs/fidelity-bonds.md b/docs/fidelity-bonds.md index a1e3470..8028fba 100644 --- a/docs/fidelity-bonds.md +++ b/docs/fidelity-bonds.md @@ -3,8 +3,6 @@ Fidelity bonds are a feature of JoinMarket which improves the resistance to sybil attacks, and therefore improves the privacy of the system. -## This feature is incomplete and so is disabled for now - A fidelity bond is a mechanism where bitcoin value is deliberately sacrificed to make a cryptographic identity expensive to obtain. The sacrifice is done in a way that can be proven to a third party. Takers in JoinMarket will @@ -22,12 +20,8 @@ long-term holder (or hodler) of bitcoins can buy time-locked fidelity bonds essentially for free, assuming they never intended to transact with their coins anyway. -Another way to create fidelity bonds is to destroy coins by sending them to a -[OP_RETURN](https://en.bitcoin.it/wiki/Script#Provably_Unspendable.2FPrunable_Outputs) -output. - The private keys to fidelity bonds can be kept in [cold storage](https://en.bitcoin.it/wiki/Cold_storage) -for added security. +for added security. (Note: not implemented in JoinMarket) For a more detailed explanation of how fidelity bonds work see these documents: @@ -36,11 +30,35 @@ bonds](https://gist.github.com/chris-belcher/18ea0e6acdb885a2bfbdee43dcd6b5af/) * [Financial mathematics of JoinMarket fidelity bonds](https://gist.github.com/chris-belcher/87ebbcbb639686057a389acb9ab3e25b) -#### Note on privacy +## How to use fidelity bonds as a taker + +In JoinMarket version v0.9 or later takers will by default use fidelity bonds +without any input needed from the user. + +The orderbook watcher script now displays information about any fidelity bonds +advertised by makers, as well as calculating how strong the system is against +hypothetical sybil attackers. + +Some makers with high-valued fidelity bonds may choose to ask for a high coinjoin fee, so +for the strongest protection from sybil attacks make sure to set your maximum coinjoin fee +high enough (or if you think the sybil protection is too expensive then set the max fee +lower, as always its your choice as a taker in the market). + +Takers will still choose makers equally (i.e. without taking into account fidelity bonds) with a +small probability. By default this probability is 12.5%, so approximately 1-in-8 makers. This can +be changed in the config file with the option `bondless_makers_allowance`. -Bitcoin outputs which create fidelity bonds will be published to the entire -world, so before and after creating them make sure the outputs are not linked -to your identity in any way. Perhaps mix with JoinMarket before and after. +The previous algorithm for choosing makers without regards to fidelity bonds can still be used by +passing the relevant CLI option when running a script (for example +`python3 sendpayment.py -R wallet.jmdat
`). As always use `--help` to get a full +list of options. + + +## How to use fidelity bonds as a yield-generator + +You need to create a fidelity bond wallet and run the yield-generator script on it. The yield +generator will automatically announce the most valuable fidelity bond in its wallet. Fidelity bonds +are only supported for native segwit wallets. ### Creating a JoinMarket wallet which supports fidelity bonds @@ -56,7 +74,7 @@ will offer an option to make the wallet support fidelity bonds. Would you like this wallet to support fidelity bonds? write 'n' if you don't know what this is (y/n): y Write down this wallet recovery mnemonic - evidence initial knee image inspire plug dad midnight blast awkward clean between + use amateur twelve unfair weekend file canal frog cotton play renew illegal Generated wallet OK @@ -65,6 +83,27 @@ as a backup. It is also recommended to write down the name of the creating walle "JoinMarket" and that the fidelity bond option was enabled. Writing the wallet creation date is also useful as it can help with rescanning. +#### Adding fidelity bonds to an existing wallet + +On any **native segwit** wallet this can be done by using the `recover` method: + + (jmvenv) $ python3 wallet-tool.py recover + +And then choosing `yes` to create a fidelity bond wallet. + +#### Note on privacy + +Bitcoin transactions which create fidelity bonds will be published to the entire world, so before +creating them make sure the coins are not linked to any of your privacy-relevant information. +Perhaps mix with JoinMarket. Also, use a sweep transaction which does not create a change output +when funding the timelocked address. Change addresses can also leak privacy information and the +best way to avoid that is to not create change outputs at all i.e. use only sweep transactions. + +Once the timelocked addresses expire and become spendable, make sure you don't leak any information +then either, mix afterwards as well. If your timelocked address expires and you want to send the +coins to another timelocked address then you don't need to mix in between, because no +privacy-relevant information linked to you has been leaked. + ### Obtaining time-locked addresses The `wallet-tool.py` script supports a new method `gettimelockaddress` used for @@ -72,13 +111,14 @@ obtaining time-locked addresses. If coins are sent to these addresses they will be locked up until the timelock date passes. Only mixdepth zero can have fidelity bonds in it. -This example creates an address which locks any coins sent to it until April 2020. +This example creates an address which locks any coins sent to it until January 2025. - (jmvenv) $ python3 wallet-tool.py testfidelity.jmdat gettimelockaddress 2020-4 + (jmvenv) $ python3 wallet-tool.py testfidelity.jmdat gettimelockaddress 2025-1 Enter wallet decryption passphrase: - path = m/49'/1'/0'/2/3:1585699200 - Coins sent to this address will be not be spendable until April 2020. Full date: 2020-04-01 00:00:00 - bcrt1qrc2qu3m2l2spayu5kr0k0rnn9xgjz46zsxmruh87a3h3f5zmnkaqlfx7v5 + path = m/84'/1'/0'/2/0:1748736000 + Coins sent to this address will be not be spendable until June 2025. Full date: 2025-06-01 00:00:00 + bcrt1qvcjcggpcw2rzk4sks94r3nxj5xhgkqm4p9h54c7mtr695se27efqqxnu0k + If coins are sent to these addresses they will appear in the usual `wallet-tool.py` display: @@ -86,21 +126,24 @@ display: (jmvenv) $ python3 wallet-tool.py -m 0 testfidelity.jmdat Enter wallet decryption passphrase: JM wallet - mixdepth 0 fbonds-mpk-tpubDDCbCPdf5wJVGYWB4mZr3E3Lys4NBcEKysrrUrLfhG6sekmrvs6KZNe4i5p5z3FyfwRmKMqB9NWEcEUiTS4LwqfrKPQzhKj6aLihu2EejaU - external addresses m/49'/1'/0'/0 tpubDEGdmPwmQRcZmGKhaudjch9Fgw4J5yP4bYw5B8LoSDkMdmhBxM4ndEQXHK4r1TPexGjLidxdpeEzsJcdXEe7khWToxCZuN6JiLzvUoHAki2 - m/49'/1'/0'/0/0 2N8jHuQaApgFtQ8UKxKbREAvNxKn4BGX4x2 0.00000000 new - m/49'/1'/0'/0/1 2Mx5CwDoNcuCT38EDmgenQxv9skHbZfXFdo 0.00000000 new - m/49'/1'/0'/0/2 2N1tNTTwNyucGGmfDWNVk3AUi3i5S8jVKqn 0.00000000 new - m/49'/1'/0'/0/3 2N8eBEU5wpWb6kS1gvbRgewtxsmXsMkShV6 0.00000000 new - m/49'/1'/0'/0/4 2MuHgeSgMsvkcn6aGNW2uk2UXP3xVVnkfh2 0.00000000 new - m/49'/1'/0'/0/5 2NA8d8um5KmBNNR8dadhbEDYGiTJPFCdjMB 0.00000000 new + mixdepth 0 fbonds-mpk-tpubDCv7SSisuV4AqNgRqHcXFAeCjV9Z5SPZgSVFjzydEZrS5Jg1uCkv4wGfMZc3wEaiC2hkEfbD753u4R6Shpgj8bR1kuKnEciB522kSpQ3j1v + external addresses m/84'/1'/0'/0 tpubDEdbDAFbNCXXN54M2qgzHBJYxkHK9hoeisyRsC2gC3WZuxziU5RkcJWgpw7nJqugPx26Ui9c2AqCy9gwZpgTtL1GW1TuPtKRX2SdrrjBY2W + m/84'/1'/0'/0/0 bcrt1qwmj5yht2xxr3juxczt453uqjltc3xdyklkjnjt 2.00000000 used + m/84'/1'/0'/0/1 bcrt1q99nzc6s8fh37rjju8gfws4dcfhrpcfz0jst829 0.00000000 new + m/84'/1'/0'/0/2 bcrt1ql3fxzq2ueyhm8kwy7du6nsv7fgpvujupd2emms 0.00000000 new + m/84'/1'/0'/0/3 bcrt1qgxr5mh8v4dj8kuqv98ckjzqdmzd4xjcn539nc6 0.00000000 new + m/84'/1'/0'/0/4 bcrt1qhyhwzkh60p26pk2v008ejqhcl8g70h5vuw2fn6 0.00000000 new + m/84'/1'/0'/0/5 bcrt1q9xuzrqql028wpj3933zyny6geg2u75rhaygv6z 0.00000000 new + m/84'/1'/0'/0/6 bcrt1q8w7ewzl4q8mwxx7erf7pjq36z5g088jxzqjdcn 0.00000000 new + Balance: 2.00000000 + internal addresses m/84'/1'/0'/1 Balance: 0.00000000 - internal addresses m/49'/1'/0'/1 + internal addresses m/84'/1'/0'/2 tpubDEdbDAFbNCXXSFfUKS5QAaxN9toxv8pFSn3TxRdhEij46wj88RCch7ZBA2fgqsocD7MZqowVAdm6LyYumKuKZbzT4V2CfudwDicrMnqqbjC + m/84'/1'/0'/2/0:1748736000 bcrt1qvcjcggpcw2rzk4sks94r3nxj5xhgkqm4p9h54c7mtr695se27efqqxnu0k 2.00000000 2025-06-01 [LOCKED] + Balance: 2.00000000 + internal addresses m/84'/1'/0'/3 tpubDEdbDAFbNCXXUpShWMdtPMDcAogMyZJVAzMMn3wM9rC364sdeUuFMS7ZmdjoMbkf14jeK56uy95UBR2SA9AFFeoVv4j4CqMeaq1tcuBVkZe Balance: 0.00000000 - internal addresses m/49'/1'/0'/2 tpubDEGdmPwmQRcZrzjRmUFqXXyLdRedwxCWQviAFqDe6sXJeZzRNTwmwqMfxN6Ka3v7hEebstrU5kqUNoHsFKaA3RoB2vopL6kLHVo1EQq6USw - m/49'/1'/0'/2/0:1585699200 bcrt1qrc2qu3m2l2spayu5kr0k0rnn9xgjz46zsxmruh87a3h3f5zmnkaqlfx7v5 0.15000000 2020-04-01 [LOCKED] - Balance: 0.15000000 - Balance for mixdepth 0: 0.15000000 + Balance for mixdepth 0: 4.00000000 #### Spending time-locked coins @@ -111,10 +154,117 @@ JoinMarket's coin control feature, so before spending you need to unfreeze the coins using `python3 wallet-tool.py -m 0 freeze`. Once unfrozen and untimelocked the coins can be spent normally with the scripts -`sendpayment.py`, `tumber.py`, or yield generator. +`sendpayment.py`, `tumber.py`, or yield generator. NB You cannot export the private keys (which is +always disadvised, anyway) of timelocked addresses to any other wallets, as they use custom scripts. +You must spend them from JoinMarket itself. + +### How many coins to lock up and for how long? + +Fidelity bonds UTXOs are valuable as soon as they confirmed. The simplified formula for a fidelity +bond value with locked coins is: + + bond_value = (locked_coins * exp(interest_rate * locktime))^2 + +A few important things to notice: +* The bond value goes as the _square_ of sacrificed value. For example if your sacrificed value is +5 BTC then the fidelity bond value is 25 (because 5 x 5 = 25). If instead you sacrificed 6 BTC the +value is 36 (because 6 x 6 = 36). The point of this is to create an incentive for makers to lump +all their coins into just one bot rather than spreading it over many bots. It makes a sybil attack +much more expensive. +* The longer you lock for the greater the value. The value increases as the `interest_rate`, which +is configurable in the config file with the option `interest_rate`. By default it is 1.5% per +annum and because of tyranny-of-the-default takers are unlikely to change it. This value is probably +not too far from the "real" interest rate, and the system still works fine even if the real rate +is something like 3% or 0.1%. +* The above formula would suggest that if you lock 3 BTC for 10000 years you get a fidelity +bond worth `1.7481837557171304e+131` (17 followed by 130 zeros). This does not happen because the +sacrificed value is capped at the value of the burned coins. So in this example the fidelity bond +value would be just 9 (equal to 3x3 or 3 squared). This feature is not included in the above +simplified equation. +* After the locktime expires and the coins are free to move, the fidelity bond will continue to be +valuable, but its value will exponentially drop following the interest rate. So it would be good +for you as a yield generator to create a transaction with the UTXO spending it to another +time-locked address, but it's not a huge rush (specifically, there's likely no need to pay massive +miner fees, you can probably wait until fees are low). + +The full details on valuing a time-locked fidelity bond are [found in the relevant section of the +"Financial mathematics of fidelity bonds" document](https://gist.github.com/chris-belcher/87ebbcbb639686057a389acb9ab3e25b#time-locked-fidelity-bonds). + +At any time you can use the orderbook watcher script to see your own fidelity bond value. + +Consider also the [warning on the bitcoin wiki page on timelocks](https://en.bitcoin.it/wiki/Timelock#Far-future_locks). + +I would recommend locking as many coins as you are comfortable with for a period of between 6 +months and 2 years. Perhaps at the very start lock for only 1 month or 2 months(?) It's a +marketplace and the rules are known to all, so ultimately you'll have to make your own decision. + +### Can I add coins to a fidelity bond that already exists? + +No. Sending more coins to the same timelocked address will not add to the existing fidelity bond, +but instead create a new one with the same locktime. + +### Creating watch-only fidelity bond wallets + +#### Note: Fidelity bond in cold storage cannot be advertised to takers right now. You can create watch-only fidelity bond wallets but cant advertise them yet. This feature is pretty easy to add though, and can be done without changing the JoinMarket protocol. + +Fidelity bonds can be held on an offline computer in +[cold storage](https://en.bitcoin.it/wiki/Cold_storage). To do this we create +a watch-only fidelity bond wallet. + +When fidelity bonds are displayed in `wallet-tool.py`, their master public key +is highlighted with a prefix `fbonds-mpk-`. + +This master public key can be used to create a watch-only wallet using +`wallet-tool.py`. + + $ python3 wallet-tool.py createwatchonly fbonds-mpk-tpubDDCbCPdf5wJVGYWB4mZr3E3Lys4NBcEKysrrUrLfhG6sekmrvs6KZNe4i5p5z3FyfwRmKMqB9NWEcEUiTS4LwqfrKPQzhKj6aLihu2EejaU + Input wallet file name (default: watchonly.jmdat): watchfidelity.jmdat + Enter wallet file encryption passphrase: + Reenter wallet file encryption passphrase: + Done + +Then the wallet can be displayed like a regular wallet, although only the zeroth +mixdepth will be shown. + + $ python3 wallet-tool.py watchfidelity.jmdat + User data location: . + Enter wallet decryption passphrase: + JM wallet + mixdepth 0 fbonds-mpk-tpubDCv7SSisuV4AqNgRqHcXFAeCjV9Z5SPZgSVFjzydEZrS5Jg1uCkv4wGfMZc3wEaiC2hkEfbD753u4R6Shpgj8bR1kuKnEciB522kSpQ3j1v + external addresses m/84'/1'/0'/0 tpubDEdbDAFbNCXXN54M2qgzHBJYxkHK9hoeisyRsC2gC3WZuxziU5RkcJWgpw7nJqugPx26Ui9c2AqCy9gwZpgTtL1GW1TuPtKRX2SdrrjBY2W + m/84'/1'/0'/0/0 bcrt1qwmj5yht2xxr3juxczt453uqjltc3xdyklkjnjt 2.00000000 used + m/84'/1'/0'/0/1 bcrt1q99nzc6s8fh37rjju8gfws4dcfhrpcfz0jst829 0.00000000 new + m/84'/1'/0'/0/2 bcrt1ql3fxzq2ueyhm8kwy7du6nsv7fgpvujupd2emms 0.00000000 new + m/84'/1'/0'/0/3 bcrt1qgxr5mh8v4dj8kuqv98ckjzqdmzd4xjcn539nc6 0.00000000 new + m/84'/1'/0'/0/4 bcrt1qhyhwzkh60p26pk2v008ejqhcl8g70h5vuw2fn6 0.00000000 new + m/84'/1'/0'/0/5 bcrt1q9xuzrqql028wpj3933zyny6geg2u75rhaygv6z 0.00000000 new + m/84'/1'/0'/0/6 bcrt1q8w7ewzl4q8mwxx7erf7pjq36z5g088jxzqjdcn 0.00000000 new + Balance: 2.00000000 + internal addresses m/84'/1'/0'/1 + Balance: 0.00000000 + internal addresses m/84'/1'/0'/2 tpubDEdbDAFbNCXXSFfUKS5QAaxN9toxv8pFSn3TxRdhEij46wj88RCch7ZBA2fgqsocD7MZqowVAdm6LyYumKuKZbzT4V2CfudwDicrMnqqbjC + m/84'/1'/0'/2/0:1748736000 bcrt1qvcjcggpcw2rzk4sks94r3nxj5xhgkqm4p9h54c7mtr695se27efqqxnu0k 2.00000000 2025-06-01 [LOCKED] + Balance: 2.00000000 + internal addresses m/84'/1'/0'/3 tpubDEdbDAFbNCXXUpShWMdtPMDcAogMyZJVAzMMn3wM9rC364sdeUuFMS7ZmdjoMbkf14jeK56uy95UBR2SA9AFFeoVv4j4CqMeaq1tcuBVkZe + Balance: 0.00000000 + Balance for mixdepth 0: 4.00000000 + +### BIP32 Paths + +Fidelity bond wallets extend the BIP32 path format to include the locktime +values. In this example we've got `m/49'/1'/0'/2/0:1583020800` where the +number after the colon is the locktime value in Unix time. + +This path can be passed to certain wallet methods like `dumpprivkey`. + + $ python3 wallet-tool.py -H "m/49'/1'/0'/2/0:1583020800" testfidelity.jmdat dumpprivkey + Enter wallet decryption passphrase: + cNEuE5ypNTxVFCyC5iH7u5AQTrddamcUHRPNweiLvmHUWd6XXDkz ### Burning coins +#### Note: There is no point using this feature. Fidelity bonds in JoinMarket cannot be created by burning coins right now. This feature is here only for historical reasons. + Coins can be burned with a special method of the `sendpayment.py` script. Set the destination to be `BURN`. Transactions which burn coins must only have one input and one output, so use coin control to freeze all coins in the zeroth @@ -210,59 +360,3 @@ The `-H` flag must point to the path containing the burn output. Then synchronizing the wallet won't output the no-merkle-proof warning. -### Creating watch-only fidelity bond wallets - -Fidelity bonds can be held on an offline computer in -[cold storage](https://en.bitcoin.it/wiki/Cold_storage). To do this we create -a watch-only fidelity bond wallet. - -When fidelity bonds are displayed in `wallet-tool.py`, their master public key -is highlighted with a prefix `fbonds-mpk-`. - -This master public key can be used to create a watch-only wallet using -`wallet-tool.py`. - - $ python3 wallet-tool.py createwatchonly fbonds-mpk-tpubDDCbCPdf5wJVGYWB4mZr3E3Lys4NBcEKysrrUrLfhG6sekmrvs6KZNe4i5p5z3FyfwRmKMqB9NWEcEUiTS4LwqfrKPQzhKj6aLihu2EejaU - Input wallet file name (default: watchonly.jmdat): watchfidelity.jmdat - Enter wallet file encryption passphrase: - Reenter wallet file encryption passphrase: - Done - -Then the wallet can be displayed like a regular wallet, although only the zeroth -mixdepth will be shown. - - $ python3 wallet-tool.py watchfidelity.jmdat - User data location: . - Enter wallet decryption passphrase: - JM wallet - mixdepth 0 fbonds-mpk-tpubDDCbCPdf5wJVGYWB4mZr3E3Lys4NBcEKysrrUrLfhG6sekmrvs6KZNe4i5p5z3FyfwRmKMqB9NWEcEUiTS4LwqfrKPQzhKj6aLihu2EejaU - external addresses m/49'/1'/0'/0 tpubDEGdmPwmQRcZmGKhaudjch9Fgw4J5yP4bYw5B8LoSDkMdmhBxM4ndEQXHK4r1TPexGjLidxdpeEzsJcdXEe7khWToxCZuN6JiLzvUoHAki2 - m/49'/1'/0'/0/0 2N8jHuQaApgFtQ8UKxKbREAvNxKn4BGX4x2 0.00000000 used - m/49'/1'/0'/0/1 2Mx5CwDoNcuCT38EDmgenQxv9skHbZfXFdo 0.00000000 new - m/49'/1'/0'/0/2 2N1tNTTwNyucGGmfDWNVk3AUi3i5S8jVKqn 0.00000000 new - m/49'/1'/0'/0/3 2N8eBEU5wpWb6kS1gvbRgewtxsmXsMkShV6 0.00000000 new - m/49'/1'/0'/0/4 2MuHgeSgMsvkcn6aGNW2uk2UXP3xVVnkfh2 0.00000000 new - m/49'/1'/0'/0/5 2NA8d8um5KmBNNR8dadhbEDYGiTJPFCdjMB 0.00000000 new - m/49'/1'/0'/0/6 2NG76BAHPccfyy6sH68EHrB9QJBycx3FKb6 0.00000000 new - Balance: 0.25000000 - internal addresses m/49'/1'/0'/1 - Balance: 0.00000000 - internal addresses m/49'/1'/0'/2 tpubDEGdmPwmQRcZrzjRmUFqXXyLdRedwxCWQviAFqDe6sXJeZzRNTwmwqMfxN6Ka3v7hEebstrU5kqUNoHsFKaA3RoB2vopL6kLHVo1EQq6USw - m/49'/1'/0'/0/3:1585699200 bcrt1qrc2qu3m2l2spayu5kr0k0rnn9xgjz46zsxmruh87a3h3f5zmnkaqlfx7v5 0.15000000 2020-04-01 [UNLOCKED] - Balance: 0.15000000 - internal addresses m/49'/1'/0'/3 tpubDEGdmPwmQRcZuX3uNrCouu5bRgp2GJcoQTvhkFAJMTA3yxhKmQyeGwecbnkms4DYmBhCJn2fGTuejTe3g8oyJW3qKcfB4b3Swj2hDk1h4Y2 - Balance: 0.00000000 - Balance for mixdepth 0: 0.15000000 - -### BIP32 Paths - -Fidelity bond wallets extend the BIP32 path format to include the locktime -values. In this example we've got `m/49'/1'/0'/2/0:1583020800` where the -number after the colon is the locktime value in Unix time. - -This path can be passed to certain wallet methods like `dumpprivkey`. - - $ python3 wallet-tool.py -H "m/49'/1'/0'/2/0:1583020800" testfidelity.jmdat dumpprivkey - Enter wallet decryption passphrase: - cNEuE5ypNTxVFCyC5iH7u5AQTrddamcUHRPNweiLvmHUWd6XXDkz - diff --git a/docs/release-notes/release-notes-fidelity-bonds.md b/docs/release-notes/release-notes-fidelity-bonds.md new file mode 100644 index 0000000..d7a76aa --- /dev/null +++ b/docs/release-notes/release-notes-fidelity-bonds.md @@ -0,0 +1,19 @@ + +Notable changes +=============== + +### Fidelity bond for improving sybil attack resistance + +From the very beginning of JoinMarket it was possible to attack the system by creating many many maker bots all controlled by the same person. If an unlucky taker came along and created a coinjoin only with those fake maker bots then their coinjoins could be easily unmixed. This is called a sybil attack and until now it was relatively cheap to do against JoinMarket. Some yield generators were already doing this by running multiple bots, because they could earn higher coinjoin fees from their multiple makers. + +Fidelity bonds are a new feature intended to make this sybil attack a lot more expensive. It works by allowing JoinMarket makers to lock up bitcoins into time locked addresses. Takers will still choose makers to coinjoin with randomly but they have a greater chance of choosing makers who have advertised more valuable fidelity bonds. Any sybil attacker then has to lock up many many bitcoins into time locked addresses. + +For full details of the scheme see: [Design for improving JoinMarket's resistance to sybil attacks using fidelity bonds](https://gist.github.com/chris-belcher/18ea0e6acdb885a2bfbdee43dcd6b5af/) + +This release implements all the features needed to add fidelity bonds to JoinMarket. Takers (via scripts such as `sendpayment.py` or `tumbler.py` or the Joinmarket-Qt app) will automatically give preference to makers who advertise fidelity bonds. Makers can optionally update their wallets to fidelity bond wallets. When a fidelity bond wallet is used with a yield generator script, it will automatically announce its fidelity bond publicly. Makers who don't create fidelity bonds by locking up bitcoins will still be chosen for coinjoins occasionally, but probably much less often than before. + +For full user documentation see the file `/docs/`fidelity-bonds.md` in the repository. + +With realistic assumptions we have calculated that an adversary would need to lock up around 50000 bitcoins for 6 months in order to sybil attack the JoinMarket system with 95% success rate. Now that fidelity bonds are being added to JoinMarket for real we can see how the system behaves in practice. + +Fidelity bond coins cannot be yet be held in cold storage, but this is easy to add later because the JoinMarket protocol is set up in a way that the change would be backward-compatible. diff --git a/jmbase/jmbase/commands.py b/jmbase/jmbase/commands.py index b7138e3..ec039c3 100644 --- a/jmbase/jmbase/commands.py +++ b/jmbase/jmbase/commands.py @@ -6,6 +6,7 @@ Used for AMP asynchronous messages. from twisted.protocols.amp import Boolean, Command, Integer, Unicode from .bigstring import BigUnicode + class DaemonNotReady(Exception): pass @@ -45,7 +46,8 @@ class JMSetup(JMCommand): role, passes initial offers for announcement (for TAKER, this data is "none") """ arguments = [(b'role', Unicode()), - (b'initdata', Unicode())] + (b'offers', Unicode()), + (b'use_fidelity_bond', Boolean())] class JMMsgSignature(JMCommand): """A response to a request for a bitcoin signature @@ -107,6 +109,11 @@ class JMAnnounceOffers(JMCommand): (b'to_cancel', Unicode()), (b'offerlist', Unicode())] +class JMFidelityBondProof(JMCommand): + """Send requested fidelity bond proof message""" + arguments = [(b'nick', Unicode()), + (b'proof', Unicode())] + class JMIOAuth(JMCommand): """Send contents of !ioauth message after verifying Taker's auth message @@ -182,7 +189,8 @@ class JMOffers(JMCommand): """Return the entire contents of the orderbook to TAKER, as a json-ified dict. """ - arguments = [(b'orderbook', BigUnicode())] + arguments = [(b'orderbook', BigUnicode()), + (b'fidelitybonds', BigUnicode())] class JMFillResponse(JMCommand): """Returns ioauth data from MAKER if successful. @@ -200,6 +208,11 @@ class JMSigReceived(JMCommand): """MAKER-specific commands """ +class JMFidelityBondProofRequest(JMCommand): + """MAKER wants to announce a fidelity bond proof message""" + arguments = [(b'takernick', Unicode()), + (b'makernick', Unicode())] + class JMAuthReceived(JMCommand): """Return the commitment and revelation provided in !fill, !auth by the TAKER, @@ -376,4 +389,4 @@ class BIP78InfoMsg(JMCommand): from the daemon about current status at network level. """ - arguments = [(b'infomsg', Unicode())] \ No newline at end of file + arguments = [(b'infomsg', Unicode())] diff --git a/jmbase/test/test_commands.py b/jmbase/test/test_commands.py index 12bceca..231ba19 100644 --- a/jmbase/test/test_commands.py +++ b/jmbase/test/test_commands.py @@ -63,8 +63,8 @@ class JMTestServerProtocol(JMBaseProtocol): return {'accepted': True} @JMSetup.responder - def on_JM_SETUP(self, role, initdata): - show_receipt("JMSETUP", role, initdata) + def on_JM_SETUP(self, role, offers, use_fidelity_bond): + show_receipt("JMSETUP", role, offers, use_fidelity_bond) d = self.callRemote(JMSetupDone) self.defaultCallbacks(d) return {'accepted': True} @@ -75,7 +75,8 @@ class JMTestServerProtocol(JMBaseProtocol): #build a huge orderbook to test BigString Argument orderbook = ["aaaa" for _ in range(2**15)] d = self.callRemote(JMOffers, - orderbook=json.dumps(orderbook)) + orderbook=json.dumps(orderbook), + fidelitybonds="dummyfidelitybonds") self.defaultCallbacks(d) return {'accepted': True} @@ -156,7 +157,8 @@ class JMTestClientProtocol(JMBaseProtocol): show_receipt("JMUP") d = self.callRemote(JMSetup, role="TAKER", - initdata="none") + offers="{}", + use_fidelity_bond=False) self.defaultCallbacks(d) return {'accepted': True} @@ -177,8 +179,8 @@ class JMTestClientProtocol(JMBaseProtocol): return {'accepted': True} @JMOffers.responder - def on_JM_OFFERS(self, orderbook): - show_receipt("JMOFFERS", orderbook) + def on_JM_OFFERS(self, orderbook, fidelitybonds): + show_receipt("JMOFFERS", orderbook, fidelitybonds) d = self.callRemote(JMFill, amount=100, commitment="dummycommitment", diff --git a/jmclient/jmclient/__init__.py b/jmclient/jmclient/__init__.py index 1e1d6a3..1a900fa 100644 --- a/jmclient/jmclient/__init__.py +++ b/jmclient/jmclient/__init__.py @@ -14,7 +14,7 @@ from .taker import Taker from .wallet import (Mnemonic, estimate_tx_fee, WalletError, BaseWallet, ImportWalletMixin, BIP39WalletMixin, BIP32Wallet, BIP49Wallet, LegacyWallet, SegwitWallet, SegwitLegacyWallet, FidelityBondMixin, - FidelityBondWatchonlyWallet, SegwitLegacyWalletFidelityBonds, + FidelityBondWatchonlyWallet, SegwitWalletFidelityBonds, UTXOManager, WALLET_IMPLEMENTATIONS, compute_tx_locktime) from .storage import (Argon2Hash, Storage, StorageError, RetryableStorageError, StoragePasswordError, VolatileStorage) @@ -24,7 +24,7 @@ from .configure import (load_test_config, process_shutdown, load_program_config, jm_single, get_network, update_persist_config, validate_address, is_burn_destination, get_irc_mchannels, get_blockchain_interface_instance, set_config, is_segwit_mode, - is_native_segwit_mode, JMPluginService) + is_native_segwit_mode, JMPluginService, get_interest_rate, get_bondless_makers_allowance) from .blockchaininterface import (BlockchainInterface, RegtestBitcoinCoreInterface, BitcoinCoreInterface) from .snicker_receiver import SNICKERError, SNICKERReceiver diff --git a/jmclient/jmclient/blockchaininterface.py b/jmclient/jmclient/blockchaininterface.py index d4fe081..cf5fbe1 100644 --- a/jmclient/jmclient/blockchaininterface.py +++ b/jmclient/jmclient/blockchaininterface.py @@ -190,7 +190,7 @@ class BitcoinCoreInterface(BlockchainInterface): def get_block(self, blockheight): """Returns full serialized block at a given height. """ - block_hash = self._rpc('getblockhash', [blockheight]) + block_hash = self.get_block_hash(blockheight) block = self._rpc('getblock', [block_hash, False]) if not block: return False @@ -503,6 +503,9 @@ class BitcoinCoreInterface(BlockchainInterface): def get_block_time(self, blockhash): return self._get_block_header_data(blockhash, 'time') + def get_block_hash(self, height): + return self._rpc("getblockhash", [height]) + def get_tx_merkle_branch(self, txid, blockhash=None): if not blockhash: tx = self._rpc("gettransaction", [txid]) @@ -522,7 +525,7 @@ class BitcoinCoreInterface(BlockchainInterface): return core_proof[80:] def verify_tx_merkle_branch(self, txid, block_height, merkle_branch): - block_hash = self._rpc("getblockhash", [block_height]) + block_hash = self.get_block_hash(block_height) core_proof = self._rpc("getblockheader", [block_hash, False]) + \ binascii.hexlify(merkle_branch).decode() ret = self._rpc("verifytxoutproof", [core_proof]) diff --git a/jmclient/jmclient/cli_options.py b/jmclient/jmclient/cli_options.py index 5840cd8..562e2f0 100644 --- a/jmclient/jmclient/cli_options.py +++ b/jmclient/jmclient/cli_options.py @@ -16,7 +16,8 @@ options which are common to more than one script in a base class. order_choose_algorithms = { 'random_under_max_order_choose': '-R', 'cheapest_order_choose': '-C', - 'weighted_order_choose': '-W' + 'weighted_order_choose': '-W', + 'fidelity_bond_weighted_order_choose': '-F' } def add_base_options(parser): @@ -90,12 +91,12 @@ def add_common_options(parser): '--order-choose-algorithm', action='callback', type='string', - default=jmclient.support.random_under_max_order_choose, + default=jmclient.support.fidelity_bond_weighted_order_choose, callback=get_order_choose_algorithm, help="Set the algorithm to use for selecting orders from the order book.\n" "Default: {}\n" "Available options: {}" - .format('random_under_max_order_choose', + .format('fidelity_bond_weighted_order_choose', ', '.join(order_choose_algorithms.keys())), dest='order_choose_fn') add_order_choose_short_options(parser) @@ -130,9 +131,9 @@ def get_order_choose_algorithm(option, opt_str, value, parser, value_kw=None): The following defaults are maintained as accessed via functions for flexibility. TODO This should be moved from this module.""" -MAX_DEFAULT_REL_FEE = 0.001 -MIN_MAX_DEFAULT_ABS_FEE = 1000 -MAX_MAX_DEFAULT_ABS_FEE = 10000 +MAX_DEFAULT_REL_FEE = 0.004 +MIN_MAX_DEFAULT_ABS_FEE = 4000 +MAX_MAX_DEFAULT_ABS_FEE = 40000 def get_default_max_relative_fee(): return MAX_DEFAULT_REL_FEE diff --git a/jmclient/jmclient/client_protocol.py b/jmclient/jmclient/client_protocol.py index 822345e..45155c9 100644 --- a/jmclient/jmclient/client_protocol.py +++ b/jmclient/jmclient/client_protocol.py @@ -394,7 +394,8 @@ class JMMakerClientProtocol(JMClientProtocol): self.offers_ready_loop.stop() d = self.callRemote(commands.JMSetup, role="MAKER", - initdata=json.dumps(self.client.offerlist)) + offers=json.dumps(self.client.offerlist), + use_fidelity_bond=(self.client.fidelity_bond is not None)) self.defaultCallbacks(d) @commands.JMSetupDone.responder @@ -427,6 +428,17 @@ class JMMakerClientProtocol(JMClientProtocol): maker_timeout_sec=maker_timeout_sec) self.defaultCallbacks(d) + @commands.JMFidelityBondProofRequest.responder + def on_JM_FIDELITY_BOND_PROOF_REQUEST(self, takernick, makernick): + proof_msg = (self.client.fidelity_bond + .create_proof(makernick, takernick) + .create_proof_msg(self.client.fidelity_bond.cert_privkey)) + d = self.callRemote(commands.JMFidelityBondProof, + nick=takernick, + proof=proof_msg) + self.defaultCallbacks(d) + return {"accepted": True} + @commands.JMAuthReceived.responder def on_JM_AUTH_RECEIVED(self, nick, offer, commitment, revelation, amount, kphex): @@ -627,7 +639,8 @@ class JMTakerClientProtocol(JMClientProtocol): def on_JM_UP(self): d = self.callRemote(commands.JMSetup, role="TAKER", - initdata="none") + offers="{}", + use_fidelity_bond=False) self.defaultCallbacks(d) return {'accepted': True} @@ -676,11 +689,12 @@ class JMTakerClientProtocol(JMClientProtocol): return {'accepted': True} @commands.JMOffers.responder - def on_JM_OFFERS(self, orderbook): + def on_JM_OFFERS(self, orderbook, fidelitybonds): self.orderbook = json.loads(orderbook) + fidelity_bonds_list = json.loads(fidelitybonds) #Removed for now, as judged too large, even for DEBUG: #jlog.debug("Got the orderbook: " + str(self.orderbook)) - retval = self.client.initialize(self.orderbook) + retval = self.client.initialize(self.orderbook, fidelity_bonds_list) #format of retval is: #True, self.cjamount, commitment, revelation, self.filtered_orderbook) if not retval[0]: diff --git a/jmclient/jmclient/configure.py b/jmclient/jmclient/configure.py index 8caa7c2..97955c5 100644 --- a/jmclient/jmclient/configure.py +++ b/jmclient/jmclient/configure.py @@ -88,6 +88,10 @@ required_options = {'BLOCKCHAIN': ['blockchain_source', 'network'], 'POLICY': ['absurd_fee_per_kb', 'taker_utxo_retries', 'taker_utxo_age', 'taker_utxo_amtpercent']} +_DEFAULT_INTEREST_RATE = "0.015" + +_DEFAULT_BONDLESS_MAKERS_ALLOWANCE = "0.125" + defaultconfig = \ """ [DAEMON] @@ -291,6 +295,21 @@ minimum_makers = 4 # whatever the value of it, and this is set with the value -1. max_sats_freeze_reuse = -1 +# Interest rate used when calculating the value of fidelity bonds created +# by locking bitcoins in timelocked addresses +# See also: +# https://gist.github.com/chris-belcher/87ebbcbb639686057a389acb9ab3e25b#determining-interest-rate-r +# Set as a real number, i.e. 1 = 100% and 0.01 = 1% +interest_rate = """ + _DEFAULT_INTEREST_RATE + """ + +# Some makers run their bots to mix their funds not just to earn money +# So to improve privacy very slightly takers dont always choose a maker based +# on his fidelity bond but allow a certain small percentage to be chosen completely +# randomly without taking into account fidelity bonds +# This parameter sets how many makers on average will be chosen regardless of bonds +# A real number, i.e. 1 = 100%, 0.125 = 1/8 = 1 in every 8 makers on average will be bondless +bondless_makers_allowance = """ + _DEFAULT_BONDLESS_MAKERS_ALLOWANCE + """ + ############################## #THE FOLLOWING SETTINGS ARE REQUIRED TO DEFEND AGAINST SNOOPERS. #DON'T ALTER THEM UNLESS YOU UNDERSTAND THE IMPLICATIONS. @@ -538,6 +557,14 @@ _BURN_DESTINATION = "BURN" def is_burn_destination(destination): return destination == _BURN_DESTINATION +def get_interest_rate(): + return float(global_singleton.config.get('POLICY', 'interest_rate', + fallback=_DEFAULT_INTEREST_RATE)) + +def get_bondless_makers_allowance(): + return float(global_singleton.config.get('POLICY', 'bondless_makers_allowance', + fallback=_DEFAULT_BONDLESS_MAKERS_ALLOWANCE)) + def remove_unwanted_default_settings(config): for section in config.sections(): if section.startswith('MESSAGING:'): diff --git a/jmclient/jmclient/cryptoengine.py b/jmclient/jmclient/cryptoengine.py index 3059f38..f5d2a2a 100644 --- a/jmclient/jmclient/cryptoengine.py +++ b/jmclient/jmclient/cryptoengine.py @@ -12,8 +12,8 @@ from .configure import get_network, jm_single #with fidelity bond wallets and watchonly fidelity bond wallet, the wallet class # can have two engines, one for single-sig addresses and the other for timelocked addresses TYPE_P2PKH, TYPE_P2SH_P2WPKH, TYPE_P2WPKH, TYPE_P2SH_M_N, TYPE_TIMELOCK_P2WSH, \ - TYPE_SEGWIT_LEGACY_WALLET_FIDELITY_BONDS, TYPE_WATCHONLY_FIDELITY_BONDS, \ - TYPE_WATCHONLY_TIMELOCK_P2WSH, TYPE_WATCHONLY_P2SH_P2WPKH = range(9) + TYPE_SEGWIT_WALLET_FIDELITY_BONDS, TYPE_WATCHONLY_FIDELITY_BONDS, \ + TYPE_WATCHONLY_TIMELOCK_P2WSH, TYPE_WATCHONLY_P2WPKH = range(9) NET_MAINNET, NET_TESTNET, NET_SIGNET = range(3) NET_MAP = {'mainnet': NET_MAINNET, 'testnet': NET_TESTNET, 'signet': NET_SIGNET} @@ -407,7 +407,7 @@ class BTC_Watchonly_Timelocked_P2WSH(BTC_Timelocked_P2WSH): hashcode=btc.SIGHASH_ALL, **kwargs): raise RuntimeError("Cannot spend from watch-only wallets") -class BTC_Watchonly_P2SH_P2WPKH(BTC_P2SH_P2WPKH): +class BTC_Watchonly_P2WPKH(BTC_P2WPKH): @classmethod def derive_bip32_privkey(cls, master_key, path): @@ -426,7 +426,7 @@ class BTC_Watchonly_P2SH_P2WPKH(BTC_P2SH_P2WPKH): @classmethod def derive_bip32_pub_export(cls, master_key, path): - return super(BTC_Watchonly_P2SH_P2WPKH, cls).derive_bip32_pub_export( + return super(BTC_Watchonly_P2WPKH, cls).derive_bip32_pub_export( master_key, BTC_Watchonly_Timelocked_P2WSH.get_watchonly_path(path)) @classmethod @@ -440,5 +440,6 @@ ENGINES = { TYPE_P2WPKH: BTC_P2WPKH, TYPE_TIMELOCK_P2WSH: BTC_Timelocked_P2WSH, TYPE_WATCHONLY_TIMELOCK_P2WSH: BTC_Watchonly_Timelocked_P2WSH, - TYPE_WATCHONLY_P2SH_P2WPKH: BTC_Watchonly_P2SH_P2WPKH + TYPE_WATCHONLY_P2WPKH: BTC_Watchonly_P2WPKH, + TYPE_SEGWIT_WALLET_FIDELITY_BONDS: BTC_P2WPKH } diff --git a/jmclient/jmclient/fidelity_bond.py b/jmclient/jmclient/fidelity_bond.py new file mode 100644 index 0000000..e59a6bb --- /dev/null +++ b/jmclient/jmclient/fidelity_bond.py @@ -0,0 +1,136 @@ +import struct +import base64 +import json +from jmbitcoin import ecdsa_sign, ecdsa_verify +from jmdaemon import fidelity_bond_sanity_check + + +def assert_is_utxo(utxo): + assert len(utxo) == 2 + assert isinstance(utxo[0], bytes) + assert len(utxo[0]) == 32 + assert isinstance(utxo[1], int) + assert utxo[1] >= 0 + + +def get_cert_msg(cert_pub, cert_expiry): + return b'fidelity-bond-cert|' + cert_pub + b'|' + str(cert_expiry).encode('ascii') + + +class FidelityBond: + def __init__(self, utxo, utxo_pubkey, locktime, cert_expiry, + cert_privkey, cert_pubkey, cert_signature): + assert_is_utxo(utxo) + assert isinstance(utxo_pubkey, bytes) + assert isinstance(locktime, int) + assert isinstance(cert_expiry, int) + assert isinstance(cert_privkey, bytes) + assert isinstance(cert_pubkey, bytes) + assert isinstance(cert_signature, bytes) + self.utxo = utxo + self.utxo_pubkey = utxo_pubkey + self.locktime = locktime + self.cert_expiry = cert_expiry + self.cert_privkey = cert_privkey + self.cert_pubkey = cert_pubkey + self.cert_signature = cert_signature + + def create_proof(self, maker_nick, taker_nick): + return FidelityBondProof( + maker_nick, taker_nick, self.cert_pubkey, self.cert_expiry, + self.cert_signature, self.utxo, self.utxo_pubkey, self.locktime) + + def serialize(self): + return json.dumps([ + self.utxo, + self.utxo_pubkey, + self.locktime, + self.cert_expiry, + self.cert_privkey, + self.cert_pubkey, + self.cert_signature, + ]) + + @classmethod + def deserialize(cls, data): + return cls(*json.loads(data)) + + +class FidelityBondProof: + # nick_sig + cert_sig + cert_pubkey + cert_expiry + utxo_pubkey + txid + vout + timelock + # 72 + 72 + 33 + 2 + 33 + 32 + 4 + 4 = 252 bytes + SER_STUCT_FMT = '<72s72s33sH33s32sII' + + def __init__(self, maker_nick, taker_nick, cert_pub, cert_expiry, + cert_sig, utxo, utxo_pub, locktime): + assert isinstance(maker_nick, str) + assert isinstance(taker_nick, str) + assert isinstance(cert_pub, bytes) + assert isinstance(cert_sig, bytes) + assert isinstance(utxo_pub, bytes) + assert isinstance(locktime, int) + assert_is_utxo(utxo) + self.maker_nick = maker_nick + self.taker_nick = taker_nick + self.cert_pub = cert_pub + self.cert_expiry = cert_expiry + self.cert_sig = cert_sig + self.utxo = utxo + self.utxo_pub = utxo_pub + self.locktime = locktime + + @property + def nick_msg(self): + return (self.taker_nick + '|' + self.maker_nick).encode('ascii') + + def create_proof_msg(self, cert_priv): + nick_sig = ecdsa_sign(self.nick_msg, cert_priv) + # FIXME: remove stupid base64 + nick_sig = base64.b64decode(nick_sig) + return self._serialize_proof_msg(nick_sig) + + def _serialize_proof_msg(self, msg_signature): + msg_signature = msg_signature.rjust(72, b'\xff') + cert_sig = self.cert_sig.rjust(72, b'\xff') + fidelity_bond_data = struct.pack( + self.SER_STUCT_FMT, + msg_signature, + cert_sig, + self.cert_pub, + self.cert_expiry, + self.utxo_pub, + self.utxo[0], + self.utxo[1], + self.locktime + ) + return base64.b64encode(fidelity_bond_data).decode('ascii') + + @staticmethod + def _verify_signature(message, signature, pubkey): + # FIXME: remove stupid base64 + return ecdsa_verify(message, base64.b64encode(signature), pubkey) + + @classmethod + def parse_and_verify_proof_msg(cls, maker_nick, taker_nick, data): + if not fidelity_bond_sanity_check.fidelity_bond_sanity_check(data): + raise ValueError("sanity check failed") + decoded_data = base64.b64decode(data) + + unpacked_data = struct.unpack(cls.SER_STUCT_FMT, decoded_data) + try: + signature = unpacked_data[0][unpacked_data[0].index(b'\x30'):] + cert_sig = unpacked_data[1][unpacked_data[1].index(b'\x30'):] + except ValueError: + #raised if index() doesnt find the position + raise ValueError("der signature header not found") + proof = cls(maker_nick, taker_nick, unpacked_data[2], unpacked_data[3], + cert_sig, (unpacked_data[5], unpacked_data[6]), + unpacked_data[4], unpacked_data[7]) + cert_msg = get_cert_msg(proof.cert_pub, proof.cert_expiry) + + if not cls._verify_signature(proof.nick_msg, signature, proof.cert_pub): + raise ValueError("nick sig does not verify") + if not cls._verify_signature(cert_msg, proof.cert_sig, proof.utxo_pub): + raise ValueError("cert sig does not verify") + + return proof diff --git a/jmclient/jmclient/maker.py b/jmclient/jmclient/maker.py index 93d5c78..b535a86 100644 --- a/jmclient/jmclient/maker.py +++ b/jmclient/jmclient/maker.py @@ -21,6 +21,7 @@ class Maker(object): self.wallet_service = wallet_service self.nextoid = -1 self.offerlist = None + self.fidelity_bond = None self.sync_wait_loop = task.LoopingCall(self.try_to_create_my_orders) # don't fire on the first tick since reactor is still starting up # and may not shutdown appropriately if we immediately recognize @@ -38,6 +39,7 @@ class Maker(object): if not self.wallet_service.synced: return self.offerlist = self.create_my_orders() + self.fidelity_bond = self.get_fidelity_bond_template() self.sync_wait_loop.stop() if not self.offerlist: jlog.info("Failed to create offers, giving up.") @@ -277,3 +279,11 @@ class Maker(object): """Performs actions on receipt of 1st confirmation of a transaction into a block (e.g. announce orders) """ + + def get_fidelity_bond_template(self): + """ + Generates information about a fidelity bond which will be announced + By default returns no fidelity bond + Does not contain nick signature which has to be calculated individually + """ + return None diff --git a/jmclient/jmclient/support.py b/jmclient/jmclient/support.py index 4177eea..e8641a0 100644 --- a/jmclient/jmclient/support.py +++ b/jmclient/jmclient/support.py @@ -2,6 +2,7 @@ from functools import reduce import random from jmbase.support import get_log from decimal import Decimal +from .configure import get_bondless_makers_allowance from math import exp @@ -218,6 +219,26 @@ def cheapest_order_choose(orders, n): """ return orders[0] +def fidelity_bond_weighted_order_choose(orders, n): + """ + choose orders based on fidelity bond for improved sybil resistance + + * with probability `bondless_makers_allowance`: will revert to previous default + order choose (random_under_max_order_choose) + * with probability `1 - bondless_makers_allowance`: if there are no bond offerings, revert + to previous default as above. If there are, choose randomly from those, with weighting + being the fidelity bond values. + """ + + if random.random() < get_bondless_makers_allowance(): + return random_under_max_order_choose(orders, n) + #remove orders without fidelity bonds + filtered_orders = list(filter(lambda x: x[0]["fidelity_bond_value"] != 0, orders)) + if len(filtered_orders) == 0: + return random_under_max_order_choose(orders, n) + weights = list(map(lambda x: x[0]["fidelity_bond_value"], filtered_orders)) + weights = [x / sum(weights) for x in weights] + return filtered_orders[rand_weighted_choice(len(filtered_orders), weights)] def _get_is_within_max_limits(max_fee_rel, max_fee_abs, cjvalue): def check_max_fee(fee): diff --git a/jmclient/jmclient/taker.py b/jmclient/jmclient/taker.py index a018b27..2115c09 100644 --- a/jmclient/jmclient/taker.py +++ b/jmclient/jmclient/taker.py @@ -7,13 +7,14 @@ from typing import Any, NamedTuple from twisted.internet import reactor, task import jmbitcoin as btc -from jmclient.configure import jm_single, validate_address +from jmclient.configure import jm_single, validate_address, get_interest_rate from jmbase import get_log, bintohex, hexbin from jmclient.support import (calc_cj_fee, weighted_order_choose, choose_orders, choose_sweep_orders) -from jmclient.wallet import estimate_tx_fee, compute_tx_locktime +from jmclient.wallet import estimate_tx_fee, compute_tx_locktime, FidelityBondMixin from jmclient.podle import generate_podle, get_podle_commitments from jmclient.wallet_service import WalletService +from jmclient.fidelity_bond import FidelityBondProof from .output import generate_podle_error_string from .cryptoengine import EngineError from .schedule import NO_ROUNDING @@ -166,7 +167,7 @@ class Taker(object): return self.honest_only = truefalse - def initialize(self, orderbook): + def initialize(self, orderbook, fidelity_bonds_info): """Once the daemon is active and has returned the current orderbook, select offers, re-initialize variables and prepare a commitment, then send it to the protocol to fill offers. @@ -227,6 +228,11 @@ class Taker(object): self.latest_tx = None self.txid = None + fidelity_bond_values = calculate_fidelity_bond_values(fidelity_bonds_info) + for offer in orderbook: + #having no fidelity bond is like having a zero value fidelity bond + offer["fidelity_bond_value"] = fidelity_bond_values.get(offer["counterparty"], 0) + sweep = True if self.cjamount == 0 else False if not self.filter_orderbook(orderbook, sweep): return (False,) @@ -987,3 +993,41 @@ def round_to_significant_figures(d, sf): sigfiged = int(round(d/power10*sf_power10)*power10/sf_power10) return sigfiged raise RuntimeError() + +def calculate_fidelity_bond_values(fidelity_bonds_info): + if len(fidelity_bonds_info) == 0: + return {} + interest_rate = get_interest_rate() + blocks = jm_single().bc_interface.get_current_block_height() + mediantime = jm_single().bc_interface.get_best_block_median_time() + + validated_bonds = {} + for bond_data in fidelity_bonds_info: + try: + fb_proof = FidelityBondProof.parse_and_verify_proof_msg( + bond_data["counterparty"], bond_data["takernick"], bond_data["proof"]) + except ValueError: + continue + if fb_proof.utxo in validated_bonds: + continue + utxo_data = FidelityBondMixin.get_validated_timelocked_fidelity_bond_utxo( + fb_proof.utxo, fb_proof.utxo_pub, fb_proof.locktime, + fb_proof.cert_expiry, blocks) + if utxo_data is not None: + validated_bonds[fb_proof.utxo] = (fb_proof, utxo_data) + + fidelity_bond_values = { + bond_data.maker_nick: + FidelityBondMixin.calculate_timelocked_fidelity_bond_value( + utxo_data["value"], + jm_single().bc_interface.get_block_time( + jm_single().bc_interface.get_block_hash( + blocks - utxo_data["confirms"] + 1 + ) + ), + bond_data.locktime, + mediantime, + interest_rate) + for bond_data, utxo_data in validated_bonds.values() + } + return fidelity_bond_values diff --git a/jmclient/jmclient/wallet.py b/jmclient/jmclient/wallet.py index 9f21e35..39e21c5 100644 --- a/jmclient/jmclient/wallet.py +++ b/jmclient/jmclient/wallet.py @@ -17,6 +17,7 @@ from hashlib import sha256 from itertools import chain from decimal import Decimal from numbers import Integral +from math import exp from .configure import jm_single @@ -24,8 +25,8 @@ from .blockchaininterface import INF_HEIGHT from .support import select_gradual, select_greedy, select_greediest, \ select from .cryptoengine import TYPE_P2PKH, TYPE_P2SH_P2WPKH,\ - TYPE_P2WPKH, TYPE_TIMELOCK_P2WSH, TYPE_SEGWIT_LEGACY_WALLET_FIDELITY_BONDS,\ - TYPE_WATCHONLY_FIDELITY_BONDS, TYPE_WATCHONLY_TIMELOCK_P2WSH, TYPE_WATCHONLY_P2SH_P2WPKH,\ + TYPE_P2WPKH, TYPE_TIMELOCK_P2WSH, TYPE_SEGWIT_WALLET_FIDELITY_BONDS,\ + TYPE_WATCHONLY_FIDELITY_BONDS, TYPE_WATCHONLY_TIMELOCK_P2WSH, TYPE_WATCHONLY_P2WPKH,\ ENGINES from .support import get_random_bytes from . import mn_encode, mn_decode @@ -404,10 +405,10 @@ class BaseWallet(object): """ if self.TYPE == TYPE_P2PKH: return 'p2pkh' - elif self.TYPE in (TYPE_P2SH_P2WPKH, - TYPE_SEGWIT_LEGACY_WALLET_FIDELITY_BONDS): + elif self.TYPE == TYPE_P2SH_P2WPKH: return 'p2sh-p2wpkh' - elif self.TYPE == TYPE_P2WPKH: + elif self.TYPE in (TYPE_P2WPKH, + TYPE_SEGWIT_WALLET_FIDELITY_BONDS): return 'p2wpkh' assert False @@ -1249,7 +1250,11 @@ class PSBTWalletMixin(object): privkeys = [] for k, v in self._utxos._utxo.items(): for k2, v2 in v.items(): - privkeys.append(self._get_key_from_path(v2[0])) + key = self._get_key_from_path(v2[0]) + if FidelityBondMixin.is_timelocked_path(v2[0]) and len(key[0]) == 2: + #key is ((privkey, locktime), engine) for timelocked addrs + key = (key[0][0], key[1]) + privkeys.append(key) jmckeys = list(btc.JMCKey(x[0][:-1]) for x in privkeys) new_keystore = btc.KeyStore.from_iterable(jmckeys) @@ -2177,7 +2182,7 @@ class FidelityBondMixin(object): For example, if TIMENUMBER_UNIT = 2 (i.e. every time number is two months) then there are 6 timelocks per year so just 600 possible - addresses per century per pubkey. Easily searchable when recovering a + addresses per century. Easily searchable when recovering a wallet from seed phrase. Therefore the user doesn't need to store any dates, the seed phrase is sufficent for recovery. """ @@ -2190,16 +2195,8 @@ class FidelityBondMixin(object): TIMELOCK_EPOCH_MONTH = 1 #january MONTHS_IN_YEAR = 12 - TIMELOCK_ERA_YEARS = 30 - TIMENUMBERS_PER_PUBKEY = TIMELOCK_ERA_YEARS * MONTHS_IN_YEAR // TIMENUMBER_UNIT - - """ - As each pubkey corresponds to hundreds of addresses, to reduce load the - given gap limit will be reduced by this factor. Also these timelocked - addresses are never handed out to takers so there wont be a problem of - having many used addresses with no transactions on them. - """ - TIMELOCK_GAP_LIMIT_REDUCTION_FACTOR = 6 + TIMELOCK_ERA_YEARS = 80 + TIMENUMBER_COUNT = TIMELOCK_ERA_YEARS * MONTHS_IN_YEAR // TIMENUMBER_UNIT _TIMELOCK_ENGINE = ENGINES[TYPE_TIMELOCK_P2WSH] @@ -2217,7 +2214,7 @@ class FidelityBondMixin(object): """ converts a time number to a unix timestamp """ - if not 0 <= timenumber < cls.TIMENUMBERS_PER_PUBKEY: + if not 0 <= timenumber < cls.TIMENUMBER_COUNT: raise ValueError() year = cls.TIMELOCK_EPOCH_YEAR + (timenumber*cls.TIMENUMBER_UNIT) // cls.MONTHS_IN_YEAR month = cls.TIMELOCK_EPOCH_MONTH + (timenumber*cls.TIMENUMBER_UNIT) % cls.MONTHS_IN_YEAR @@ -2238,7 +2235,7 @@ class FidelityBondMixin(object): raise ValueError() timenumber = (dt.year - cls.TIMELOCK_EPOCH_YEAR)*(cls.MONTHS_IN_YEAR // cls.TIMENUMBER_UNIT) + ((dt.month - cls.TIMELOCK_EPOCH_MONTH) // cls.TIMENUMBER_UNIT) - if timenumber < 0 or timenumber > cls.TIMENUMBERS_PER_PUBKEY: + if timenumber < 0 or timenumber > cls.TIMENUMBER_COUNT: raise ValueError("datetime out of range") return timenumber @@ -2261,13 +2258,12 @@ class FidelityBondMixin(object): def _populate_script_map(self): super()._populate_script_map() - for md in self._index_cache: - address_type = self.BIP32_TIMELOCK_ID - for i in range(self._index_cache[md][address_type]): - for timenumber in range(self.TIMENUMBERS_PER_PUBKEY): - path = self.get_path(md, address_type, i, timenumber) - script = self.get_script_from_path(path) - self._script_map[script] = path + md = self.FIDELITY_BOND_MIXDEPTH + address_type = self.BIP32_TIMELOCK_ID + for timenumber in range(self.TIMENUMBER_COUNT): + path = self.get_path(md, address_type, timenumber, timenumber) + script = self.get_script_from_path(path) + self._script_map[script] = path def add_utxo(self, txid, index, script, value, height=None): super().add_utxo(txid, index, script, value, height) @@ -2380,6 +2376,42 @@ class FidelityBondMixin(object): self._storage.data[self._BURNER_OUTPUT_STORAGE_KEY][path][2] = \ merkle_branch + + @classmethod + def calculate_timelocked_fidelity_bond_value(cls, utxo_value, confirmation_time, locktime, + current_time, interest_rate): + """ + utxo_value is in satoshi + interest rate is per year + all times are seconds + """ + YEAR = 60 * 60 * 24 * 365.2425 #gregorian calender year length + + r = interest_rate + T = (locktime - confirmation_time) / YEAR + L = locktime / YEAR + t = current_time / YEAR + + a = max(0, min(1, exp(r*T) - 1) - min(1, exp(r*max(0, t-L)) - 1)) + return utxo_value*utxo_value*a*a + + @classmethod + def get_validated_timelocked_fidelity_bond_utxo(cls, utxo, utxo_pubkey, locktime, + cert_expiry, current_block_height): + + utxo_data = jm_single().bc_interface.query_utxo_set(utxo, includeconf=True) + if utxo_data[0] == None: + return None + if utxo_data[0]["confirms"] <= 0: + return None + RETARGET_INTERVAL = 2016 + if current_block_height > cert_expiry*RETARGET_INTERVAL: + return None + implied_spk = btc.redeem_script_to_p2wsh_script(btc.mk_freeze_script(utxo_pubkey, locktime)) + if utxo_data[0]["script"] != implied_spk: + return None + return utxo_data[0] + class BIP49Wallet(BIP32PurposedWallet): _PURPOSE = 2**31 + 49 _ENGINE = ENGINES[TYPE_P2SH_P2WPKH] @@ -2394,13 +2426,13 @@ class SegwitLegacyWallet(ImportWalletMixin, BIP39WalletMixin, PSBTWalletMixin, S class SegwitWallet(ImportWalletMixin, BIP39WalletMixin, PSBTWalletMixin, SNICKERWalletMixin, BIP84Wallet): TYPE = TYPE_P2WPKH -class SegwitLegacyWalletFidelityBonds(FidelityBondMixin, SegwitLegacyWallet): - TYPE = TYPE_SEGWIT_LEGACY_WALLET_FIDELITY_BONDS +class SegwitWalletFidelityBonds(FidelityBondMixin, SegwitWallet): + TYPE = TYPE_SEGWIT_WALLET_FIDELITY_BONDS -class FidelityBondWatchonlyWallet(FidelityBondMixin, BIP49Wallet): +class FidelityBondWatchonlyWallet(FidelityBondMixin, BIP84Wallet): TYPE = TYPE_WATCHONLY_FIDELITY_BONDS - _ENGINE = ENGINES[TYPE_WATCHONLY_P2SH_P2WPKH] + _ENGINE = ENGINES[TYPE_WATCHONLY_P2WPKH] _TIMELOCK_ENGINE = ENGINES[TYPE_WATCHONLY_TIMELOCK_P2WSH] @classmethod @@ -2420,6 +2452,6 @@ WALLET_IMPLEMENTATIONS = { LegacyWallet.TYPE: LegacyWallet, SegwitLegacyWallet.TYPE: SegwitLegacyWallet, SegwitWallet.TYPE: SegwitWallet, - SegwitLegacyWalletFidelityBonds.TYPE: SegwitLegacyWalletFidelityBonds, + SegwitWalletFidelityBonds.TYPE: SegwitWalletFidelityBonds, FidelityBondWatchonlyWallet.TYPE: FidelityBondWatchonlyWallet } diff --git a/jmclient/jmclient/wallet_service.py b/jmclient/jmclient/wallet_service.py index 7cb5854..c4312b5 100644 --- a/jmclient/jmclient/wallet_service.py +++ b/jmclient/jmclient/wallet_service.py @@ -923,18 +923,8 @@ class WalletService(Service): if isinstance(self.wallet, FidelityBondMixin): md = FidelityBondMixin.FIDELITY_BOND_MIXDEPTH address_type = FidelityBondMixin.BIP32_TIMELOCK_ID - saved_indices[md] += [0] - next_unused = self.get_next_unused_index(md, address_type) - for index in range(next_unused): - for timenumber in range(FidelityBondMixin.TIMENUMBERS_PER_PUBKEY): - addresses.add(self.get_addr(md, address_type, index, timenumber)) - for index in range(self.gap_limit // FidelityBondMixin.TIMELOCK_GAP_LIMIT_REDUCTION_FACTOR): - index += next_unused - assert self.wallet.get_index_cache_and_increment(md, address_type) == index - for timenumber in range(FidelityBondMixin.TIMENUMBERS_PER_PUBKEY): - self.wallet.get_script_and_update_map(md, address_type, index, timenumber) - addresses.add(self.get_addr(md, address_type, index, timenumber)) - self.wallet.set_next_index(md, address_type, next_unused) + for timenumber in range(FidelityBondMixin.TIMENUMBER_COUNT): + addresses.add(self.get_addr(md, address_type, timenumber, timenumber)) return addresses, saved_indices @@ -950,17 +940,6 @@ class WalletService(Service): addresses.add(self.get_new_addr(md, address_type)) self.set_next_index(md, address_type, old_next) - if isinstance(self.wallet, FidelityBondMixin): - md = FidelityBondMixin.FIDELITY_BOND_MIXDEPTH - address_type = FidelityBondMixin.BIP32_TIMELOCK_ID - old_next = self.get_next_unused_index(md, address_type) - for ii in range(gap_limit // FidelityBondMixin.TIMELOCK_GAP_LIMIT_REDUCTION_FACTOR): - index = self.wallet.get_index_cache_and_increment(md, address_type) - for timenumber in range(FidelityBondMixin.TIMENUMBERS_PER_PUBKEY): - self.wallet.get_script_and_update_map(md, address_type, index, timenumber) - addresses.add(self.get_addr(md, address_type, index, timenumber)) - self.set_next_index(md, address_type, old_next) - return addresses def get_external_addr(self, mixdepth): diff --git a/jmclient/jmclient/wallet_utils.py b/jmclient/jmclient/wallet_utils.py index 51bb10e..ed1b2a1 100644 --- a/jmclient/jmclient/wallet_utils.py +++ b/jmclient/jmclient/wallet_utils.py @@ -21,7 +21,7 @@ from jmbase.support import (get_password, jmprint, EXIT_FAILURE, IndentedHelpFormatterWithNL) from .cryptoengine import TYPE_P2PKH, TYPE_P2SH_P2WPKH, TYPE_P2WPKH, \ - TYPE_SEGWIT_LEGACY_WALLET_FIDELITY_BONDS + TYPE_SEGWIT_WALLET_FIDELITY_BONDS from .output import fmt_utxo import jmbitcoin as btc @@ -246,7 +246,8 @@ class WalletViewAccount(WalletViewBase): self.account_name = account_name self.xpub = xpub if branches: - assert len(branches) in [2, 3, 4] #3 if imported keys, 4 if fidelity bonds + assert len(branches) in [2, 3, 4, 5] #3 if imported keys, 4 if fidelity bonds + #5 if all those plus imported assert all([isinstance(x, WalletViewBranch) for x in branches]) self.branches = branches @@ -360,7 +361,11 @@ def wallet_showutxos(wallet_service, showprivkey): for u, av in utxos[md].items(): success, us = utxo_to_utxostr(u) assert success - key = wallet_service.get_key_from_addr(av['address']) + key = wallet_service._get_key_from_path(av["path"])[0] + if FidelityBondMixin.is_timelocked_path(av["path"]): + key, locktime = key + else: + locktime = None tries = podle.get_podle_tries(u, key, max_tries) tries_remaining = max(0, max_tries - tries) mixdepth = wallet_service.wallet.get_details(av['path'])[0] @@ -372,6 +377,8 @@ def wallet_showutxos(wallet_service, showprivkey): 'frozen': True if u in utxo_d else False} if showprivkey: unsp[us]['privkey'] = wallet_service.get_wif_path(av['path']) + if locktime: + unsp[us]["locktime"] = str(datetime.utcfromtimestamp(locktime)) used_commitments, external_commitments = podle.get_podle_commitments() for u, ec in external_commitments.items(): @@ -466,27 +473,23 @@ def wallet_display(wallet_service, showprivkey, displayall=False, if m == FidelityBondMixin.FIDELITY_BOND_MIXDEPTH and \ isinstance(wallet_service.wallet, FidelityBondMixin): address_type = FidelityBondMixin.BIP32_TIMELOCK_ID - unused_index = wallet_service.get_next_unused_index(m, address_type) - timelocked_gaplimit = (wallet_service.wallet.gap_limit - // FidelityBondMixin.TIMELOCK_GAP_LIMIT_REDUCTION_FACTOR) entrylist = [] - for k in range(unused_index + timelocked_gaplimit): - for timenumber in range(FidelityBondMixin.TIMENUMBERS_PER_PUBKEY): - path = wallet_service.get_path(m, address_type, k, timenumber) - addr = wallet_service.get_address_from_path(path) - timelock = datetime.utcfromtimestamp(path[-1]) - - balance = sum([utxodata["value"] for utxo, utxodata in - utxos[m].items() if path == utxodata["path"]]) - status = timelock.strftime("%Y-%m-%d") + " [" + ( - "LOCKED" if datetime.now() < timelock else "UNLOCKED") + "]" - privkey = "" - if showprivkey: - privkey = wallet_service.get_wif_path(path) - if displayall or balance > 0: - entrylist.append(WalletViewEntry( - wallet_service.get_path_repr(path), m, address_type, k, - addr, [balance, balance], priv=privkey, used=status)) + for timenumber in range(FidelityBondMixin.TIMENUMBER_COUNT): + path = wallet_service.get_path(m, address_type, timenumber, timenumber) + addr = wallet_service.get_address_from_path(path) + timelock = datetime.utcfromtimestamp(path[-1]) + + balance = sum([utxodata["value"] for utxo, utxodata in + utxos[m].items() if path == utxodata["path"]]) + status = timelock.strftime("%Y-%m-%d") + " [" + ( + "LOCKED" if datetime.now() < timelock else "UNLOCKED") + "]" + privkey = "" + if showprivkey: + privkey = wallet_service.get_wif_path(path) + if displayall or balance > 0: + entrylist.append(WalletViewEntry( + wallet_service.get_path_repr(path), m, address_type, k, + addr, [balance, balance], priv=privkey, used=status)) xpub_key = wallet_service.get_bip32_pub_export(m, address_type) path = wallet_service.get_path_repr(wallet_service.get_path(m, address_type)) branchlist.append(WalletViewBranch(path, m, address_type, entrylist, @@ -633,13 +636,8 @@ def wallet_generate_recover_bip39(method, walletspath, default_wallet_name, if not wallet_name: wallet_name = default_wallet_name wallet_path = os.path.join(walletspath, wallet_name) - - # disable creating fidelity bond wallets for now until the - # rest of the fidelity bond feature is created - #support_fidelity_bonds = enter_do_support_fidelity_bonds() - support_fidelity_bonds = False + support_fidelity_bonds = enter_do_support_fidelity_bonds() wallet_cls = get_wallet_cls(get_configured_wallet_type(support_fidelity_bonds)) - wallet = create_wallet(wallet_path, password, mixdepth, wallet_cls, entropy=entropy, entropy_extension=mnemonic_extension) @@ -1227,16 +1225,20 @@ def wallet_gettimelockaddress(wallet, locktime_string): m = FidelityBondMixin.FIDELITY_BOND_MIXDEPTH address_type = FidelityBondMixin.BIP32_TIMELOCK_ID - index = wallet.get_next_unused_index(m, address_type) lock_datetime = datetime.strptime(locktime_string, "%Y-%m") timenumber = FidelityBondMixin.timestamp_to_time_number(timegm( lock_datetime.timetuple())) + index = timenumber path = wallet.get_path(m, address_type, index, timenumber) jmprint("path = " + wallet.get_path_repr(path), "info") jmprint("Coins sent to this address will be not be spendable until " + lock_datetime.strftime("%B %Y") + ". Full date: " + str(lock_datetime)) + jmprint("WARNING: Only send coins here which are from coinjoins or otherwise" + + " not linked to your identity. Also, use a sweep transaction when funding the" + + " timelocked address, i.e. Don't create a change address. See the privacy warnings in" + + " fidelity-bonds.md") addr = wallet.get_address_from_path(path) return addr @@ -1296,8 +1298,8 @@ def get_configured_wallet_type(support_fidelity_bonds): if not support_fidelity_bonds: return configured_type - if configured_type == TYPE_P2SH_P2WPKH: - return TYPE_SEGWIT_LEGACY_WALLET_FIDELITY_BONDS + if configured_type == TYPE_P2WPKH: + return TYPE_SEGWIT_WALLET_FIDELITY_BONDS else: raise ValueError("Fidelity bonds not supported with the configured " "options of segwit and native. Edit joinmarket.cfg") diff --git a/jmclient/jmclient/yieldgenerator.py b/jmclient/jmclient/yieldgenerator.py index ba789fb..9f8c4bb 100644 --- a/jmclient/jmclient/yieldgenerator.py +++ b/jmclient/jmclient/yieldgenerator.py @@ -4,15 +4,19 @@ import datetime import os import time import abc +import base64 from twisted.python.log import startLogging from optparse import OptionParser from jmbase import get_log from jmclient import (Maker, jm_single, load_program_config, JMClientProtocolFactory, start_reactor, calc_cj_fee, WalletService, add_base_options, SNICKERReceiver, - SNICKERClientProtocolFactory) + SNICKERClientProtocolFactory, FidelityBondMixin, + get_interest_rate, fmt_utxo) from .wallet_utils import open_test_wallet_maybe, get_wallet_path from jmbase.support import EXIT_ARGERROR, EXIT_FAILURE, get_jm_version_str +import jmbitcoin as btc +from jmclient.fidelity_bond import FidelityBond jlog = get_log() @@ -66,7 +70,6 @@ class YieldGenerator(Maker): return to_cancel, to_announce - class YieldGeneratorBasic(YieldGenerator): """A simplest possible instantiation of a yieldgenerator. It will often (but not always) reannounce orders after transactions, @@ -112,6 +115,50 @@ class YieldGeneratorBasic(YieldGenerator): return [order] + def get_fidelity_bond_template(self): + if not isinstance(self.wallet_service.wallet, FidelityBondMixin): + jlog.info("Not a fidelity bond wallet, not announcing fidelity bond") + return None + blocks = jm_single().bc_interface.get_current_block_height() + mediantime = jm_single().bc_interface.get_best_block_median_time() + + BLOCK_COUNT_SAFETY = 2 #use this safety number to reduce chances of the proof expiring + #before the taker gets a chance to verify it + RETARGET_INTERVAL = 2016 + CERT_MAX_VALIDITY_TIME = 1 + cert_expiry = ((blocks + BLOCK_COUNT_SAFETY) // RETARGET_INTERVAL) + CERT_MAX_VALIDITY_TIME + + utxos = self.wallet_service.wallet.get_utxos_by_mixdepth(include_disabled=True, + includeheight=True)[FidelityBondMixin.FIDELITY_BOND_MIXDEPTH] + timelocked_utxos = [(outpoint, info) for outpoint, info in utxos.items() + if FidelityBondMixin.is_timelocked_path(info["path"])] + if len(timelocked_utxos) == 0: + jlog.info("No timelocked coins in wallet, not announcing fidelity bond") + return + timelocked_utxos_with_confirmation_time = [(outpoint, info, + jm_single().bc_interface.get_block_time( + jm_single().bc_interface.get_block_hash(info["height"]) + )) + for (outpoint, info) in timelocked_utxos] + + interest_rate = get_interest_rate() + max_valued_bond = max(timelocked_utxos_with_confirmation_time, key=lambda x: + FidelityBondMixin.calculate_timelocked_fidelity_bond_value(x[1]["value"], x[2], + x[1]["path"][-1], mediantime, interest_rate) + ) + (utxo_priv, locktime), engine = self.wallet_service.wallet._get_key_from_path( + max_valued_bond[1]["path"]) + utxo_pub = engine.privkey_to_pubkey(utxo_priv) + cert_priv = os.urandom(32) + b"\x01" + cert_pub = btc.privkey_to_pubkey(cert_priv) + cert_msg = b"fidelity-bond-cert|" + cert_pub + b"|" + str(cert_expiry).encode("ascii") + cert_sig = base64.b64decode(btc.ecdsa_sign(cert_msg, utxo_priv)) + utxo = (max_valued_bond[0][0], max_valued_bond[0][1]) + fidelity_bond = FidelityBond(utxo, utxo_pub, locktime, cert_expiry, + cert_priv, cert_pub, cert_sig) + jlog.info("Announcing fidelity bond coin {}".format(fmt_utxo(utxo))) + return fidelity_bond + def oid_to_order(self, offer, amount): total_amount = amount + offer["txfee"] real_cjfee = calc_cj_fee(offer["ordertype"], offer["cjfee"], amount) diff --git a/jmclient/test/test_client_protocol.py b/jmclient/test/test_client_protocol.py index d16f8df..7ade6db 100644 --- a/jmclient/test/test_client_protocol.py +++ b/jmclient/test/test_client_protocol.py @@ -43,7 +43,7 @@ class DummyTaker(Taker): def default_taker_info_callback(self, infotype, msg): jlog.debug(infotype + ":" + msg) - def initialize(self, orderbook): + def initialize(self, orderbook, fidelity_bonds_info): """Once the daemon is active and has returned the current orderbook, select offers, re-initialize variables and prepare a commitment, then send it to the protocol to fill offers. @@ -90,6 +90,7 @@ class DummyMaker(Maker): self.aborted = False self.wallet_service = WalletService(DummyWallet()) self.offerlist = self.create_my_orders() + self.fidelity_bond = None def try_to_create_my_orders(self): pass @@ -184,8 +185,8 @@ class JMTestServerProtocol(JMBaseProtocol): return {'accepted': True} @JMSetup.responder - def on_JM_SETUP(self, role, initdata): - show_receipt("JMSETUP", role, initdata) + def on_JM_SETUP(self, role, offers, use_fidelity_bond): + show_receipt("JMSETUP", role, offers, use_fidelity_bond) d = self.callRemote(JMSetupDone) self.defaultCallbacks(d) return {'accepted': True} @@ -195,8 +196,10 @@ class JMTestServerProtocol(JMBaseProtocol): show_receipt("JMREQUESTOFFERS") #build a huge orderbook to test BigString Argument orderbook = ["aaaa" for _ in range(15)] + fidelitybonds = ["bbbb" for _ in range(15)] d = self.callRemote(JMOffers, - orderbook=json.dumps(orderbook)) + orderbook=json.dumps(orderbook), + fidelitybonds=json.dumps(fidelitybonds)) self.defaultCallbacks(d) return {'accepted': True} diff --git a/jmclient/test/test_coinjoin.py b/jmclient/test/test_coinjoin.py index b2e694d..431699f 100644 --- a/jmclient/test/test_coinjoin.py +++ b/jmclient/test/test_coinjoin.py @@ -80,7 +80,7 @@ def create_orders(makers): maker.try_to_create_my_orders() def init_coinjoin(taker, makers, orderbook, cj_amount): - init_data = taker.initialize(orderbook) + init_data = taker.initialize(orderbook, []) assert init_data[0], "taker.initialize error" active_orders = init_data[4] maker_data = {} diff --git a/jmclient/test/test_support.py b/jmclient/test/test_support.py index cb0872e..dcb1075 100644 --- a/jmclient/test/test_support.py +++ b/jmclient/test/test_support.py @@ -6,7 +6,7 @@ from jmclient import (select, select_gradual, select_greedy, select_greediest, choose_orders, choose_sweep_orders, weighted_order_choose) from jmclient.support import (calc_cj_fee, rand_exp_array, rand_norm_array, rand_weighted_choice, - cheapest_order_choose) + cheapest_order_choose, fidelity_bond_weighted_order_choose) from taker_test_data import t_orderbook import copy @@ -73,6 +73,11 @@ def test_choose_orders(): #test the hated 'cheapest' orders_fees = choose_orders(orderbook, 100000000, 3, cheapest_order_choose) assert len(orders_fees[0]) == 3 + #test the fidelity bond one + for i, o in enumerate(orderbook): + o["fidelity_bond_value"] = i+1 + orders_fees = choose_orders(orderbook, 100000000, 3, fidelity_bond_weighted_order_choose) + assert len(orders_fees[0]) == 3 #test sweep result, cjamount, total_fee = choose_sweep_orders(orderbook, 50000000, 30000, diff --git a/jmclient/test/test_taker.py b/jmclient/test/test_taker.py index 31cd525..3734e3a 100644 --- a/jmclient/test/test_taker.py +++ b/jmclient/test/test_taker.py @@ -165,11 +165,11 @@ def test_filter_rejection(setup_taker): return False taker = get_taker(filter_orders=filter_orders_reject) taker.schedule = [[0, 20000000, 3, "mnsquzxrHXpFsZeL42qwbKdCP2y1esN3qw", 0, NO_ROUNDING]] - res = taker.initialize(t_orderbook) + res = taker.initialize(t_orderbook, []) assert not res[0] taker = get_taker(filter_orders=filter_orders_reject) taker.schedule = [[0, 0, 3, "mnsquzxrHXpFsZeL42qwbKdCP2y1esN3qw", 0, NO_ROUNDING]] - res = taker.initialize(t_orderbook) + res = taker.initialize(t_orderbook, []) assert not res[0] @pytest.mark.parametrize( @@ -258,7 +258,7 @@ def test_make_commitment(setup_taker, mixdepth, cjamt, failquery, external, def test_not_found_maker_utxos(setup_taker): taker = get_taker([(0, 20000000, 3, "mnsquzxrHXpFsZeL42qwbKdCP2y1esN3qw", 0, NO_ROUNDING)]) orderbook = copy.deepcopy(t_orderbook) - res = taker.initialize(orderbook) + res = taker.initialize(orderbook, []) taker.orderbook = copy.deepcopy(t_chosen_orders) #total_cjfee unaffected, all same maker_response = copy.deepcopy(t_maker_response) jm_single().bc_interface.setQUSFail(True) @@ -270,7 +270,7 @@ def test_not_found_maker_utxos(setup_taker): def test_auth_pub_not_found(setup_taker): taker = get_taker([(0, 20000000, 3, "mnsquzxrHXpFsZeL42qwbKdCP2y1esN3qw", 0, NO_ROUNDING)]) orderbook = copy.deepcopy(t_orderbook) - res = taker.initialize(orderbook) + res = taker.initialize(orderbook, []) taker.orderbook = copy.deepcopy(t_chosen_orders) #total_cjfee unaffected, all same maker_response = copy.deepcopy(t_maker_response) utxos = [utxostr_to_utxo(x)[1] for x in [ @@ -351,11 +351,11 @@ def test_taker_init(setup_taker, schedule, highfee, toomuchcoins, minmakers, if schedule[0][1] == 0.2: #triggers calc-ing amount based on a fraction jm_single().mincjamount = 50000000 #bigger than 40m = 0.2 * 200m - res = taker.initialize(orderbook) + res = taker.initialize(orderbook, []) assert res[0] assert res[1] == jm_single().mincjamount return clean_up() - res = taker.initialize(orderbook) + res = taker.initialize(orderbook, []) if toomuchcoins or ignored: assert not res[0] return clean_up() @@ -427,7 +427,7 @@ def test_taker_init(setup_taker, schedule, highfee, toomuchcoins, minmakers, assert res[0] #re-calling will trigger "finished" code, since schedule is "complete". - res = taker.initialize(orderbook) + res = taker.initialize(orderbook, []) assert not res[0] #some exception cases: no coinjoin address, no change address: @@ -454,7 +454,7 @@ def test_custom_change(setup_taker): for script, addr in zip(scripts, addrs): taker = get_taker(schedule, custom_change=addr) orderbook = copy.deepcopy(t_orderbook) - res = taker.initialize(orderbook) + res = taker.initialize(orderbook, []) taker.orderbook = copy.deepcopy(t_chosen_orders) maker_response = copy.deepcopy(t_maker_response) res = taker.receive_utxos(maker_response) diff --git a/jmclient/test/test_wallet.py b/jmclient/test/test_wallet.py index 2fedf1b..693b3a6 100644 --- a/jmclient/test/test_wallet.py +++ b/jmclient/test/test_wallet.py @@ -11,7 +11,7 @@ from jmbase import get_log, hextobin from jmclient import load_test_config, jm_single, BaseWallet, \ SegwitLegacyWallet,BIP32Wallet, BIP49Wallet, LegacyWallet,\ VolatileStorage, get_network, cryptoengine, WalletError,\ - SegwitWallet, WalletService, SegwitLegacyWalletFidelityBonds,\ + SegwitWallet, WalletService, SegwitWalletFidelityBonds,\ create_wallet, open_test_wallet_maybe, \ FidelityBondMixin, FidelityBondWatchonlyWallet, wallet_gettimelockaddress from test_blockchaininterface import sync_test_wallet @@ -243,19 +243,19 @@ def test_bip32_addresses_p2sh_p2wpkh(setup_wallet, mixdepth, internal, index, ad assert address == wallet.get_addr(mixdepth, internal, index) @pytest.mark.parametrize('index,timenumber,address,wif', [ - [0, 0, 'bcrt1qndcqwedwa4lu77ryqpvp738d6p034a2fv8mufw3pw5smfcn39sgqpesn76', 'cST4g5R3mKp44K4J8PRVyys4XJu6EFavZyssq67PJKCnbhjdEdBY'], - [0, 50, 'bcrt1q73zhrfcu0ttkk4er9esrmvnpl6wpzhny5aly97jj9nw52agf8ncqjv8rda', 'cST4g5R3mKp44K4J8PRVyys4XJu6EFavZyssq67PJKCnbhjdEdBY'], - [5, 0, 'bcrt1qz5208jdm6399ja309ra28d0a34qlt0859u77uxc94v5mgk7auhtssau4pw', 'cRnUaBYTmyZURPe72YCrtvgxpBMvLKPZaCoXvKuWRPMryeJeAZx2'], - [9, 1, 'bcrt1qa7pd6qnadpmlm29vtvqnykalc34tr33eclaz7eeqal59n4gwr28qwnka2r', 'cQCxEPCWMwXVB16zCikDBTXMUccx6ioHQipPhYEp1euihkJUafyD'] + [0, 0, 'bcrt1qgysu2eynn6klarz200ctgev7gqhhp7hwsdaaec3c7h0ltmc3r68q87c2d3', 'cVASAS6bpC5yctGmnsKaDz7D8CxEwccUtpjSNBQzeV2fw8ox8RR9'], + [0, 50, 'bcrt1qyrdhyqzj87vq20e853x7gzhx9lp8ta6cd8mwp8haqex8r4vrg2wsf7rcxm', 'cVASAS6bpC5yctGmnsKaDz7D8CxEwccUtpjSNBQzeV2fw8ox8RR9'], + [5, 0, 'bcrt1quunmmsudhpsuksa2ke8m6aj7757mst966mqq50nckx9wdrs4y6fs9gjuww', 'cUgT5jRjYi6i8Fc7TirJrrvhbs7ceSqJ6USKboVrLYghJKDzEQHQ'], + [9, 1, 'bcrt1qvpgmrn5a7yc0h2j6fp8jhtwzd8eetlt7hsu3cn098qftzp4t2h6sp5p35p', 'cW7H2pv6Rr5NWaTAnDC6r7bviHwDsAwyqh4XdZTE4xf2H2DB2hmb'] ]) def test_bip32_timelocked_addresses(setup_wallet, index, timenumber, address, wif): jm_single().config.set('BLOCKCHAIN', 'network', 'testnet') entropy = unhexlify('2e0339ba89b4a1272cdf78b27ee62669ee01992a59e836e2807051be128ca817') storage = VolatileStorage() - SegwitLegacyWalletFidelityBonds.initialize( + SegwitWalletFidelityBonds.initialize( storage, get_network(), entropy=entropy, max_mixdepth=1) - wallet = SegwitLegacyWalletFidelityBonds(storage) + wallet = SegwitWalletFidelityBonds(storage) mixdepth = FidelityBondMixin.FIDELITY_BOND_MIXDEPTH address_type = FidelityBondMixin.BIP32_TIMELOCK_ID @@ -274,12 +274,12 @@ def test_bip32_timelocked_addresses(setup_wallet, index, timenumber, address, wi ]) def test_gettimelockaddress_method(setup_wallet, timenumber, locktime_string): storage = VolatileStorage() - SegwitLegacyWalletFidelityBonds.initialize(storage, get_network()) - wallet = SegwitLegacyWalletFidelityBonds(storage) + SegwitWalletFidelityBonds.initialize(storage, get_network()) + wallet = SegwitWalletFidelityBonds(storage) m = FidelityBondMixin.FIDELITY_BOND_MIXDEPTH address_type = FidelityBondMixin.BIP32_TIMELOCK_ID - index = wallet.get_next_unused_index(m, address_type) + index = timenumber script = wallet.get_script_and_update_map(m, address_type, index, timenumber) addr = wallet.script_to_addr(script) @@ -289,18 +289,18 @@ def test_gettimelockaddress_method(setup_wallet, timenumber, locktime_string): assert addr == addr_from_method @pytest.mark.parametrize('index,wif', [ - [0, 'cMg9eH3fW2JDSyggvXucjmECRwiheCMDo2Qik8y1keeYaxynzrYa'], - [9, 'cURA1Qgxhd7QnhhwxCnCHD4pZddVrJdu2BkTdzNaTp9owRSkUvPy'], - [50, 'cRTaHZ1eezb8s6xsT2V7EAevYToQMi7cxQD9vgFZzaJZDfhMhf3c'] + [0, 'cVQbz7DB5JQ1TGsg9Dbm32VtJbXBHaj39Yc9QLkaGpRgXcibHTDH'], + [9, 'cULqe2sYZ4z8jZTGybr2Bzf4EyiT5Ts6wAE3mvCUofRuTVsofR8N'], + [50, 'cQNp7cQbrwjWuxmbkZF8ax9ogmTuWp3Ykb9LEpainhRTJXYc8Deu'] ]) def test_bip32_burn_keys(setup_wallet, index, wif): jm_single().config.set('BLOCKCHAIN', 'network', 'testnet') entropy = unhexlify('2e0339ba89b4a1272cdf78b27ee62669ee01992a59e836e2807051be128ca817') storage = VolatileStorage() - SegwitLegacyWalletFidelityBonds.initialize( + SegwitWalletFidelityBonds.initialize( storage, get_network(), entropy=entropy, max_mixdepth=1) - wallet = SegwitLegacyWalletFidelityBonds(storage) + wallet = SegwitWalletFidelityBonds(storage) mixdepth = FidelityBondMixin.FIDELITY_BOND_MIXDEPTH address_type = FidelityBondMixin.BIP32_BURN_ID @@ -402,8 +402,8 @@ def test_timelocked_output_signing(setup_wallet): jm_single().config.set('BLOCKCHAIN', 'network', 'testnet') ensure_bip65_activated() storage = VolatileStorage() - SegwitLegacyWalletFidelityBonds.initialize(storage, get_network()) - wallet = SegwitLegacyWalletFidelityBonds(storage) + SegwitWalletFidelityBonds.initialize(storage, get_network()) + wallet = SegwitWalletFidelityBonds(storage) index = 0 timenumber = 0 @@ -629,7 +629,7 @@ def test_path_repr_imported(setup_wallet): [0, 1577836800], [50, 1709251200], [300, 2366841600], - [400, None], #too far in the future + [1000, None], #too far in the future [-1, None] #before epoch ]) def test_timenumber_to_timestamp(setup_wallet, timenumber, timestamp): @@ -646,7 +646,7 @@ def test_timenumber_to_timestamp(setup_wallet, timenumber, timestamp): [1709251200, 50], [2366841600, 300], [1577836801, None], #not exactly midnight on first of month - [2629670400, None], #too far in future + [4133980800, None], #too far in future [1575158400, None] #before epoch ]) def test_timestamp_to_timenumber(setup_wallet, timestamp, timenumber): @@ -787,14 +787,14 @@ def test_wallet_mixdepth_decrease(setup_wallet): def test_watchonly_wallet(setup_wallet): jm_single().config.set('BLOCKCHAIN', 'network', 'testnet') storage = VolatileStorage() - SegwitLegacyWalletFidelityBonds.initialize(storage, get_network()) - wallet = SegwitLegacyWalletFidelityBonds(storage) + SegwitWalletFidelityBonds.initialize(storage, get_network()) + wallet = SegwitWalletFidelityBonds(storage) paths = [ - "m/49'/1'/0'/0/0", - "m/49'/1'/0'/1/0", - "m/49'/1'/0'/2/0:1577836800", - "m/49'/1'/0'/2/0:2314051200" + "m/84'/1'/0'/0/0", + "m/84'/1'/0'/1/0", + "m/84'/1'/0'/2/0:1577836800", + "m/84'/1'/0'/2/0:2314051200" ] burn_path = "m/49'/1'/0'/3/0" @@ -821,6 +821,67 @@ def test_watchonly_wallet(setup_wallet): assert script == watchonly_script assert burn_pubkey == watchonly_burn_pubkey +def test_calculate_timelocked_fidelity_bond_value(setup_wallet): + EPSILON = 0.000001 + YEAR = 60*60*24*356.25 + + #the function should be flat anywhere before the locktime ends + values = [FidelityBondMixin.calculate_timelocked_fidelity_bond_value( + utxo_value=100000000, + confirmation_time=0, + locktime=6*YEAR, + current_time=y*YEAR, + interest_rate=0.01 + ) + for y in range(4) + ] + value_diff = [values[i] - values[i+1] for i in range(len(values)-1)] + for vd in value_diff: + assert abs(vd) < EPSILON + + #after locktime, the value should go down + values = [FidelityBondMixin.calculate_timelocked_fidelity_bond_value( + utxo_value=100000000, + confirmation_time=0, + locktime=6*YEAR, + current_time=(6+y)*YEAR, + interest_rate=0.01 + ) + for y in range(5) + ] + value_diff = [values[i+1] - values[i] for i in range(len(values)-1)] + for vrd in value_diff: + assert vrd < 0 + + #value of a bond goes up as the locktime goes up + values = [FidelityBondMixin.calculate_timelocked_fidelity_bond_value( + utxo_value=100000000, + confirmation_time=0, + locktime=y*YEAR, + current_time=0, + interest_rate=0.01 + ) + for y in range(5) + ] + value_ratio = [values[i] / values[i+1] for i in range(len(values)-1)] + value_ratio_diff = [value_ratio[i] - value_ratio[i+1] for i in range(len(value_ratio)-1)] + for vrd in value_ratio_diff: + assert vrd < 0 + + #value of a bond locked into the far future is constant, clamped at the value of burned coins + values = [FidelityBondMixin.calculate_timelocked_fidelity_bond_value( + utxo_value=100000000, + confirmation_time=0, + locktime=(200+y)*YEAR, + current_time=0, + interest_rate=0.01 + ) + for y in range(5) + ] + value_diff = [values[i] - values[i+1] for i in range(len(values)-1)] + for vd in value_diff: + assert abs(vd) < EPSILON + @pytest.mark.parametrize('password, wallet_cls', [ ["hunter2", SegwitLegacyWallet], ["hunter2", SegwitWallet], diff --git a/jmdaemon/jmdaemon/__init__.py b/jmdaemon/jmdaemon/__init__.py index 0e854ba..384b5f7 100644 --- a/jmdaemon/jmdaemon/__init__.py +++ b/jmdaemon/jmdaemon/__init__.py @@ -14,6 +14,7 @@ from .daemon_protocol import (JMDaemonServerProtocolFactory, JMDaemonServerProto from .protocol import (COMMAND_PREFIX, ORDER_KEYS, NICK_HASH_LENGTH, NICK_MAX_ENCODED, JM_VERSION, JOINMARKET_NICK_HEADER) from .message_channel import MessageChannelCollection + # Set default logging handler to avoid "No handler found" warnings. try: from logging import NullHandler diff --git a/jmdaemon/jmdaemon/daemon_protocol.py b/jmdaemon/jmdaemon/daemon_protocol.py index 954293c..c2992fd 100644 --- a/jmdaemon/jmdaemon/daemon_protocol.py +++ b/jmdaemon/jmdaemon/daemon_protocol.py @@ -474,6 +474,7 @@ class JMDaemonServerProtocol(amp.AMP, OrderbookWatch): self.crypto_boxes = {} self.sig_lock = threading.Lock() self.active_orders = {} + self.use_fidelity_bond = False def checkClientResponse(self, response): """A generic check of client acceptance; any failure @@ -551,7 +552,7 @@ class JMDaemonServerProtocol(amp.AMP, OrderbookWatch): return {'accepted': True} @JMSetup.responder - def on_JM_SETUP(self, role, initdata): + def on_JM_SETUP(self, role, offers, use_fidelity_bond): assert self.jm_state == 0 self.role = role self.crypto_boxes = {} @@ -565,8 +566,9 @@ class JMDaemonServerProtocol(amp.AMP, OrderbookWatch): if self.role == "TAKER": self.mcc.pubmsg(COMMAND_PREFIX + "orderbook") elif self.role == "MAKER": - self.offerlist = json.loads(initdata) - self.mcc.announce_orders(self.offerlist) + self.offerlist = json.loads(offers) + self.use_fidelity_bond = use_fidelity_bond + self.mcc.announce_orders(self.offerlist, None, None, None) self.jm_state = 1 return {'accepted': True} @@ -592,10 +594,16 @@ class JMDaemonServerProtocol(amp.AMP, OrderbookWatch): This call is stateless.""" rows = self.db.execute('SELECT * FROM orderbook;').fetchall() self.orderbook = [dict([(k, o[k]) for k in ORDER_KEYS]) for o in rows] - log.msg("About to send orderbook of size: " + str(len(self.orderbook))) string_orderbook = json.dumps(self.orderbook) - d = self.callRemote(JMOffers, - orderbook=string_orderbook) + + fbond_rows = self.db.execute("SELECT * FROM fidelitybonds;").fetchall() + fidelitybonds = [fb for fb in fbond_rows] + string_fidelitybonds = json.dumps(fidelitybonds) + + log.msg("About to send orderbook (size=" + str(len(self.orderbook)) + + " with fidelity bonds (size=" + str(len(fidelitybonds))) + d = self.callRemote(JMOffers, orderbook=string_orderbook, + fidelitybonds=string_fidelitybonds) self.defaultCallbacks(d) return {'accepted': True} @@ -655,7 +663,13 @@ class JMDaemonServerProtocol(amp.AMP, OrderbookWatch): if len(to_cancel) > 0: self.mcc.cancel_orders(to_cancel) if len(to_announce) > 0: - self.mcc.announce_orders(to_announce, None, None) + self.mcc.announce_orders(to_announce, None, None, None) + return {"accepted": True} + + @JMFidelityBondProof.responder + def on_JM_FIDELITY_BOND_PROOF(self, nick, proof): + """Called by maker client as a reply to a request of a fidelity bond proof""" + self.mcc.announce_orders(self.offerlist, nick, proof, new_mc=None) return {"accepted": True} @JMIOAuth.responder @@ -711,7 +725,16 @@ class JMDaemonServerProtocol(amp.AMP, OrderbookWatch): def on_orderbook_requested(self, nick, mc=None): """Dealt with by daemon, assuming offerlist is up to date """ - self.mcc.announce_orders(self.offerlist, nick, mc) + if self.use_fidelity_bond: + taker_nick = nick + maker_nick = self.mcc.nick + d = self.callRemote(JMFidelityBondProofRequest, + takernick=taker_nick, + makernick=maker_nick) + self.defaultCallbacks(d) + else: + self.mcc.announce_orders(self.offerlist, nick, fidelity_bond_proof_msg=None, + new_mc=mc) @maker_only def on_order_fill(self, nick, oid, amount, taker_pk, commit): @@ -1015,6 +1038,7 @@ class JMDaemonServerProtocol(amp.AMP, OrderbookWatch): if self.mcc: self.mcc.shutdown() + class JMDaemonServerProtocolFactory(ServerFactory): protocol = JMDaemonServerProtocol diff --git a/jmdaemon/jmdaemon/fidelity_bond_sanity_check.py b/jmdaemon/jmdaemon/fidelity_bond_sanity_check.py new file mode 100644 index 0000000..44251f3 --- /dev/null +++ b/jmdaemon/jmdaemon/fidelity_bond_sanity_check.py @@ -0,0 +1,12 @@ + +import base64 + +def fidelity_bond_sanity_check(proof): + try: + decoded_data = base64.b64decode(proof, validate=True) + if len(decoded_data) != 252: + return False + except Exception: + return False + return True + diff --git a/jmdaemon/jmdaemon/message_channel.py b/jmdaemon/jmdaemon/message_channel.py index a5cbe6c..1abdee4 100644 --- a/jmdaemon/jmdaemon/message_channel.py +++ b/jmdaemon/jmdaemon/message_channel.py @@ -6,7 +6,8 @@ import threading from twisted.internet import reactor from jmdaemon import encrypt_encode, decode_decrypt, COMMAND_PREFIX,\ NICK_HASH_LENGTH, NICK_MAX_ENCODED, plaintext_commands,\ - encrypted_commands, commitment_broadcast_list, offername_list + encrypted_commands, commitment_broadcast_list, offername_list,\ + fidelity_bond_cmd_list from jmbase.support import get_log from functools import wraps @@ -284,12 +285,13 @@ class MessageChannelCollection(object): "; cannot find on any message channel.") return - def announce_orders(self, orderlist, nick=None, new_mc=None): + def announce_orders(self, orderlist, nick, fidelity_bond_proof_msg, new_mc): """Send orders defined in list orderlist either to the shared public channel (pit), on all message channels, if nick=None, or to an individual counterparty nick, as privmsg, on a specific mc. + Fidelity bonds can only be announced over privmsg, nick must be nonNone """ order_keys = ['oid', 'minsize', 'maxsize', 'txfee', 'cjfee'] orderlines = [] @@ -301,6 +303,7 @@ class MessageChannelCollection(object): "Tried to announce orders on an unavailable message channel.") return if nick is None: + assert fidelity_bond_proof_msg is None for mc in self.available_channels(): mc.announce_orders(orderlines) else: @@ -310,6 +313,9 @@ class MessageChannelCollection(object): cmd = orderlist[0]['ordertype'] msg = ' '.join(orderlines[0].split(' ')[1:]) msg += ''.join(orderlines[1:]) + if fidelity_bond_proof_msg: + msg += (COMMAND_PREFIX + fidelity_bond_cmd_list[0] + " " + + fidelity_bond_proof_msg) if new_mc: self.prepare_privmsg(nick, cmd, msg, mc=new_mc) else: @@ -563,7 +569,8 @@ class MessageChannelCollection(object): # orderbook watcher commands def register_orderbookwatch_callbacks(self, on_order_seen=None, - on_order_cancel=None): + on_order_cancel=None, + on_fidelity_bond_seen=None): """Special cases: on_order_seen: use it as a trigger for presence of nick. on_order_cancel: what happens if cancel/modify in one place @@ -572,7 +579,7 @@ class MessageChannelCollection(object): self.on_order_seen = on_order_seen for mc in self.mchannels: mc.register_orderbookwatch_callbacks(self.on_order_seen_trigger, - on_order_cancel) + on_order_cancel, on_fidelity_bond_seen) def on_orderbook_requested_trigger(self, nick, mc): """Update nicks_seen state to reflect presence of @@ -647,6 +654,7 @@ class MessageChannel(object): # orderbook watch functions self.on_order_seen = None self.on_order_cancel = None + self.on_fidelity_bond_seen = None # taker functions self.on_error = None self.on_pubkey = None @@ -730,9 +738,11 @@ class MessageChannel(object): # orderbook watcher commands def register_orderbookwatch_callbacks(self, on_order_seen=None, - on_order_cancel=None): + on_order_cancel=None, + on_fidelity_bond_seen=None): self.on_order_seen = on_order_seen self.on_order_cancel = on_order_cancel + self.on_fidelity_bond_seen = on_fidelity_bond_seen # taker commands def register_taker_callbacks(self, @@ -813,6 +823,21 @@ class MessageChannel(object): return True return False + def check_for_fidelity_bond(self, nick, _chunks): + if _chunks[0] in fidelity_bond_cmd_list: + try: + fidelity_bond_proof_msg = _chunks[1] + if self.on_fidelity_bond_seen: + self.on_fidelity_bond_seen(nick, _chunks[0], fidelity_bond_proof_msg) + except IndexError as e: + log.debug(e) + log.debug('index error parsing chunks, possibly malformed ' + 'offer by other party. No user action required. ' + 'Triggered by: ' + str(nick)) + finally: + return True + return False + def cancel_orders(self, oid_list): clines = [COMMAND_PREFIX + 'cancel ' + str(oid) for oid in oid_list] self.pubmsg(''.join(clines)) @@ -952,6 +977,8 @@ class MessageChannel(object): # orderbook watch commands if self.check_for_orders(nick, _chunks): pass + elif self.check_for_fidelity_bond(nick, _chunks): + pass # taker commands elif _chunks[0] == 'error': error = " ".join(_chunks[1:]) diff --git a/jmdaemon/jmdaemon/orderbookwatch.py b/jmdaemon/jmdaemon/orderbookwatch.py index 62edcb7..abd8b57 100644 --- a/jmdaemon/jmdaemon/orderbookwatch.py +++ b/jmdaemon/jmdaemon/orderbookwatch.py @@ -7,6 +7,7 @@ from decimal import InvalidOperation, Decimal from numbers import Integral from jmdaemon.protocol import JM_VERSION +from jmdaemon import fidelity_bond_sanity_check from jmbase.support import get_log, joinmarket_alert, DUST_THRESHOLD log = get_log() @@ -21,13 +22,12 @@ def dict_factory(cursor, row): class JMTakerError(Exception): pass - class OrderbookWatch(object): def set_msgchan(self, msgchan): self.msgchan = msgchan self.msgchan.register_orderbookwatch_callbacks(self.on_order_seen, - self.on_order_cancel) + self.on_order_cancel, self.on_fidelity_bond_seen) self.msgchan.register_channel_callbacks( self.on_welcome, self.on_set_topic, None, self.on_disconnect, self.on_nick_leave, None) @@ -41,6 +41,8 @@ class OrderbookWatch(object): self.db.execute("CREATE TABLE orderbook(counterparty TEXT, " "oid INTEGER, ordertype TEXT, minsize INTEGER, " "maxsize INTEGER, txfee INTEGER, cjfee TEXT);") + self.db.execute("CREATE TABLE fidelitybonds(counterparty TEXT, " + "takernick TEXT, proof TEXT);"); finally: self.dblock.release() @@ -134,11 +136,29 @@ class OrderbookWatch(object): finally: self.dblock.release() + def on_fidelity_bond_seen(self, nick, bond_type, fidelity_bond_proof_msg): + taker_nick = self.msgchan.nick + maker_nick = nick + if not fidelity_bond_sanity_check.fidelity_bond_sanity_check(fidelity_bond_proof_msg): + log.debug("Failed to verify fidelity bond for {}, skipping." + .format(maker_nick)) + return + try: + self.dblock.acquire(True) + self.db.execute("DELETE FROM fidelitybonds WHERE counterparty=?;", + (nick, )) + self.db.execute("INSERT INTO fidelitybonds VALUES(?, ?, ?);", + (nick, taker_nick, fidelity_bond_proof_msg)) + finally: + self.dblock.release() + def on_nick_leave(self, nick): try: self.dblock.acquire(True) self.db.execute('DELETE FROM orderbook WHERE counterparty=?;', (nick,)) + self.db.execute('DELETE FROM fidelitybonds WHERE counterparty=?;', + (nick,)) finally: self.dblock.release() @@ -146,5 +166,6 @@ class OrderbookWatch(object): try: self.dblock.acquire(True) self.db.execute('DELETE FROM orderbook;') + self.db.execute('DELETE FROM fidelitybonds;') finally: self.dblock.release() diff --git a/jmdaemon/jmdaemon/protocol.py b/jmdaemon/jmdaemon/protocol.py index 7d531aa..eefe72d 100644 --- a/jmdaemon/jmdaemon/protocol.py +++ b/jmdaemon/jmdaemon/protocol.py @@ -20,6 +20,8 @@ offertypes = {"reloffer": [(int, "oid"), (int, "minsize"), (int, "maxsize"), offername_list = list(offertypes.keys()) +fidelity_bond_cmd_list = ["tbond"] + ORDER_KEYS = ['counterparty', 'oid', 'ordertype', 'minsize', 'maxsize', 'txfee', 'cjfee'] diff --git a/jmdaemon/test/test_daemon_protocol.py b/jmdaemon/test/test_daemon_protocol.py index 26083e8..8d44d47 100644 --- a/jmdaemon/test/test_daemon_protocol.py +++ b/jmdaemon/test/test_daemon_protocol.py @@ -84,7 +84,8 @@ class JMTestClientProtocol(JMBaseProtocol): show_receipt("JMUP") d = self.callRemote(JMSetup, role="TAKER", - initdata="none") + offers="{}", + use_fidelity_bond=False) self.defaultCallbacks(d) return {'accepted': True} @@ -110,7 +111,7 @@ class JMTestClientProtocol(JMBaseProtocol): self.defaultCallbacks(d) @JMOffers.responder - def on_JM_OFFERS(self, orderbook): + def on_JM_OFFERS(self, orderbook, fidelitybonds): if end_early: return {'accepted': True} jlog.debug("JMOFFERS" + str(orderbook)) diff --git a/jmdaemon/test/test_message_channel.py b/jmdaemon/test/test_message_channel.py index 9ca3891..684128e 100644 --- a/jmdaemon/test/test_message_channel.py +++ b/jmdaemon/test/test_message_channel.py @@ -194,13 +194,13 @@ def test_setup_mc(): del mcc.active_channels[cp1] mcc.prepare_privmsg(cp1, "auth", "a b c") #try announcing orders; first public - mcc.announce_orders(t_orderbook) + mcc.announce_orders(t_orderbook, nick=None, fidelity_bond_proof_msg=None, new_mc=None) #try on fake mc - mcc.announce_orders(t_orderbook, new_mc="fakemc") + mcc.announce_orders(t_orderbook, nick=None, fidelity_bond_proof_msg=None, new_mc="fakemc") #direct to one cp - mcc.announce_orders(t_orderbook, nick=cp1) + mcc.announce_orders(t_orderbook, nick=cp1, fidelity_bond_proof_msg=None, new_mc=None) #direct to one cp on one mc - mcc.announce_orders(t_orderbook, nick=cp1, new_mc=dmcs[0]) + mcc.announce_orders(t_orderbook, nick=cp1, fidelity_bond_proof_msg=None, new_mc=dmcs[0]) #Next, set up 6 counterparties and fill their offers, #send txs to them cps = [make_valid_nick(i) for i in range(1, 7)] diff --git a/jmdaemon/test/test_orderbookwatch.py b/jmdaemon/test/test_orderbookwatch.py index d02be08..1232787 100644 --- a/jmdaemon/test/test_orderbookwatch.py +++ b/jmdaemon/test/test_orderbookwatch.py @@ -3,9 +3,12 @@ import pytest from jmdaemon.orderbookwatch import OrderbookWatch -from jmdaemon import IRCMessageChannel +from jmdaemon import IRCMessageChannel, fidelity_bond_cmd_list from jmclient import get_irc_mchannels, load_test_config from jmdaemon.protocol import JM_VERSION, ORDER_KEYS +from jmbase.support import hextobin +from jmclient.fidelity_bond import FidelityBondProof + class DummyDaemon(object): def request_signature_verify(self, a, b, c, d, e, f, g, h): @@ -122,7 +125,234 @@ def test_disconnect_leave(): rows = ob.db.execute('SELECT * FROM orderbook;').fetchall() orderbook = [dict([(k, o[k]) for k in ORDER_KEYS]) for o in rows] assert len(orderbook) == 0 - - - +@pytest.mark.parametrize( + "valid, fidelity_bond_proof, maker_nick, taker_nick", + [ + ( + True, + { + #nicksig len = 71, certsig len = 71 + "nick-signature": (b'0E\x02!\x00\xdbb\x15\x96\xa0\x87\xb8\x1d\xe05\xddV\xa1\x1bn\x8f' + + b'q\x90&\x8cG@\x89"2\xb2\x81\x9b\xc00\xa5\xb6\x02 \x03\x14l\xd7BR\xba\x8c:\x88(' + + b'\x8e3l\xac\xf5`T\x87\xfa\xf5\xa9\x1f\x19\xc0\xb6\xe9\xbb\xdc\xc7y\x99'), + "certificate-signature": ("3045022100eb512af938113badb4d7b29e0c22061c51dadb113a9395e" + + "9ed81a46103391213022029170de414964f07228c4f0d404b1386272bae337f0133f1329d948a" + + "252fa2a0"), + "certificate-pubkey": "0258efb077960d6848f001904857f062fa453de26c1ad8736f55497254f56e8a74", + "certificate-expiry": 1, + "utxo-pubkey": "02f54f027377e84171296453828aa863c23fc4489453025f49bd3addfb3a359b3d", + "txid": "84c88fafe0bb75f507fe3bfb29a93d10b2e80c15a63b2943c1a5fecb5a55cba2", + "vout": 0, + "locktime": 1640995200 + }, + "J5A4k9ecQzRRDfBx", + "J55VZ6U6ZyFDNeuv" + ), + ( + True, + { + #nicksig len = 71, certsig len = 70 + "nick-signature": (b'0E\x02!\x00\x80\xc6$\x0c\xa1\x15YS\xacHB\xb33\xfa~\x9f\xb9`\xb3' + + b'\xfe\xed0\xadHq\xc1~\x03.B\xbb#\x02 #y~]\xd9\xbbX2\xc0\x1b\xe57\xf4\x0f\x1f' + + b'\xd6$\x01\xf9\x15Z\xc9X\xa5\x18\xbe\x83\x1a&4Y\xd4'), + "certificate-signature": ("304402205669ea394f7381e9abf0b3c013fac2b79d24c02feb86ff153" + + "cff83c658d7cf7402200b295ace655687f80738f3733c1dc5f1e2b8f351c017a05b8bd31983dd" + + "4d723f"), + "certificate-pubkey": "031d1c006a6310dbdf57341efc19c3a43c402379d7ccd2480416cadc7579f973f7", + "certificate-expiry": 1, + "utxo-pubkey": "02616c56412eb738a9eacfb0550b43a5a2e77e5d5205ea9e2ca8dfac34e50c9754", + "txid": "84c88fafe0bb75f507fe3bfb29a93d10b2e80c15a63b2943c1a5fecb5a55cba2", + "vout": 1, + "locktime": 1893456000 + }, + "J54LS6YyJPoseqFS", + "J55VZ6U6ZyFDNeuv" + ), + ( + True, + { #nicksig len = 70, certsig len = 71 + "nick-signature": (b'0D\x02 K)\xe9\x17d\x0b\xc0\x82(\xd1\xa2*l\xd8\x0eJ\xc7\x01NV\xbf' + + b'\xcb\x02O]\xc0\x11\x01\x01B"\xed\x02 ob\xa1\xf8>\x80U)\xc8\x96\x86\x1b \x0e' + + b'\x00.\xf8\x86}\xcd\xf8\x82T\xa2\xb5\x8a4\xdb4\xbe\xf3{'), + "certificate-signature": ("3045022100d3beb5660bef33d095f92a3023bbbab15ece48ab2f211fa" + + "935b62fe8b764c8c002204892deffb4c9aa0d734aa3f55cc8e2baae4a03fc5a9e571b4f671493" + + "f1254df9"), + "certificate-pubkey": "03a2d1d15290d6d21204d1153c062970b4ff757a675e47a451fd0ba5c084127807", + "certificate-expiry": 1, + "utxo-pubkey": "03b9c12c9c31286772349b986653d07232327b284bd0787ad5829a04ac68f59b89", + "txid": "70c2995b283db086813d97817264f10b8823b870298d30ab09cb43c6bf2670cf", + "vout": 0, + "locktime": 1735689600 + }, + "J59PRzM6ZsdA5uyJ", + "J55VZ6U6ZyFDNeuv" + ), + ( + False, + { #nick signature with no DER header + "nick-signature": (b'ZD\x02 K)\xe9\x17d\x0b\xc0\x82(\xd1\xa2*l\xd8\x0eJ\xc7\x01NV\xbf' + + b'\xcb\x02O]\xc0\x11\x01\x01B"\xed\x02 ob\xa1\xf8>\x80U)\xc8\x96\x86\x1b \x0e' + + b'\x00.\xf8\x86}\xcd\xf8\x82T\xa2\xb5\x8a4\xdb4\xbe\xf3{'), + "certificate-signature": ("3045022100d3beb5660bef33d095f92a3023bbbab15ece48ab2f211fa" + + "935b62fe8b764c8c002204892deffb4c9aa0d734aa3f55cc8e2baae4a03fc5a9e571b4f671493" + + "f1254df9"), + "certificate-pubkey": "03a2d1d15290d6d21204d1153c062970b4ff757a675e47a451fd0ba5c084127807", + "certificate-expiry": 1, + "utxo-pubkey": "03b9c12c9c31286772349b986653d07232327b284bd0787ad5829a04ac68f59b89", + "txid": "70c2995b283db086813d97817264f10b8823b870298d30ab09cb43c6bf2670cf", + "vout": 0, + "locktime": 1735689600 + }, + "J59PRzM6ZsdA5uyJ", + "J55VZ6U6ZyFDNeuv" + ), + ( + False, + { #nick signature which fails ecdsa_verify + "nick-signature": (b'0E\x02 K)\xe9\x17d\x0b\xc0\x82(\xd1\xa2*l\xd8\x0eJ\xc7\x01NV\xbf' + + b'\xcb\x02O]\xc0\x11\x01\x01B"\xed\x02 ob\xa1\xf8>\x80U)\xc8\x96\x86\x1b \x0e' + + b'\x00.\xf8\x86}\xcd\xf8\x82T\xa2\xb5\x8a4\xdb4\xbe\xf3{'), + "certificate-signature": ("3045022100d3beb5660bef33d095f92a3023bbbab15ece48ab2f211fa" + + "935b62fe8b764c8c002204892deffb4c9aa0d734aa3f55cc8e2baae4a03fc5a9e571b4f671493" + + "f1254df9"), + "certificate-pubkey": "03a2d1d15290d6d21204d1153c062970b4ff757a675e47a451fd0ba5c084127807", + "certificate-expiry": 1, + "utxo-pubkey": "03b9c12c9c31286772349b986653d07232327b284bd0787ad5829a04ac68f59b89", + "txid": "70c2995b283db086813d97817264f10b8823b870298d30ab09cb43c6bf2670cf", + "vout": 0, + "locktime": 1735689600 + }, + "J59PRzM6ZsdA5uyJ", + "J55VZ6U6ZyFDNeuv" + ), + ( + False, + { #cert signature which fails ecdsa_verify + "nick-signature": (b'0D\x02 K)\xe9\x17d\x0b\xc0\x82(\xd1\xa2*l\xd8\x0eJ\xc7\x01NV\xbf' + + b'\xcb\x02O]\xc0\x11\x01\x01B"\xed\x02 ob\xa1\xf8>\x80U)\xc8\x96\x86\x1b \x0e' + + b'\x00.\xf8\x86}\xcd\xf8\x82T\xa2\xb5\x8a4\xdb4\xbe\xf3{'), + "certificate-signature": ("3055022100d3beb5660bef33d095f92a3023bbbab15ece48ab2f211fa" + + "935b62fe8b764c8c002204892deffb4c9aa0d734aa3f55cc8e2baae4a03fc5a9e571b4f671493" + + "f1254df9"), + "certificate-pubkey": "03a2d1d15290d6d21204d1153c062970b4ff757a675e47a451fd0ba5c084127807", + "certificate-expiry": 1, + "utxo-pubkey": "03b9c12c9c31286772349b986653d07232327b284bd0787ad5829a04ac68f59b89", + "txid": "70c2995b283db086813d97817264f10b8823b870298d30ab09cb43c6bf2670cf", + "vout": 0, + "locktime": 1735689600 + }, + "J59PRzM6ZsdA5uyJ", + "J55VZ6U6ZyFDNeuv" + ) + ]) +def test_fidelity_bond_seen(valid, fidelity_bond_proof, maker_nick, taker_nick): + proof = FidelityBondProof( + maker_nick, taker_nick, hextobin(fidelity_bond_proof['certificate-pubkey']), + fidelity_bond_proof['certificate-expiry'], + hextobin(fidelity_bond_proof['certificate-signature']), + (hextobin(fidelity_bond_proof['txid']), fidelity_bond_proof['vout']), + hextobin(fidelity_bond_proof['utxo-pubkey']), fidelity_bond_proof['locktime'] + ) + serialized = proof._serialize_proof_msg(fidelity_bond_proof['nick-signature']) + + ob = get_ob() + ob.msgchan.nick = taker_nick + ob.on_fidelity_bond_seen(maker_nick, fidelity_bond_cmd_list[0], serialized) + rows = ob.db.execute("SELECT * FROM fidelitybonds;").fetchall() + assert len(rows) == 1 + assert rows[0]["counterparty"] == maker_nick + assert rows[0]["takernick"] == taker_nick + try: + parsed_proof = FidelityBondProof.parse_and_verify_proof_msg(rows[0]["counterparty"], + rows[0]["takernick"], rows[0]["proof"]) + except ValueError: + parsed_proof = None + if valid: + assert parsed_proof is not None + assert parsed_proof.utxo[0] == hextobin(fidelity_bond_proof["txid"]) + assert parsed_proof.utxo[1] == fidelity_bond_proof["vout"] + assert parsed_proof.locktime == fidelity_bond_proof["locktime"] + assert parsed_proof.cert_expiry == fidelity_bond_proof["certificate-expiry"] + assert parsed_proof.utxo_pub == hextobin(fidelity_bond_proof["utxo-pubkey"]) + else: + assert parsed_proof is None + +def test_duplicate_fidelity_bond_rejected(): + + fidelity_bond_info = ( + ( + { + "nick-signature": (b'0E\x02!\x00\xdbb\x15\x96\xa0\x87\xb8\x1d\xe05\xddV\xa1\x1bn\x8f' + + b'q\x90&\x8cG@\x89"2\xb2\x81\x9b\xc00\xa5\xb6\x02 \x03\x14l\xd7BR\xba\x8c:\x88(' + + b'\x8e3l\xac\xf5`T\x87\xfa\xf5\xa9\x1f\x19\xc0\xb6\xe9\xbb\xdc\xc7y\x99'), + "certificate-signature": ("3045022100eb512af938113badb4d7b29e0c22061c51dadb113a9395e" + + "9ed81a46103391213022029170de414964f07228c4f0d404b1386272bae337f0133f1329d948a" + + "252fa2a0"), + "certificate-pubkey": "0258efb077960d6848f001904857f062fa453de26c1ad8736f55497254f56e8a74", + "certificate-expiry": 1, + "utxo-pubkey": "02f54f027377e84171296453828aa863c23fc4489453025f49bd3addfb3a359b3d", + "txid": "84c88fafe0bb75f507fe3bfb29a93d10b2e80c15a63b2943c1a5fecb5a55cba2", + "vout": 0, + "locktime": 1640995200 + }, + "J5A4k9ecQzRRDfBx", + "J55VZ6U6ZyFDNeuv" + ), + ( + { + "nick-signature": (b'0E\x02!\x00\x80\xc6$\x0c\xa1\x15YS\xacHB\xb33\xfa~\x9f\xb9`\xb3' + + b'\xfe\xed0\xadHq\xc1~\x03.B\xbb#\x02 #y~]\xd9\xbbX2\xc0\x1b\xe57\xf4\x0f\x1f' + + b'\xd6$\x01\xf9\x15Z\xc9X\xa5\x18\xbe\x83\x1a&4Y\xd4'), + "certificate-signature": ("304402205669ea394f7381e9abf0b3c013fac2b79d24c02feb86ff153" + + "cff83c658d7cf7402200b295ace655687f80738f3733c1dc5f1e2b8f351c017a05b8bd31983dd" + + "4d723f"), + "certificate-pubkey": "031d1c006a6310dbdf57341efc19c3a43c402379d7ccd2480416cadc7579f973f7", + "certificate-expiry": 1, + "utxo-pubkey": "02616c56412eb738a9eacfb0550b43a5a2e77e5d5205ea9e2ca8dfac34e50c9754", + "txid": "84c88fafe0bb75f507fe3bfb29a93d10b2e80c15a63b2943c1a5fecb5a55cba2", + "vout": 1, + "locktime": 1893456000 + }, + "J54LS6YyJPoseqFS", + "J55VZ6U6ZyFDNeuv" + ) + ) + + ob = get_ob() + + fidelity_bond_proof1, maker_nick1, taker_nick1 = fidelity_bond_info[0] + proof = FidelityBondProof( + maker_nick1, taker_nick1, hextobin(fidelity_bond_proof1['certificate-pubkey']), + fidelity_bond_proof1['certificate-expiry'], + hextobin(fidelity_bond_proof1['certificate-signature']), + (hextobin(fidelity_bond_proof1['txid']), fidelity_bond_proof1['vout']), + hextobin(fidelity_bond_proof1['utxo-pubkey']), fidelity_bond_proof1['locktime'] + ) + serialized1 = proof._serialize_proof_msg(fidelity_bond_proof1['nick-signature']) + ob.msgchan.nick = taker_nick1 + + ob.on_fidelity_bond_seen(maker_nick1, fidelity_bond_cmd_list[0], serialized1) + rows = ob.db.execute("SELECT * FROM fidelitybonds;").fetchall() + assert len(rows) == 1 + + #show the same fidelity bond message again, check it gets rejected as duplicate + ob.on_fidelity_bond_seen(maker_nick1, fidelity_bond_cmd_list[0], serialized1) + rows = ob.db.execute("SELECT * FROM fidelitybonds;").fetchall() + assert len(rows) == 1 + + #show a different fidelity bond and check it does get accepted + fidelity_bond_proof2, maker_nick2, taker_nick2 = fidelity_bond_info[1] + proof2 = FidelityBondProof( + maker_nick1, taker_nick1, hextobin(fidelity_bond_proof2['certificate-pubkey']), + fidelity_bond_proof2['certificate-expiry'], + hextobin(fidelity_bond_proof2['certificate-signature']), + (hextobin(fidelity_bond_proof2['txid']), fidelity_bond_proof2['vout']), + hextobin(fidelity_bond_proof2['utxo-pubkey']), fidelity_bond_proof2['locktime'] + ) + serialized2 = proof2._serialize_proof_msg(fidelity_bond_proof2['nick-signature']) + ob.msgchan.nick = taker_nick2 + + ob.on_fidelity_bond_seen(maker_nick2, fidelity_bond_cmd_list[0], serialized2) + rows = ob.db.execute("SELECT * FROM fidelitybonds;").fetchall() + assert len(rows) == 2 diff --git a/scripts/obwatch/ob-watcher.py b/scripts/obwatch/ob-watcher.py index f5339ab..fe6cf7e 100755 --- a/scripts/obwatch/ob-watcher.py +++ b/scripts/obwatch/ob-watcher.py @@ -16,12 +16,18 @@ from future.moves.urllib.parse import parse_qs from decimal import Decimal from optparse import OptionParser from twisted.internet import reactor +from datetime import datetime if sys.version_info < (3, 7): print("ERROR: this script requires at least python 3.7") exit(1) from jmbase.support import EXIT_FAILURE +from jmbase import bintohex +from jmclient import FidelityBondMixin, get_interest_rate +from jmclient.fidelity_bond import FidelityBondProof + +import sybil_attack_calculations as sybil from jmbase import get_log log = get_log() @@ -86,31 +92,45 @@ def cjfee_display(cjfee, order, btc_unit, rel_unit): return str(Decimal(cjfee) * Decimal(rel_unit_to_factor[rel_unit])) + rel_unit -def satoshi_to_unit(sat, order, btc_unit, rel_unit): - power = unit_to_power[btc_unit] +def satoshi_to_unit_power(sat, power): return ("%." + str(power) + "f") % float( Decimal(sat) / Decimal(10 ** power)) +def satoshi_to_unit(sat, order, btc_unit, rel_unit): + return satoshi_to_unit_power(sat, unit_to_power[btc_unit]) def order_str(s, order, btc_unit, rel_unit): return str(s) -def create_table_heading(btc_unit, rel_unit): +def create_offerbook_table_heading(btc_unit, rel_unit): col = ' {1}\n' # .format(field,label) tableheading = '\n ' + ''.join( [ - col.format('ordertype', 'Type'), col.format( - 'counterparty', 'Counterparty'), + col.format('ordertype', 'Type'), + col.format('counterparty', 'Counterparty'), col.format('oid', 'Order ID'), - col.format('cjfee', 'Fee'), col.format( - 'txfee', 'Miner Fee Contribution / ' + btc_unit), - col.format( - 'minsize', 'Minimum Size / ' + btc_unit), col.format( - 'maxsize', 'Maximum Size / ' + btc_unit) + col.format('cjfee', 'Fee'), + col.format('txfee', 'Miner Fee Contribution / ' + btc_unit), + col.format('minsize', 'Minimum Size / ' + btc_unit), + col.format('maxsize', 'Maximum Size / ' + btc_unit), + col.format('bondvalue', 'Bond value / ' + btc_unit + '²') ]) + ' ' return tableheading +def create_bonds_table_heading(btc_unit): + tableheading = ('
' + + '' + + '' + + '' + + '' + + '' + + '' + + '' + + '' + + '' + ) + return tableheading def create_choose_units_form(selected_btc, selected_rel): choose_units_form = ( @@ -128,6 +148,52 @@ def create_choose_units_form(selected_btc, selected_rel): '" + bondtable = "" + for (bond_data, utxo_data), bond_value, conf_time in zip( + fidelity_bond_data, fidelity_bond_values, bond_outpoint_conf_times): + + if bond_value == -1 or conf_time == -1 or utxo_data == None: + bond_value_str = "No data" + conf_time_str = "No data" + utxo_value_str = "No data" + else: + bond_value_str = satoshi_to_unit_power(bond_value, 2*unit_to_power[btc_unit]) + conf_time_str = str(datetime.utcfromtimestamp(conf_time)) + utxo_value_str = satoshi_to_unit(utxo_data["value"], None, btc_unit, 0) + bondtable += ("" + + elem(bond_data.maker_nick) + + elem(bintohex(bond_data.utxo[0]) + ":" + str(bond_data.utxo[1])) + + elem(bond_value_str) + + elem(datetime.utcfromtimestamp(bond_data.locktime).strftime("%Y-%m-%d")) + + elem(utxo_value_str) + + elem(conf_time_str) + + elem(str(bond_data.cert_expiry*RETARGET_INTERVAL)) + + elem(bintohex(btc.mk_freeze_script(bond_data.utxo_pub, + bond_data.locktime))) + + "" + ) + + heading2 = (str(len(fidelity_bond_data)) + " fidelity bonds found with " + + total_btc_committed_str + " " + btc_unit + + " total locked up") + choose_units_form = ( + '' + + '') + choose_units_form = choose_units_form.replace( + '
CounterpartyUTXOBond value / ' + btc_unit + '²LocktimeLocked coins / ' + btc_unit + 'Confirmation timeSignature expiry heightRedeem script
" + e + "
" + + decodescript_tip) + + def create_sybil_resistance_page(self, btc_unit): + if jm_single().bc_interface == None: + return "", "Calculations unavailable, requires configured bitcoin node." + + (fidelity_bond_data, fidelity_bond_values, bond_outpoint_conf_times) =\ + get_fidelity_bond_data(self.taker) + + choose_units_form = ( + '
' + + '
') + choose_units_form = choose_units_form.replace( + '