This week’s newsletter links to a discussion about using enhanced QR codes for communicating large transactions, includes a field report from Suredbits about building a high-availability LN node, and briefly summarizes several recently transcribed talks and conversations. Also included are our regular sections with releases, release candidates, and notable code changes from popular Bitcoin infrastructure software.

Action items

Happy halving!

News

  • ● QR codes for large transactions: QR codes can practically contain up to about 3 kilobytes of data, which is enough to accommodate typical user transactions but far too small to contain the largest transactions users can normally send. Riccardo Casatta and Christopher Allen each posted to the Bitcoin-Dev mailing list (1, 2) a request for discussion hopefully leading to standardization of a method for visual communication of Partially Signed Bitcoin Transactions (PSBTs) and other potentially large data blobs related to Bitcoin wallet interaction. See the previous discussion in the Specter DIY repository and continue the discussion in the Airgapped Signing repository.

Field report: running a Lightning node in an enterprise environment

by Roman Taranchenko, Engineer at Suredbits

After the first excitement of sending and, more importantly, receiving a payment over the Lightning Network has faded away, it’s always good to think about how to operate your node in a safe and reliable way. Failures almost always happen unexpectedly. How do you recover after a possible failure? How do you make backups reliable? How do you keep the seed in a secure location? Et cetera, et cetera…

At Suredbits we use Eclair for our nodes. Even though Eclair is pretty robust on its own, we took some steps to make it even more reliable—such as using PostgreSQL as a database backend (using this PR ) and AWS Secrets Manager to store private keys.

Eclair has a built-in online backup feature, but it requires manual setup and script writing to automate, which is not really scalable and is error prone. Running PostgreSQL at AWS RDS allows us to automate backups and replication in a way that is familiar to many DevOps engineers and which makes restoring the database state easier.

Using PostgreSQL as a remote database backend makes node failover simpler to implement because, if the node crashes for some reason, there’s no need to restore the database from a backup—all you need is to point a new Eclair instance to the correct database server. Here’s a quick demo of an automated failover implemented with two Eclair instances plus AWS’s RDS, ELB, and NAT Gateway.

In the failover scenario depicted in the demo, we needed a secure way to allow the node’s seed for its private keys to be shared between the Eclair instances. Eclair stores the seed in a file on the local file system which should be backed up somewhere and restored when needed. The current Eclair implementation requires extra steps to do so in an automated fashion. We instead use AWS Secrets Manager—an encrypted key/value store specifically designed to securely hold various kinds of secrets including database passwords and encryption keys. Now all you need to do to share the seed between the instances is to point them to the correct secrets location in the config file. Once configured, the instance can be stored as an AMI image which can re-imaged as many times as needed without manual configuration.

The measures we took are just the first steps to building enterprise-grade Lightning nodes. There are still some more problems that need to be solved. For example, which Hardware Security Module (HSM) can be used for a Lightning node, or how to failover a Bitcoin Core node in a multi-instance setting. But we believe that our work is a solid base for scaling out Eclair and making it more fault-tolerant.

For more information about this topic, see our presentation.

Disclaimer: Since private keys are involved, don’t use third party cloud services without a thorough risk assessment.

Recently transcribed talks and conversations

Bitcoin Transcripts is the home for transcripts of technical Bitcoin presentations and discussions. In this new monthly feature, we highlight a selection of the transcripts from the previous month.

  • ● Simplicity—Next-Generation Smart Contracting Adam Back presented at a Blockstream webinar on Simplicity, a next-generation, low-level alternative to Bitcoin Script focused on provable security and expressiveness. Back discussed how Simplicity, if enabled in Bitcoin today, would allow developers to implement new functionality like SIGHASH_NOINPUT without necessarily needing a soft fork. He also displayed a demo showing what you can do with Simplicity today. (transcript, video, slides)

  • ● Attacking Bitcoin Core Amiti Uttarwar presented at LA BitDevs. Uttarwar discussed how changes to Bitcoin’s peer-to-peer layer should be assessed according to five goals: reliability, timeliness, accessibility, privacy, and upgradability. She discussed the dangers of network partitions and eclipse attacks—and then explained why block-relay-only connections and anchor nodes are effective mitigations. (transcript, video)

  • ● LND v0.10 Laolu Osuntokun, Joost Jager, and Oliver Gugger presented in virtual reality at Reckless VR. Osuntokun presented Tor and RPC enhancements in the latest release of LND plus a new channel feature called anchor outputs that addresses the challenge of estimating onchain fees months in advance. Jager discussed the challenges of multipart payments including the splitting algorithm, what happens when the shards of the payment arrive at different times, and strategies for handling multipart payment failures. Gugger ended by discussing Partially Signed Bitcoin Transaction (PSBT) channel funding and the channel abstraction work that made this possible. (transcript, video)

  • ● Grokking Bitcoin Kalle Rosenbaum participated in a Bitcoin developer meetup and gave a presentation at London Bitcoin Devs. The meetup discussion focused on the role of Bitcoin technical education, BIP32 HD wallets, and soft fork upgrades. For the presentation, Rosenbaum used the content in his book to discuss how the segwit upgrade of 2017 addressed transaction malleability and quadratic hashing. (Meetup transcript, presentation transcript, presentation video, presentation slides)

Releases and release candidates

New releases and release candidates for popular Bitcoin infrastructure projects. Please consider upgrading to new releases or helping to test release candidates.

Notable code and documentation changes

Notable changes this week in Bitcoin Core, C-Lightning, Eclair, LND, libsecp256k1, Rust-Lightning, Bitcoin Improvement Proposals (BIPs), and Lightning BOLTs.

Note: the commits to Bitcoin Core mentioned below apply to its master development branch and so those changes will likely not be released until version 0.21, about six months after the release of the upcoming version 0.20.

  • ● Bitcoin Core #16528 allows the createwallet RPC to create a wallet that uses output script descriptors to derive just the particular scriptPubKeys that the wallet uses to receive payments. This is a major improvement over the way legacy wallets scan for payments by deriving every type of script handled by the wallet for each public key in the wallet—a technique that dates back to the original Bitcoin 0.1 release’s support for receiving to both P2PK and P2PKH scriptPubKeys. Descriptor wallets should be more efficient (because they don’t need to scan for unused script types), easier to upgrade to new types of script (e.g. for taproot), and easier to use with external tools (e.g. multisig wallets or HWI-compatible hardware wallets via partially-signed bitcoin transactions).

    By default, descriptor wallets use the popular BIP32 HD wallet paths specified by BIPs 44, 49, and 84 rather than the non-standardized fully-hardened path used in legacy Bitcoin Core HD wallets. A number of wallet RPCs are unavailable with descriptor wallets, either because they don’t make sense with descriptors or because developers are still adapting them to new edge cases. The merge of this PR early in the 0.21 development cycle and the decision to make descriptor wallets a non-default option will give the new features six months to mature before their expected release.

  • ● Bitcoin Core #18038 improves privacy when initially broadcasting a transaction by reducing the frequency that a wallet will attempt to resend from about 30 minutes to about once a day. Previously, an entity surveilling the network could see multiple broadcasts of the same transaction during these resend periods from the same node and conclude that wallet was the originator. By reducing the frequency of resending attempts from the wallet, the originator of the transaction has less chance of being identified.

    To ensure new transactions reach the network even without frequent indiscriminate rebroadcast by the wallet, this PR also adds a type of unbroadcast transaction within the mempool. An unbroadcast transaction is one that has been submitted locally via the wallet or RPC but which has not yet been successfully relayed to peers on the network. Such unbroadcast transactions remain in the mempool and will be rebroadcast every 10-15 minutes until a peer has fetched the transaction by sending the node a getdata P2P message for that transaction.

  • ● BIPs #893 makes several changes to the BIP340 specification of schnorr pubkeys and signatures, with related changes also being made to the BIP341 specification of taproot. The major changes include:

    • ● Alternative x-only pubkey tiebreaker: this changes which variant of a public key to use when only its x-coordinate is known, as previously discussed (see Newsletter #83).

    • ● Updated nonce generation recommendations: the suggested method to use for generating a signature nonce has been updated to prevent implementation-specific vulnerabilities (see previous discussion in Newsletters #83 and #91).

    • ● Updated tagged hashes: the tag prefixed to hash function input for schnorr signatures has been updated to deliberately break compatibility with the old draft specification. Otherwise, libraries written for the old code might sometimes generate signatures valid under the new code, and sometimes might not, leading to confusion. This was also previously mentioned in Newsletter #83.

  • ● BIPs #903 simplifies the BIP322 specification of generic signed messages as previously proposed (see Newsletter #91), the change mainly removes details that allowed signing the same message for multiple scripts (addresses) in the same proof.

  • ● BIPs #900 updates the BIP325 specification of signet so that all signets use the same hardcoded genesis block (block 0) but independent signets can be differentiated by their network magic (message start bytes). In the updated protocol, the message start bytes are the first four bytes of a hash digest of the network’s challenge script (the script used to determine whether a block has a valid signature). The change was motivated by a desire to simplify the development of applications that want to use multiple signets but which need to call libraries that hardcode the genesis blocks for the networks they support.