At Farcaster, our backend servers execute thousands of Ethereum transactions daily across multiple chains. This post covers a few challenges we faced building a production transaction broadcasting system that can reliably land transactions in spite of network congestion, gas price spikes, RPC failures, and other failure modes.
Note: this not about Farcaster Wallet transactions, which are submitted directly from the user’s device.
Our wallets need to be funded with different assets on different chains to support things like registering Farcaster accounts on Optimism, settling Collectible Cast auctions on Base, and paying out tens of thousands weekly USDC rewards.
For each transacting (wallet, chain, asset) we care about, we:
There is no magic formula for how much to keep funded or what schedule to notify on; it depends on the specific asset and how it is spent. For instance, we spend OP ETH gradually and notify anytime the balance drops below a predefined threshold, whereas Base USDC is spent in large batches and requires a weekly alert that checks the balance against an upcoming payouts.
One assumption we maintain is that our backend system can get compromised at any time, so we keep a minimal amount of funds while keeping the operational burden of transferring funds in at a reasonable level—we target funding once per week.
In addition to our primary transacting wallet, we keep a few fallback wallets funded. If we don’t respond to alerts in time and our primary wallet runs entirely out of funds, we automatically promote a fallback wallet and trigger a page that we need to fund the primary wallet and re-promote it.