Prysmatic Labs has put out a fix for Ethereum 2.0’s Medalla testnet after a bug on August 14 brought the testnet to its knees.
The Medalla Testnet is a public network designed to implement Ethereum 2.0’s proof-of-stake blockchain network upgrade. Just last week, the testnet had 26,000 people keeping the network running, with more than 830,000 testnet Ethereum in play. But a simple bug crashed the network in just a few hours.
How did the testnet crash?
The problem started when a third-party kit stopped working properly.
On Friday, Cloudflare roughtime—a public measure of time developed by Google—was off by about four hours. Cloudflare is a third-party web infrastructure provider that Prysm relies on and uses to adjust users’ clocks if they are skewed.
The bug caused havoc with the network’s internal time clocks. Different parts of the network were showing time lags of up to four hours.
The network’s validators, participants who run nodes to propose and validate blocks, began incorrectly proposing future blocks. Not only that but they could no longer process blocks properly or receive rewards and transaction fees.
As a result, the number of people on the testnet who were successfully validating blocks dropped significantly, from 75% to nearly 5%.
Another aspect was that Prysm’s nodes did not have a proper fallback in place in the event that the roughtime servers went down.
How are developers fixing the problem?
Prysmatic Labs initially made a few fixes before later publishing an upgrade, which includes a bug fix.
The Prysm team developed Alpha.22, which is the main bug fix for the Medalla testnet. It’s designed to provide “initial sync improvements that may assist in resolving ongoing sync issues in the Medalla test network.”
“We need all the help we can get to get the testnet back on track and updating your nodes is a great way to add more healthy peers to the network,” the Prysmatic Labs team announced via Twitter.
Van Loon, Ethereum protocol developer and co-founder of Prysmatic Labs, also underlined the importance of using a testnet in order to find these kinds of bugs—and to do so before the network goes live.