Extended service outage after Polygon network upgrade

Layer 2 ethereum scaling solution Polygon has not produced new blocks for more than 11 hours, and developers attribute the problem to a technical upgrade on the network.

On March 10th at 4:20pm, Polygon (MATIC) network developers informed users on the project’s forum that an outage may begin around 5:50pm due to maintenance on one of the network’s three layers. A recent upgrade is believed to have caused errors in the network's ability to reach consensus.

“We suspect that there may be a consensus-affecting bug in the upgrade and cause different Heimdall validators to sit on different versions of the chain, preventing 2/3 consensus.”

As expected, the outage started at 5:54pm UTC.

The Polygon team released an update around 1:30AM UTC, explaining that it was still working on fixing the bug and deployed a hotfix to help continue block production. As of this writing, the fix does not appear to be in effect.

The team also assured users that all funds and data on-chain are safe.

Polygon users clearly understood that this outage was to be expected, but by the 6th hour, complaints started to appear in the community. Most users pointed to concerns about extended downtime.

Still others noted that their crypto businesses were negatively impacted by the outage. Twitter user @Miklos211 said:

"I'm literally out of work, can you tell us when the network will hopefully be back to normal?"

The Polygon network consists of three layers, each with a unique role. Smart contracts run on the Ethereum (ETH) layer, while the Bor layer helps generate blocks. The third layer of Heimdall is the problem.

Heimdall is a group of proof-of-stake (PoS) nodes that help the network reach consensus. Polygon was unable to achieve network consensus due to the Heimdall problem.

The last major bug fixed by Polygon put $24 billion worth of funds at risk, but was fixed without issue in late 2021.

In related news, Polygon recently partnered with TokenTraxx, a multi-chain music NFT marketplace, to support the marketplace within the Polygon ecosystem. TokenTraxx will allow musicians to tokenize their compositions and sell them as NFTs.