Ezrinix - Notice history

At 15:15 UTC today, Wednesday, March 12th, 2025, we will be performing additional work on our on-site DDoS protection system. Please note that this has the possibility for disruptions. We are doing this during low traffic times to ensure we impact as little traffic as possible.

This decision has been made due to testing we have been conducting over the past few days and us hitting a milestone development yesterday that we shared with clients where no connections were disrupted when changing between global and on-site equipment.

As noted previously, this system is extremely complex. It has been under development for a while now, and we understand no downtime is ideal, however, this must be done to ensure we have all scenarios covered for how this system will react when in use with production traffic. I am sure everyone is tired of these upstream issues; we are too, and we truly know on-site is the solution we are all looking for.

We apologize for any inconvenience this may cause and appreciate your understanding. If you have any questions or concerns, please reach out to our support team.

We have been informed that Path Miami has been brought fully online today. Chicago is also expected to come online very soon and we have requested for it to come back online closer to off-peak hours to avoid serious interruptions.

You will notice a full outage while Path brings up Chicago and we are routed back onto Chicago. However after Chicago is brought up latency will start normalizing again as we will no longer be routed to different PoPs!

Hello everyone, our network team has identified the root cause of the frequent short drops that have plagued the network over the past few days. We understand the severity of these drops and have identified the root cause. To permanently resolve these issues, we will be conducting emergency maintenance at 9:00 AM Central Time. During this maintenance, you will experience brief disruptions as mass changes will be done. We apologize in advance for any inconvenience this may pose to you, especially over these past few weeks when we have suffered instability from one of our upstream providers as well as the network maintenance we conducted to upgrade our core late last month. To be frank, the timing could not have been worse.

For transparency, the issue stems from a Layer 2 issue inside our EVPN-VXLAN network, which is affecting our spine switches that process essentially all traffic on our network. As a result, this was felt widespread and not just related to one upstream or one cabinet. This issue appears to have sprung from the preparation of onboarding our on-prem protection suite into our core. We have been working at a fast pace to get that online and operational in hopes of shifting all traffic over to provide stability.

We know the past few weeks have been difficult, and we recognize the importance of a reliable network for our customers and are tirelessly working to improve our offerings.

At specifically 9AM Central Time we will be continuing development with our onsite network. We apologize for any interruptions this may cause, we are doing this early in the AM to ensure we impact as little traffic as possible.

We understand the frequent issues from upstream isn't making this an easy pill to swallow, but we must continue work on this to ensure we are independent from future network issues once Onsite goes live.

To give context on what onsite will bring for those who may have missed it:

Ezrinix Onsite DDoS Protection will allow us to have a localized Path environment that will be isolated from the global network. This will allow us to attempt to seamlessly, without dropping traffic move between the global network and local network when needed.

Take for example the issues that we have been facing now, these are all from the global network with Onsite we would be-able to push traffic onto our local mitigation stack, effectively isolating us from global network issues, ensuring full uptime. This also works in reverse, if the local network was to go offline for some reason, we would route back onto the global network, while again attempting to not drop any traffic.

This system is extremely complex, and has been under development for a while now, we are speeding up progress but also doing so at the least impactful times, we understand no downtime is ideal, however this must be done in order to ensure we have a fully redundant network. I am sure everyone is tired of these upstream issues, we are too, and we truly believe onsite is the solution we are all looking for.