Тема: Что же было вчера (06.09.2011)

Вчера около трех часов дня выросли значительно пинги, с обычных 45мс до 150, а потом и 190.
Потом начали пропадать пакеты, и в треках это тоже заметно.
Виной тому порыв на магистрали оптоволокна Франкфурта, хронология всего остального ниже:

Type:     Fault report    
Categories:     Network
Start:     September 6, 2011 3:20:00 PM CEST
End:     September 6, 2011 11:20:00 PM CEST (expected)
Description:     Owing to a current fault, the Frankfurt/Main-Falkenstein backbone connection is not available. Our technicians are routing the traffic via other connections so that the fault should be rectified within a short space of time.
Affected:     All servers in datacenters RZ10-RZ15

Update:     September 6, 2011 4:05:00 PM CEST    
    Our technicians are busy distributing traffic from the faulty route to other connections which means that the situation will gradually improve. We shall inform you when all traffic has been successfully rerouted. Until then we kindly ask the customers concerned to be patient.

Update:     September 6, 2011 4:20:00 PM CEST    
    We have received an official statement from the line provider (Wingas):

>We are writing to inform you of a fault in the section of pipeline
>between Dürrengleina and Saaletal/Jena.
>Owing to excavation work a fibre optic cable has been cut through.
>The LWL (fibre-optic cable) service team is already on its way.
>As soon as we have new information regarding the progress made in
>rectifying the fault, we shall inform you immediately.

As a result of the fault, we are currently routing our traffic via redundant connections exclusively to Falkenstein, which in many cases leads to less optimal routing and partially to higher latency.

Update:     September 6, 2011 5:30:00 PM CEST    
    Our network is still affected by the current fiber cut. As repair work on the damaged optical fibers is expected to last several hours, we anticipate that the fault will only be finally cleared during the coming night. We apologize to all customers affected who have experienced performance problems as a result.

Update:     September 6, 2011 8:30:00 PM CEST    
    We have just received a status update from Wingas:
              According to recent information from the field, the repair work will still take several more hours.

Update:     September 6, 2011 9:18:00 PM CEST    
    We have received an update of our provider.

> die Reparaturarbeiten des beschädigten Glasfaserkabels werden aufgrund der
> massiven Beschädigung des Kabels und der Lage des Störungsfalles noch
> weitere Zeit in Anspruch nehmen.
> Die Kollegen des LWL-Teams arbeiten mit Hochdruck an der Behebung.

Update:     September 6, 2011 10:30:00 PM CEST    
    Wingas is still busy trying to repair the fiber optic cable as quickly as possible.

Update:     September 6, 2011 11:30:00 PM CEST    
    The fiber cable has been fixed. We are switching back to normal operations right now. We ares still closely monitoring the situation.

Отредактировано Noname (07-09-2011 09:36:55)