There was an unscheduled downtime on 5th of July from about 13:00 EEST+3 (that's 10:00 UTC or 3:00 PDT-7) until about 1:30 EEST (22:30 UTC or 15:30 PDT) for a total of 12 hours 30 mins. There might have been problems with services for 2 hours or even longer before the start of downtime.

This was due to disk I/O failures on our main server that meant it had to be brought down for a disk check and temporary repairs. There was no critical data affected as the only file blocks permanently lost due to bad sectors were parts of log files. All other files were repaired successfully.

The affected disk will need to be replaced as it will most likely fail completely soon. This will require additional downtime in coming days.

For now, I'm bringing services back up slowly while the backlog gets processed.