Some systems are experiencing issues

Stickied Incidents

2024-06-07

Backup Backup outage

Multiple consecutive disk failure in the backup resulted in major outage in the backup service

We're investigating a proper solution.

Maintenance
Matrix maintenance

The Matrix databases are undergoing regular maintenance. All Matrix services may experience degraded performance or small outages.

Past Incidents

2025-03-17

Telegram bridge Matrix Telegram bridge

The Matrix Telegram bridge is experiencing a major outage.

2025-02-20

cPanel cPanel outage

At this time the website and mail component of Hornet (cPanel) is experiencing a major outage.

  • cPanel has been restarted and has resumed its normal operation.

    • 2025-02-09

    Homeserver Matrix

    The Matrix homeserver is experiencing issues loading media.

  • The issues with the media repository have been fixed

    • 2025-01-26

    Telegram bridge Telegram bridge maintenance

    The Telegram bridge's homeserver database is being moved to different storage. This is expected to take up to two hours.

  • The maintenance has been completed.

    • 2024-12-29

    IRC bridges Matrix-IRCnet outage

    The bridge stopped relaying messages around 21:45. The bridge was restarted in order to resolve the problem. It may take a while before all users have rejoined every channel.

  • The bridge has stopped relaying messages again, a temporary fix has been applied but further investigation will be conducted. It may take a while before all users have rejoined every channel.

    • 2024-10-30

    Cloud Cloud platform outage

    An issue with the cloud platform is causing outages on vColo machines and internal services.

  • An issue with the cloud platform is causing outages on vColo machines and internal services.

    • 2024-06-25

    cPanel Wordpress update failures

    An automatic Wordpress update has caused a couple of of sites hosted on cPanel to break. While this issue is not related to Hornet, we post this here as notification.

    For more information, see https://core.trac.wordpress.org/ticket/61489. If your site is affected, you can fix it yourself by replacing wp-includes/default-filters.php with the correct one from the Github mirror (note the version!): https://github.com/WordPress/wordpress-develop/blob/6.5.5/src/wp-includes/default-filters.php. You can use the wordpress CLI tool to verify the file checksums in your installation.

  • This issue should no longer occur.

  • It appears that an automated security patch applied by Patchman caused this issue, which is related to the use of an older Wordpress version. While they are working on a fix, we recommend all Wordpress users to upgrade their site to the latest available version.

    • 2024-06-07

    Backup Backup outage

    Multiple consecutive disk failure in the backup resulted in major outage in the backup service

    We're investigating a proper solution.

    2024-06-01

    Homeserver Matrix outage and performance issues

    Performance issues encountered during the regular cleanup are causing the homeserver to have severe performance issues.

  • Mitigations have had effect. Performance should be back to normal.

  • We have identified the issue and are working on deploying workarounds on our infrastructure.

  • We have reconfigured the homeserver to prioritise client requests. Clients should be somewhat functional, but messages between utwente.io and other homeservers experience significant delays (up to several hours).