Skip to main content

Update on murena.io service outage

 

GaelDuval
6d
Since Oct 6 19:50 CET, most murena.io 523 services have been unreachable.

Affected services

  • all murena.io 523 services including drive, calendar, notes and email (@murena.io / @e.email)

  • Over the air (OTA) /e/OS updates

  • App Lounge

All other websites including e.foundation, gitlab.e.foundation, community.e.foundation and murena.com 84 remain unaffected.

Context

The reason behind this outage is related to our storage infrastructure that is getting old while the number of murena.io 523 active users has grown a lot during the past two years.

An infrastructure evolution and consolidation was planned, but last week, our current infrastructure lost some storage nodes and switched to a degraded mode.

Unfortunately, yesterday we faced another issue that forced us to put the infrastructure in maintenance mode until it gets fully restored.

Actions we have taken:

  1. We are prioritizing email service recovery as soon as possible, and it is ongoing progress. We will update this today to give a first ETA.

  2. Meanwhile, we have already fixed the App Lounge service, which is now back, still in a degraded mode: only commercial apps (from Play Store) can currently be installed. An ETA for the service to be fully restored will be communicated later.

  3. We’re taking the current outage as an opportunity to accelerate the murena.io 523 storage infrastructure evolution and consolidation that was planned. New storage nodes have been added and the synchronization process is already in progress. But due to the high volume of data and the tests procedures we need to perform before putting it back online, we prefer to be clear about the fact that it is probable that the ETA for murena.io 523 services (excepted email services that will be back sooner) will be counted in days, not in hours.

  4. OTA updates repair have been low prioritized since the impact for users is the lowest.

We’d like also to make it clear that there was no data leak involved in this situation.

We sincerely apologize for the inconvenience and will keep you updated about progresses and ETA during the next hours.

The Murena Team

Update Oct 7th

  • we have an ETA for the mail service @murena.io/@e.email on Thursday 10/10 morning CET.
  • we don’t have yet an ETA for murena.io 523 other services, as we are still evaluating a few different options to make it up again safely in the most reasonable amount of time

Update Oct 9th

  • ETA for mail service @murena.io/@e.email: Thursday Oct 10th, afternoon CEST
  • ETA for drive at murena.io 523: (hopefully) Monday Oct 14th
  • ETA for calendar, contacts, mail, passwords at https://murena.io 202 : Friday Oct 11th
  • ETA for /e/OS images & OTA updates: start rolling on Thursday Oct 10th (not all devices will be available at the begining)

Update Oct 10th

  • ETA for mail service @murena.io/@e.email: Thursday Oct 10th, evening CEST.
  • ETA for drive at murena.io 523: end of next week.
  • ETA for calendar, contacts, mail, passwords at https://murena.io 202 : Friday Oct 11th.
  • ETA for /e/OS images & OTA updates: OTA service reopened, /e/OS downloads have started to roll out for most devices - some IPv6 access issues are currently being fixed.

Update Oct 11th

  • update about email services @murena.io/@e.email: our email servers have been reopened yesterday evening as expected to handle incoming and outgoing email queues that were pending for several days. During the night, the service has also been reopened briefly to users, but as the load was extremely high on servers, we have suspended it until incoming and outgoing queues get emptied. ETA for reopening the service to users is this morning if the load is acceptable. If the load remains too high we will have to modify an email route, which might take a few more hours.
  • update about /e/OS images & OTA updates: IPv6 access is now working.

Update2/ Oct 11th

  • ETA for mail service @murena.io/@e.email: we have started to open again email access, starting with a bit more than 50% Premium users. Note: only fetching emails is possible for now. Sending will be opened once the load is acceptable.
  • ETA for calendar, contacts, mail, passwords at https://murena.io 202 : Monday October 14th 2024 (testing/QA is in progress)
  • ETA for /e/OS images & OTA updates: OTA service reopened, as well as /e/OS downloads for most devices. Download speed should be way better than in the past.
  • ETA for drive at murena.io 523 (files/images/videos) is still uncertain as we have remaining issues to fix with the storage infrastructure. Will update early next week.

Update Oct 12th

  • Email service is now fully operational (receiving and sending) for Premium accounts. Please note that some emails that would have been received between Sunday 13 5:00 CEST and Sunday 13 19:50 won’t show up for now. Email service for free users will start opening progressively on Monday Oct 14th.

Regain your privacy! Adopt /e/OS the deGoogled mobile OS and online services 126phone

Comments

Post a Comment