MagiStream

Rosehill wrote:As Myrin said, we don't really know for certain what's causing the problem.

The issue surfaced out of the blue, keeps happening erratically, and then vanishes for periods of time with no clear indication as to why or how.

We've checked any code changes made near the issue's bigger appearances, tried disabling some of them, then re-enabling them without it having much of an effect. We've troubleshooted the issue from multiple angles, tried possible solutions both on the Magistream user interface side and on the server side. There are people who have reported similar kinds of problems with almost identical symptoms, but the solutions that have worked for them haven't worked for us in the long term. All the fixes that we've tried seem to work for a while, but then the issue starts to appear again, and it doesn't leave any clear traces behind, which makes trying to solve it even harder.

At this point my best guess as to why the errors keep happening is the version of the software that's running behind the scenes, that for some reason has started to slow down. Magistream has been alive for 13 years, and while updates have been made to MS's code base, nowadays web technologies go forward in such a fast pace, that at this point we are a bit outdated and there are certain code libraries that are in need of updating. To get to the most up to date versions of all the things that are running behind the scenes requires most of all time. This is something that I do want to do at some point, but the shift will be a big one, and it will need proper preparations, backups, and probably a couple days of down-time for Magistream. This will not be happening that soon, since as stated, it requires a lot of work and preparation time before it can be done.

Another possibility is that there may be a memory leak somewhere in the code, and I've been going through Magistream's code base to see if there is one and to fix it. So far, I haven't found one. I haven't been through every single line of code yet, though, as there's just so much of it. The issue seems to mostly happen during peak hours in the US, so it can be related to the load on the site, even though that shouldn't be happening, as the server is hosted on such a service that should take care of things like that.

I'm sorry for the radio silence, and that we still can't offer any definite answers or solutions to the problem. We keep trying to solve them as best as we can, and have not given up on the site. I will aim to keep you all better posted on the progress (or the lack of...) how the site troubleshooting is going.

Eschaton wrote:Welp I see that nothing has changed, still. It's been almost 10 months now. This is beyond absurd.

Rosehill wrote:Here's where we're at, at the moment:

Due to the intermittent and elusive nature of the lag, we still haven't been able to pinpoint what's causing the issue. It may be Cloudflare, it may be the database, it may be the server, it may be some of the software running on the server, it may be a combination of any of these. I've scoured the internet for all and every similar occurrence others are having and tried fixes that have worked for others. Yet they don't work for us.

At this point it's pretty obvious that more troubleshooting isn't really going to give more information, so we've been exploring and throwing around different options on how to proceed. Those ideas have included e.g. changing servers, or updating the majority of MS codebase, or updating some parts of the MS codebase, or changing some of the software running on the server, or doing something to cloudflare, among other things. As many of these would require a bit bigger overhaul, we're still weighing in all the options before deciding what course to take next. Some things have already been decided, but we still have some work to do in order to be able to proceed.

Our team is painfully aware of the closing holiday season, and hopefully we'll have at least some more information about our next steps very soon, so that we'll have more to tell you all, but for now I'm afraid it's still the same old "it's broken, we don't know why, so we don't specifically how to fix it".