Another update on ongoing Latency Issues

    This site uses cookies. By continuing to browse this site, you are agreeing to our Cookie Policy.

    • Is this good news? You've been working with your provider all this time, but neither of you realized you weren't under ddos attack? And after figuring it out, your best solution is to scrap everything and go someplace else? Did you not investigate your partners first? This isn't some bbs we donate to, who the hell is leading over there and why are they so unfamiliar with the internet and games and games on the internet? All subs should be gifted free premium until a solution is found and acted upon, and all premium spent during this time should be refunded, just as a start to correcting this.

      The post was edited 1 time, last by Aesop ().

    • Shevarage wrote:

      Move the damn servers to the moon. This way everyone's gonna be on the same foot. :)
      Although it sounds like a fair solution, distance to the Moon as it closest distance is 363105 km (~225623 miles) and 405696,3 km (~252088) as its farthest point.

      Considering our current technology, and speed of light sitting at 300000 km/s (~186000 miles/s) would suggest our latency would be at best 2420ms and as its worst 2700ms, certainly an issue, unless we move the Moon closer..., which might prove to require a considerable effort and have it crashing on our planet in the process.

      Think will leave with servers on any location within this rock we're riding now.
    • AcOrP wrote:

      Yea there is no freakin reason to not have the servers in frankfurt.
      It will give even ping to to both US and Russian players around 100.
      Right now we have shitty US datacenters and high ping for 90% of the population.
      What about south america and north america?
      There should be multiple servers, one in each continent, as most games have.
      There has been a lot of marketing going on about this game and a lot of new people coming and enjoying it, but after playing a few minutes with a huge ping everyone just leaves.
    • Aesop wrote:

      Is this good news? You've been working with your provider all this time, but neither of you realized you weren't under ddos attack? And after figuring it out, your best solution is to scrap everything and go someplace else? Did you not investigate your partners first? This isn't some bbs we donate to, who the hell is leading over there and why are they so unfamiliar with the internet and games and games on the internet? All subs should be gifted free premium until a solution is found and acted upon, and all premium spent during this time should be refunded, just as a start to correcting this.
      Maybe we did not explain this well.

      Of course, everybody has been aware that DDOS was happening.

      In simple terms, our servers sit inside a huge data center run by a universally known global tech company. Normally, when a game is being DDOSed, the attack is target at that game's servers. In our case, these are properly protected, we learned our lessons here from the past.

      However, the attacks in the past 9 days have been on the datacenter provider directly.

      We assumed that - due to our provider being one of the largest, most reputable tech companies in the world - would be adequately protected against this. That, however, was not the case. While they do have defenses in place, they caused the massive lags and disconnects. This would affect various customers in that datacenter, in our case, it's especially bad as we run an always online game as opposed to, say, just a website.

      We had assumed that the data center attack was indirectly related to us as there had been an attack on us directly 1 day prior - i.e. that an attacker decided to go for the data center itself after noticing that our own defenses were working as well as they did. However, at the same time, we did not receive any sort of extortion attempts.

      Very recently, we have been told by the data center provider that we are not the intended target - another company in the same data center is, and we were getting affected alongside all other customers in that data center.
    • Aesop wrote:

      It still sounds like things you should've known on day 1 instead of day 9.
      No it sounds like the hosting provider is a real shitter in terms of customer service, if they've withheld this information from SBI for such a long time. If they're a reputable hoster as mentioned, they've been well aware from the get go reg. circumstances or they're simply incompetent in regards to infosec, which is scary if they're a major player, but not surprising really. Literally have major banks that get caught with open networks and misconfigured networks every now and then, so hardly suprising to find that with a hoster.
    • Aesop wrote:

      It still sounds like things you should've known on day 1 instead of day 9.

      Artur White wrote:

      what you have done is a temporary solution and the attackers will soon find out again the address of the new server, and again there will be a DDOS. Move the game server to a normal data center in Europe, and not these half measures
      "The reason why we didn't follow up on requests from the press—and there were a lot—is because communication centering around DDoS attacks is a very sensitive matter and, for the most part, counter-productive.", from OVH one of the major (if not the top one) CSP providers in Europe, when they were receiving one of the biggest DDoS attacks ever recorded (1 Tbps)

      Don't rush to conclusions without the right information.

      Keep an eye on the news and you'll find this attack on SoftLayer datacenter might be reported at some point.
    • Eltharyon wrote:

      Hello everyone,

      It's time for another update regarding the ongoing latency issues we’re experiencing.

      Since Monday, we’ve finally made some progress on finding out what is behind these disruptions.
      It appears that we are not actually the target of the attacks, but instead the attacks are directed against our datacenter in general and we are “just” collateral damage. This explains why we’ve not received any threats or extortion demands. Instead, we seem to be caught in a larger crossfire that our partners are apparently unable to deal with.

      We’re extremely unhappy with the situation and have communicated our dismay, but unfortunately as long as Albion’s servers reside in this datacenter there is very little we can do ourselves.
      Therefore we’ve initiated a move of Albion’s servers to a different datacenter, but this requires re-configuring of a very large amount of individual machines, routers, and firewalls, as well as a migration of databases etc. We expect to be able to complete this move in a week at the earliest.

      We’re not giving up on a faster solution however, and are currently trying to work around the issues with a fancy networking trick*. If successful, this may partially resolve the situation much sooner.
      We will keep you posted as usual.

      Our sincere apologies again for this dire state, we’ll announce appropriate compensation once the issues have been completely dealt with.
      Thanks for bearing with us and stay safe out there!

      Robin ‘Eltharyon’ Henkys
      Game Director

      * I can’t disclose the nature of this trick for security reasons, but if successful we should have a much better experience until we switch datacenters.
      I'm currently leading a migration to a new datacenter at work and I must say, your plan of doing so in a week has some bravado (we've been at it for a while). If/when you pull it off, please consider me impressed and enthused. ;) 8)

      Good luck and keep at it!

      Devyn
    • Korn wrote:

      Aesop wrote:

      Is this good news? You've been working with your provider all this time, but neither of you realized you weren't under ddos attack? And after figuring it out, your best solution is to scrap everything and go someplace else? Did you not investigate your partners first? This isn't some bbs we donate to, who the hell is leading over there and why are they so unfamiliar with the internet and games and games on the internet? All subs should be gifted free premium until a solution is found and acted upon, and all premium spent during this time should be refunded, just as a start to correcting this.
      Maybe we did not explain this well.
      Of course, everybody has been aware that DDOS was happening.

      In simple terms, our servers sit inside a huge data center run by a universally known global tech company. Normally, when a game is being DDOSed, the attack is target at that game's servers. In our case, these are properly protected, we learned our lessons here from the past.

      However, the attacks in the past 9 days have been on the datacenter provider directly.

      We assumed that - due to our provider being one of the largest, most reputable tech companies in the world - would be adequately protected against this. That, however, was not the case. While they do have defenses in place, they caused the massive lags and disconnects. This would affect various customers in that datacenter, in our case, it's especially bad as we run an always online game as opposed to, say, just a website.

      We had assumed that the data center attack was indirectly related to us as there had been an attack on us directly 1 day prior - i.e. that an attacker decided to go for the data center itself after noticing that our own defenses were working as well as they did. However, at the same time, we did not receive any sort of extortion attempts.

      Very recently, we have been told by the data center provider that we are not the intended target - another company in the same data center is, and we were getting affected alongside all other customers in that data center.
      You're handling these troglodytic responses quite well clap . I'm going to take a lesson out of your playbook the next time I need to be courteous to some entitled snot (not often).
    • Switching host its already a good step guys, calm down.
      Just because a host its the biggest doesn't mean they are the best for online game as host too.

      And even if we have 1 offline day to that accomplish all that move and re-setup everything,
      I'd happily wait if the things really work out.
      Because its sick seeing a ping of 147ms, but its truelly working around 350-500ms plus lots of de-sync and dc.

      And its enought time to everyone get out of your room, and get a tan LOL

      Thanks per transparency anyways.
    • MegaEdu13 wrote:

      Artur White wrote:

      what you have done is a temporary solution and the attackers will soon find out again the address of the new server, and again there will be a DDOS. Move the game server to a normal data center in Europe, and not these half measures
      You got the ideia when he said that they weren't the aim, right?

      Vadokishi wrote:

      Aesop wrote:

      It still sounds like things you should've known on day 1 instead of day 9.

      Artur White wrote:

      what you have done is a temporary solution and the attackers will soon find out again the address of the new server, and again there will be a DDOS. Move the game server to a normal data center in Europe, and not these half measures
      "The reason why we didn't follow up on requests from the press—and there were a lot—is because communication centering around DDoS attacks is a very sensitive matter and, for the most part, counter-productive.", from OVH one of the major (if not the top one) CSP providers in Europe, when they were receiving one of the biggest DDoS attacks ever recorded (1 Tbps)
      Don't rush to conclusions without the right information.

      Keep an eye on the news and you'll find this attack on SoftLayer datacenter might be reported at some point.
      but in the end we still have a very high ping for a European player, a very high ping. 150+ is a non-playable ping for a game where skills are instantly cast. Let them show open statistics on players, who are from which countries and in what quantity, because the players are not blind and see that there are more European players than Brazilians.
    • Artur White wrote:

      but in the end we still have a very high ping for a European player, a very high ping. 150+ is a non-playable ping for a game where skills are instantly cast. Let them show open statistics on players, who are from which countries and in what quantity, because the players are not blind and see that there are more European players than Brazilians.
      We have 150+ ping here in Brazil too, mate. Only NA have 50- ping. What makes you think we are different?

      This is a global server, someone will be far from the host. Let's face it.
    • Artur White wrote:

      but in the end we still have a very high ping for a European player, a very high ping. 150+ is a non-playable ping for a game where skills are instantly cast. Let them show open statistics on players, who are from which countries and in what quantity, because the players are not blind and see that there are more European players than Brazilians.
      This is a different issue not related to external factors (DDoS at hosting provider) affecting game performance.

      They have tried running separate servers in the beginning and dropped the idea based to sync issues and latency.
      Having said that, that was back when they would only dream of having so many players spread across the world, it could be think off as League of Legends having one server for the all American continent prior having it split between different sub-regions.

      To reduce ping servers have to be moved closer to player communities, that's right, but you want to be careful about how it's implemented.
      It's not has simple as counting players on each region, due at the end of the day, when splitting you end with two smaller player bases than the one you had before.

      Also there are business considerations, as of now everything cost the same for everyone.
      If servers were moved to other locations, those might have cheaper or more expensive fees, would those differences be transferred to players in the form of gold/premium prices? At what point it stops being fair for everyone involved?

      As stated before, if they decide to split worlds/servers/shards whatever, they really need to think the best approach.
      Check this graphic (newzoo.com/wp-content/uploads/…arket_per_Region_2018.png) sourced from this article newzoo.com/insights/articles/g…8-mobile-games-take-half/; it helps on understanding the sort of analysis SBI is probably doing when thinking where to invest the most.
    • MegaEdu13 wrote:

      Artur White wrote:

      but in the end we still have a very high ping for a European player, a very high ping. 150+ is a non-playable ping for a game where skills are instantly cast. Let them show open statistics on players, who are from which countries and in what quantity, because the players are not blind and see that there are more European players than Brazilians.
      We have 150+ ping here in Brazil too, mate. Only NA have 50- ping. What makes you think we are different?
      This is a global server, someone will be far from the host. Let's face it.
      Only people in the East Coast have closer to 50ms, while the West Coast reports being closer to 100ms at times.

      There is really no way around the distances to cover, even within the same country.

      There was presentation last year where it was mentioned "One world (as EVE Online)" which clearly shows it's rather important for them from a core design point of view.