I’d like to give you all an update on the current instability issues we’ve been experiencing and provide you with more insight into the things we have looked into, and the steps we have taken to try to mitigate these ongoing issues.
One of the first things I’d like to address is the frequency of our communication during this time. A few years ago, a former CEO here at Nexon America compared our work to “that of working in an emergency room. Everything is happening live and everything has to go right (which is not always the case).” These words rang true for us these past few days more than they ever have before. Communication to our players is very important to us, but equally important is making sure our internal teams are focused, have all the information they need, and are creating actionable plans for how we can address these issues. Regarding player communication, we often make minute-to-minute decisions regarding “do we have any new information to share now?”, “is this information helpful for the player?”, or “maybe we’ll find a way to fix it soon and can announce that instead?”. Often times we may be more silent than we intend to be, so we understand how that can be misinterpreted as us not caring about our community. This is one area of our work we will continue to improve.
As many of you all know, starting around September 2nd we started noticing some odd behavior with our game servers in terms of certain channels crashing or being unresponsive. This resulted in players experiencing login and latency issues and disconnections when trying to play MapleStory. On September 4th we were able to identify a packet hack that players were using to intentionally cause instability in the servers. This vulnerability was blocked, however we noticed that players were still experiencing disconnection and latency.
As more of our teams around the world worked around the clock to investigate this issue, we attempted several different measures to mitigate the problems that players were experiencing. These included changes to lower the load being placed on both our game databases and network controllers. We also needed to temporarily revert some of the player requested features that we added in our v188 Tune Up update to help identify the possible root cause of the issues. Due to the unique combination of our live server structure, public network environment, and large amount of players, it is - at times - difficult to fully predict how these changes may or may not affect the situation. We understand that it is extremely frustrating to see multiple maintenances and still experience the same instability issues, but these changes allow our teams to rule out possible causes and narrow our focus even further to get closer to a final resolution.
For now, we were able to make some additional network changes that seem to have alleviated these issues for the time being. We are continuing to monitor the situation and are investigating further so that we can be confident we find a permanent resolution to the recent instability issues.
I’d like to close this message by saying we appreciate the community’s patience and support while we continue to work through difficult situations such as these. We are fully aware of all the feedback you have provided during this time, and it drives our passion to work even harder to resolve these issues. I speak for all of our teams when I say, we sincerely apologize for any inconvenience these issues may have caused you and hope to be able to resolve them once and for all as quickly as possible.
Due to the continued instability, we’ll be offering compensation for the gameplay time lost and inability to access certain events. More details for that will be coming soon. We hope to receive your continued support as we continue to resolve these issues and take measures to prevent them from occurring again in the future.
Dennis "Savage Ace" Bernardo
Producer - MapleStory