A2S_INFO Responses Being Malformed Should Be Fixed

[Roy 10,832 / 0]

Hi everyone,

 

For a while now, people may have noticed servers being password-protected randomly, showing 255/255 player count, and the host names were cut off. We had this issue last year as well, but it had to do with our Sydney edge server and was corrected here.

 

I wasn't sure what was wrong this time, so I started a packet capture on our Redis server that handles distributing cached responses to all of our edge servers to send to our clients. I stopped the packet capture as soon as the issue was reported again and the packet capture file contained 2 million+ packets. I then filtered out all the packets besides the one containing the reported server's IP (in this case, was CS:GO Surf Timer #1, 92.119.148.18) and found this:

 

These were the packets being distributed from Redis to the other edge servers meaning one of our edge servers reported this response to our Redis server. It didn't take me long to figure out our Chicago edge server was the one (and only) edge server reporting these malformed responses. I then ran uname -r on all of our edge servers and every edge server besides the Chicago edge server reported Linux kernel 5.4.0+ whereas Chicago was still running on 4.18. The reason the server was running such an outdated kernel was due to this being our oldest edge server to date and still running Ubuntu 18.04. I decided to create a new edge server in Chicago, set it up, stop announcing/destroy the old server, and start announcing the new one. This one is running a similar kernel to the others and Ubuntu 20.04. Things appear to be running okay so far, but the malformed A2S_INFO responses issue should be fixed.

 

This also wouldn't have only impacted Surf Timer. This would have impacted all of our servers. Basically, when a server's A2S_INFO cache time was expired and the next A2S_INFO request went through our old Chicago edge server, there was a high chance it would have been malformed. It didn't do this for every single request, but I'd say a good chunk of them.

 

If you start seeing malformed responses again, please let me know!

[Roy 10,832 / 0]

7 minutes ago, AlejandraM said:

I stopped the packet capture as soon as the issue was reported again and the packet capture file contained 2 million+ packets. 

juice places near me
These were the packets being distributed from Redis to the other edge servers meaning one of our edge servers reported this response to our Redis server. 

 

I know this is a bot and all, but where did you get the two million+ packets from smh?

 

#DoBetter