fistful of frags The FoF servers could be cancelled, what's next?

[craZy_y0 13 / 1,175]

Hey, I'm not sure if any of you (particularly those responsible for the Fistful of Frags servers) keep tabs on the Steam discussion forums for FoF. I usually check every couple times a day but looking at this thread, the servers are now at risk of being taken off the server browser by the developer, Rebel_Y. This is due to the fact that the servers are displaying an artificial ping (obviously lower) than what the server's ping actually is.

 

I'm not too sure what is next for GFLClan on Fistful of Frags; is there any confirmation that Rebel has reached out to anyone to get this resolved; will you guys try to keep the servers but display the server's real ping on the browser; or will the servers just go bust, end of?

 

Personally I hope that the servers stay but display the proper ping than a lower one just to lure people on into thinking they will have a smooth experience. I've also had a lot of fun on those servers compared to others but personally I'm with the whole thing being a ping fraud and hopefully that will be fixed.

 

My only fear if the GFLClan servers are to be shut down is that there is also another problematic server known as fastpath.fr which seems to only be populated when the GFLClan servers are not up.

Why is it problematic you may ask? Well, they block players who have any form of Steam ban on record whether it is a VAC, game, trade or even a community ban from connecting to their servers. Without a doubt that will likely affect at least 25% of the playerbase, including myself (with a CS:GO VAC and game ban, from phone sharing). I have attempted to appeal this ban on their Steam group to no avail, they simply told me to read this thread and basically you have to make a donation on their website to show your interest "in playing on their servers", like fuck off am I going to bribe them to unban me - such a good way to kill off the small playerbase this game has by the way.

 

All in all, this was just a message to let the admins or anyone else reading know why the servers got taken down suddenly if this isn't addressed by the time that happens. Apologies if the fastpath thing sounds comes off like I was angry, to be fair that winds me up on it's own because I know it will kill of the playerbase if GFLClan's servers going down doesn't. Thanks for reading, I really hope a solution for this whole thing does get sorted, it will be a real shame to see this server go otherwise...

[Liloz01 1,256 / 19,820]

Thanks for informing us, it would be sad for GFL to lose its FoF division; while the users who play on our servers are not very active in the overall community of GFL, it still thrives in it's own way and makes people happy. I hope we don't lose this.

 

I've let @Roy know about the steam thread you linked, so he could possibly reach out to Rebel_Y. The thread will probably make another vein pop in his forehead because of what "Apologet" claimed, but that doesn't really matter as what Rebel_Y takes issue with is the fact the ping is misleading, no matter if it provides protection or not.

 

It is true that the ping is misleading; what is not true is that it doesn't provide any protection whatsoever. You can read about our network by looking at the posts in this forum, if you'd like: https://gflclan.com/forums/forum/959-gfls-network/ - specifically this post if you'd like to hear @Roys justification for implementing anycast with A2S_INFO caching as we have: https://gflclan.com/forums/topic/59681-anycast-expansion-plansapproaches/.

 

I don't see a future where we turn off our A2S_INFO caching, on all servers, just because our FoF servers become blacklisted. I'm no network administrator so I'm not sure if it can be turned off on a per-game or per-server basis feasibly. So I hope to see a future where Rebel_Y comes to understand the necessity of our custom anycast implementation  - it's what allows our community and servers to function. If the servers are blacklisted, you can still connect to them - they just won't show up in the server browser. So it's possible that we won't take them down, even if removed from the browser; it depends how many people will take the route of connecting via console just to get to us. No point in keeping the server up if no one plays on it.

 

Again, thanks. We'll keep you informed of what is happening with our servers as we figure it out.

[Roy 10,832 / 0]

I plan on looking more into this later today (currently busy with work). As @Liloz01 mentioned, a lot of the thoughts behind A2S_INFO caching were mentioned in my Anycast expansions thread as a post:

 

 

I'd especially suggest reading my Google Doc as well here which goes over my thought process with it all. I'm not expecting people to agree with them and I understand why they wouldn't. Though, this was the decision I made after deploying the network in March of 2019.

 

I don't blame the person who made that Steam thread as well. Their points are valid in terms of the inaccurate pings. However, caching the A2S_INFO packet does actually provide a benefit in terms of (D)DoS protection. The reason attackers are targeting this query to begin with is because it allows them to send A2S_INFO requests which are only 25 bytes in payload size. When the server responds (without caching the packet), it has to first gather all the information that the A2S_INFO response contains (e.g. mod directory, host name, player count, map name, etc) and send it back to the source which is usually always larger than the request size. The combination of getting all of the information when each request comes in and also sending back a response each time results in it being easier to peg the server's CPU resources. Given SRCDS is also mostly single-threaded, it really isn't that difficult to peg the server with these attacks from what I've seen (I've easily done this with my packet flooding tool I made here that supports specifying payloads when pen-testing internally). With that said, the attacker could technically use any of the other queries Valve provides to their advantage, but the reason people choose the A2S_INFO query specifically is if you don't have any caching mechanism, it puts the servers into a lose-lose situation. Valve DOES have commands to limit the in-game effects from the A2S_INFO attacks which are the following:

 

sv_max_queries_sec
sv_max_queries_window
sv_max_queries_sec_global

 

The problem is, typically, the global limit will be reached and this results in your server being thrown off of the server browser from the attack. Therefore, in situations where you're getting attacked via an A2S_INFO spam attack, your server will either become pegged and probably unplayable in-game or you set the above limits and your server doesn't come up on the server browser. This is why I say it's a lose-lose situation for the server owners/community.

 

Now, you COULD also install extensions from let's say SourceMod/MetaMod (for most Source Engine games) or lua/C++ (for GMod typically) that caches the query on the game server level. Technically, this is a pretty good defense. However, firstly, it's hard to find an extension that actually works, especially for SourceMod/MetaMod (the three I've found I couldn't get to work on my test CS:S Linux server). If I spent more time trying to get it work, I probably could, but I'm assuming some things have to be rewritten and I don't have much experience with making C++ extensions for Source Engine games (using SourceMod or MetaMod as the addons). However, if you were to get this working, it'd still not be as good as caching the packet on each POP in terms of protection. This is because the game server would still be replying to each request with a cached response. Given a point for making the Anycast network was to allow us to find hosting providers with less (D)DoS protection for our game server machines (since we'd be relying on the Anycast network for protection which is the whole point), this would allow attackers to potentially still use the A2S_INFO vector to their advantage because they'd be able to send so many request that it'd over flood the game server machine itself (that doesn't have any real (D)DoS protection and just an one gigabit NIC/link). This is why caching it on each POP would be better since it'd distribute the load and each POP would respond to the queries instead of just the single game server machine.

 

Anyways, given our FoF servers are less likely to see these attacks, we can disable caching for these servers specifically if Rebel wants us to do so or w/e. I also wouldn't mind if they reached out to me and we talked about this more in PM. I may reach out to them once time frees up.

 

I hope the above helps some understand. Like I said, all my views are within that Google Doc. Some points I can definitely see people disagreeing with since it isn't all about protection and I understand them completely as mentioned above. However, given the combination of the points I listed along with the concerns, this is why I decided we'd be caching the A2S_INFO query on all POPs.

 

Thank you.

[annoying furry 6,300 / 63,950]

I have removed the mitigation that results in the artificially lowered pings for just those two servers, so we shouldn't have any issues with getting blacklisted over these next few days. I believe we're still trying to reach out to rebel so that we may resolve this in a way that doesn't leave us open to such attacks, but so far we haven't been able to contact him.

[craZy_y0 13 / 1,175]

17 hours ago, Xy said:

I have removed the mitigation that results in the artificially lowered pings for just those two servers, so we shouldn't have any issues with getting blacklisted over these next few days. I believe we're still trying to reach out to rebel so that we may resolve this in a way that doesn't leave us open to such attacks, but so far we haven't been able to contact him.

 

I can confirm the ping in the server browser has not changed as of this post, as mentioned on the steam discussion thread. I'm not too sure if this is either something that will change in time or if you have forgotten to do something, hopefully it will be fixed soon. Also Rebel said he has no time to discuss which is a real shame but I can only assume that FoF isn't everyone's main focus, nothing I can do about it either unfortunately but we all know that this has been a thing for well over a year now - I never really paid attention to the ping when I played on the GFLClan servers in Garry's Mod before then but I wasn't too bothered about it personally.

 

Personally I have no problem with GFLClan and any problems I did have in the past have all been resolved civilly. The only problem is that I have suspicions on particular admins abusing their power, I believe that CedarLUG who was a moderator on the steam discussion forum for FoF banned Apologet from the discussions for a month (this ban was later lifted by Rebel within a couple of hours) because he was causing "beef" between him and the community, as shown in screenshot 1 and screenshot 2. As far as I'm aware, this is out of your guy's control but as said previously, Rebel lifted the ban within a couple of hours. CedarLUG has also globally banned Apologet from the servers for two months over him making the thread itself because CedarLUG claimed that Apologet was trolling when he actually spoke out over what others have also brought up in the past but unsuccessfully got a response from Rebel_Y himself. I personally think that this is unfair conduct from CedarLUG and I would hate for him to continue doing this to others all because he doesn't like being given criticism about how your server network is run. Look at what Gramps has done in the past, I admit that he is quite a fair person when it comes to moderating the servers but he has slipped up a few times and falsely banned people based on the assumptions that they were either cheating or abusing exploits, a few notable names in the Fistful of Frags community such as Rubbery Lee and Chitch.

 

Regarding any other things that Apologet has done behind the scenes, he has only talked to me privately on Steam about the drama between him and CedarLUG but also the discussion he had with Liloz01 however he did not leak any chat logs about Liloz to me nor would I want to reveal them here out of respect for both parties' privacy.

 

Just to clarify as well, I don't have a problem with CedarLUG either and I'm not even sure if this is the right place to bring that up but it should be brought to light regarding how this situation has unfolded. Hopefully that situation gets addressed because I don't think that behaviour is acceptable from an admin representing your community. I will also reiterate that I don't have a problem with Gramps, I haven't talked to him in ages, I saw him on FoF a few weeks ago but I haven't talked to him one-to-one because my old account on discord was disabled (I have not rejoined the GFL server on my new account, as such I don't have any contact with him on discord). He was also very helpful when it came to getting my past permanent ban on GFL appealed and eased down a one year ban and I'm still thankful that I was successfully appealed - since then I have had a clean record on the servers.

 

I am aware that I haven't replied to this thread since I created it but nonetheless, I appreciate the positive responses on this thread nonetheless. I can tell this is very stressful for you guys because you care about the community on top of the integrity of fair play on your servers along with any other added controversy along the way, hope you guys can keep at it and we will one day get to see the light on the other side!

 

I also took a screenshot as shown in the attachment prior to writing up this post and this demonstrates that the ping has not changed in the browser since Xy's reply.

[annoying furry 6,300 / 63,950]

1 hour ago, craZy_y0 said:

I can confirm the ping in the server browser has not changed as of this post, as mentioned on the steam discussion thread. I'm not too sure if this is either something that will change in time or if you have forgotten to do something, hopefully it will be fixed soon.

That's odd. I did push a new configuration to the POPs, so it shouldn't be doing it anymore. I'll look into this.

 

I removed it from the wrong server. I hit 92.119.148.60 when the server's IP is actually 92.119.148.6. It should be fixed now. Sorry for any confusion.