Roy

HI GLITCH!

I've been keeping up with your Twitter regarding your Twitch channel and it seems you're making great progress

I don't think it'd work out in GMod because GMod's server browser sorts by game mode popularity and I don't believe GMod's DR game mode is popular at all sadly, but CS:GO and TF2 may have a chance. We had the #1 CS:S Deathrun server in the past, but CS:S is dead now

10 minutes ago, Ravioli Ravioli said:

C:\Users\User1>tracert goze.gflclan.com

Tracing route to goze.gflclan.com [216.52.148.47]
over a maximum of 30 hops:

  1    <1 ms    <1 ms    <1 ms  gateway.home [192.168.254.254]
  2    14 ms     9 ms    18 ms  32.214.60.2
  3     *        *        9 ms  32.223.0.194
  4    11 ms    11 ms    11 ms  32.222.231.21
  5     8 ms    11 ms     8 ms  ae4---0.car01.wlfr.ct.frontiernet.net [74.40.71.93]
  6     *        *        *     Request timed out.
  7    11 ms    14 ms    12 ms  ae0---0.scr01.sccs.nj.frontiernet.net [74.40.3.234]
  8    10 ms    12 ms    12 ms  12.127.149.21
  9    13 ms    15 ms    14 ms  12.122.154.26
 10    15 ms    15 ms    15 ms  cr1.n54ny.ip.att.net [12.122.105.29]
 11    14 ms    14 ms    11 ms  cgr1.n54ny.ip.att.net [12.122.131.85]
 12    11 ms    14 ms    12 ms  192.205.34.54
 13    13 ms    15 ms    12 ms  nyk-bb3-link.telia.net [62.115.115.0]
 14     *        *        *     Request timed out.
 15    32 ms    32 ms    32 ms  telia-2.e10.router2.chicago.nfoservers.com [64.74.97.253]
 16    32 ms    34 ms    31 ms  c-216-52-148-47.managed-ded.premium-chicago.nfoservers.com [216.52.148.47]

Trace complete.
 This is what came up

I am currently having packet loss

 

You will need to run an MTR. A trace route only shows the latency of three ICMP request/responses at each hop and doesn't include packet loss.

7 minutes ago, Kazarov29 said:

I can normally play GFL even at 20 fps which has happened a lot. For client side i have been playing just fine as my laptop can play in the server just fine even at 64 players. The top var while fluctuating doesn't hinder my play that much it's the bottom i think that makes playing some time really hard

CS:GO Zombie Escape is a pretty hectic game mode, so it's pretty hard on the machine's resources (primarily CPU). At points in CS:GO ZE where it's intense (e.g. a lot of players are inside each other shooting zombies, hold points), the server experiences bad performance (<15 server FPS at times I'm sure). Unfortunately, there's nothing you can do about this.

We will be likely upgrading the server's hardware at some point in the future though. You can read more about this in the following thread under "NFO Machine Upgrade".

Unfortunately, I don't have any ETA on this since the situation is quite complicated (CS:GO ZE is on an older box of ours under a different hosting provider, NFOServers). 

1 minute ago, Ravioli Ravioli said:

Its been random so far. Occuring in maybe 30 to 60 minute sessions but I leave the server at least a few times during. I have random times where it works normal as well.

Hm, I'd recommend running an MTR against the CS:GO ZE IP and leave it running until you notice packet loss in-game. Afterwards, you can PM me the results and I'll take a look further.

The guide I linked should help show you how to run an MTR using WinMTR.

There's a chance you're using a sub-optimal route to the CS:GO Zombie Escape server.

How often does this packet loss occur?

The next step I'd suggest is running an MTR towards the CS:GO ZE IP. I made the following thread using CS:GO ZE as an example and would suggest giving it a read.

On 10/20/2020 at 11:12 AM, Kazarov29 said:

Here you go, just taken from now:

 

The sv and var values at the bottom represent the server-side performance. The var at the top represents client-side I believe. You'll want to primary focus on fps though. You're running at 30 FPS in that screenshot, but that isn't uncommon in CS:GO Zombie Escape due to all the players, particles/effects, etc.

Do you feel your client-side performance has got worse recently? If so, what specs are you running (processor, GPU, RAM, and hard drive(s))? Unfortunately, nobody is going to get good performance at higher player counts due to Source Engine limits.

Would you be able to take a screenshot of this? These items in the net_graph typically indicate server-side performance and not client-side.

I like it so far. Also, AK-47 = the best weapon.

For those interested, I made a gameplay video here:

Though, the playback is sorta jumpy at times for some reason (only does it for this game). I'm sorry for that.

Hey everyone,

I'm writing this thread for documentation purposes and I'm a bit late to writing it (I'm sorry, I've been all over the place recently, lol).

On October 12th, we experienced two network-related issues (one somewhat related to the other).

Redis Server Outage

Around 9:42 PM CST, Vultr (one of our hosting providers) had an outage in their Atlanta location:

Our Redis server for A2S_INFO caching is hosted at this DC and I wasn't able to route to it (there seemed to be a BGP issue with one of their upstreams since I didn't make it out of my ISP when running an MTR to the server's IP). This caused A2S_INFO responses to break temporarily until Compressor stopped using the Redis server (which I believe took around 10 - 15 minutes to kick in). During this time, players wouldn't be able to see the servers in the server browser. However, since we don't have any actual POPs in Vultr's Atlanta location, everybody should have been able to route to our game servers successfully (you'd need to connect directly via console if you wanted to play, though).

This lasted around an hour.

Miami POP Outage

Our Miami POP experienced an outage at around 10:45 PM CST.

When the Redis server came back online, I restarted Compressor on all of our POP servers so they'd be able to establish a valid connection with the Redis server again for A2S_INFO caching. However, Compressor failed to restart on our Miami POP. Since our IPv4 block was still being announced via BGP (from BIRD), any clients routing through this POP would have lost access to our game servers.

Compressor failed to start because there wasn't enough memory available on the VM. Some of our POPs are still running with 1 GB of RAM and while adding SWAP has helped with this Compressor issue, sometimes our POPs fail to restart Compressor due to being out of memory still. A simple full VM restart resolves this issue and that's what I did.

This issue will be fully resolved once we finish expanding our Anycast infrastructure and all of our POPs run with more than 1 GB of RAM. With that said, I believe I can add a dependency to BIRD's systemd configuration so when Compressor's service is down, it will also take down BIRD which will stop announcing our IPv4 block via BGP. This won't fix the main issue, but at least if Compressor is found offline, BIRD will stop and clients will stop routing to the affected POP.

@Alexis This was the issue we talked about two days ago

If you have any questions, please let me know.

Thank you!

I started announcing NTT again. This is because we started seeing sub-optimal routes since we still need NTT announced to some of our Europe/Asia POPs (GTT isn't available in these specific locations). Basically, if an ISP in NA peers with NTT directly, but not GTT, it would route them to the closest POP that uses NTT which would be overseas. This is obviously a big no-no.

If we get attacked, we can withdraw NTT temporarily until the attacks stop.

Thank you.

Hey everyone,

I just wanted to share an idea for a project I'm probably going to be working on in the future.

As many of you know, Bifrost is a huge project Dreae and I are working on. This will be a firewall that supports features such as forwarding traffic and blocking specific (malicious) traffic via BGP Flowspec (so the upstreams filter the attack instead of our POPs). With that said, we will be implementing a module-like system. But what will be neat is Dreae and I also plan on making our own programming language for writing these modules. While this will be additional work, it is very good experience to obtain and it's very interesting to me. I've been reading up on how to create your own programming language and there are many open source guides out there that includes source code such as this article. I'm also thinking about buying this book. We're likely going to be making a compiled programming language since performance is obviously the most important for packet processing, etc. However, I need to confirm with Dreae. We can also make this compiled language using an existing compiled language such as C or C++ which is even better (gain more experience in C while making the new programming language for example).

Anyways, with the above said, I also want to implement a very flexible API with Bifrost. This leads to another idea for a personal project I have. Once Bifrost is finished, I'd like to buy my own personal /24 IPv4 block, setup two small VPSs (e.g. one in North America while the other in Europe), and setup my own Anycast network. The POPs will be running Bifrost and I'd like to make a website that'll act as a demo. In this demo, users will be able to securely sign up and if they reach out to me, I can give them credits that'll allow them to setup either a protected IP or a website.

For the protected IP, the user will be able to setup forwarding rules that'll utilize Bifrost's forwarding module API. This part should be more simpler since it's simply just sending POST/PUT/DELETE requests to the Bifrost API to add/remove forwarding rules and this should all be done securely (over HTTPS/SSL).

In regards to the website packages, this is where it gets fun. I'd really like to make it so when a user orders a website, it gives them basic SFTP access to the site's files. I'd also like to use Kubernetes and setup a container within a cluster for each website for security and scaling. With that said, each POP on the Anycast network would serve all static files like images, CSS, and JavaScript which will operate like a CDN. The actual server-side scripts should be deployed on one of the VPSs as well.

Again, this would all be a demo website. However, it could turn into something bigger if development goes well.

My main goal with this is to gain experience with Kubernetes along with setting up my own CDN and load balance traffic. Perhaps down the line, I can try to find out how to setup high availability (so each POP would serve the server-side scripts so if one POP goes down, the other will pick everything back up). I'd also like to setup MySQL/SQL for high availability under this network too. This also gives me the opportunity to do a lot more with BGP without fearing I could bring a production network offline (such as GFL's, since that's the only network I perform BGP changes on at the moment). I think gaining this experience and being able to put it on a resume will look pretty good too

Finally, I'd really like to get my personal website up with a blog and document ALL of this down publicly.

What do you guys think of this idea and if there is anybody interested in helping or the progress of the project, I'll be posting a lot more updates on this when we complete Bifrost.

Thank you!

Hey everyone,

This morning I stopped announcing NTT to all of our NA (North America) POPs. This is so if one of our servers get (D)DoS'd, GSK will be able to perform GTT offload which will now result in all traffic going into our NA POPs to go through GSK's network and their upstreams.

We had an attack from a couple nights ago that I was informed on where our NA POPs were forwarding over 1 gbps malicious traffic to our game server machines. This is because our NA POPs do not have my newest filters non-like our Europe/Asia POPs. This is because something needs to be implemented into the new filters that I feel isn't worth implementing at this moment (I feel I'm spending way too much time on a "temporary" solution). You can read more on this here. Our Asia/Europe POPs haven't forwarded any malicious traffic yet due to my filters which is a good sign. With this change, we should be mostly protected on the network.

There's a chance this may alter some player's routes who were routing to our NA POPs via NTT before. If you see any negative effects from this (e.g. sub-optimal routes, higher latency, etc), please reach out to me.

If you have any questions, please feel free to reply!

Thank you.

EDIT

I also want to note that only announcing to GTT as a direct peer on our NA POPs is a temporary solution. Once the network is fully expanded out regarding the infrastructure and Bifrost is completed, I will be introducing a blend of carriers/peers such as Telia (AS1299), Hurricane Electric (AS6939), and more which will result in better routing as long as we tune our BGP properly (which is the plan, of course).

Hey everyone,

I'm creating this thread for documentation purposes and this is mostly a copy and paste from the announcement I made on our Discord servers two days ago.

Recently, our newer hosting provider, GSK, started peering with Hurricane Electric. This resulted in many routes from players within our Asia region to route to our GSK POP through Hurricane Electric. This is bad because they should be routing through our POPs in Asia instead. With that said, it seems this tripped an uRPF filter on GSK's end which resulted in the player's IPs being blocked on GSK's behalf (since it's a really strange route going from Asia to GSK through Hurricane Electric). If one of our game servers got (D)DoS attacked, GSK would enable GTT offload which results in our traffic going through GSK's network for (D)DoS mitigation (for the servers that are attacked, single /32 IPs). Therefore, this would block a lot of our players who tripped the uRPF filter. Any game servers on our GSK machines would also block these players since we send traffic back to the client directly from these machines via GSK's network.

I've talked to GSK and as a quick solution, they created a BGP community that withdraws Hurricane Electric from our routes which I've applied two days ago. This will allow the affected players to route to our servers again assuming GTT offload isn't enabled and the game server isn't on one of our GSK machines. We've unblocked the player IPs who were blocked that we were aware of. We're working with GSK and looking into a better long-term solution for this issue.

If you experience any issues connecting to our game servers, please reach out to @Liloz01, @Aurora, or I since we all have a group DM regarding this with GSK.

I do apologize for the inconvenience and if you have any questions, please let me know.

Thank you for your time.

Nintendo 64  And then probably Playstation 2 and the original Xbox.

Welcome back!

Is this only impacting CS:GO (e.g. are there any other games you can play online, preferably Source Engine games)? With that said, do you have Windows Firewall enabled or third-party anti-virus software installed/running?

I wanna say programming/networking, but I don't know if I'm "talented" at it.

If you have Netflix and haven't watched it yet, I'd recommend watching The Punisher. I really enjoyed it

Already posted this in the Staff Discord server, but figured I'd note them down here for documentation purposes.

Here are my suggestions so far:

• On the F4 page, under the "Info" tab, the rules button redirects to a Google Doc instead of the MOTD.
• When linking a thread to the forums, you should link the specific post itself and not just the thread. This makes it so it starts at the post itself instead of seeing the website header, navbar, etc. For example, instead of https://gflclan.com/forums/topic/63631-darkrp-motd/ do https://gflclan.com/forums/topic/63631-darkrp-motd/?tab=comments#comment-307861.

Once we have the Intel i9-10900K machine which should be available sometime next week, I'd like to move the server to the new machine after we confirm via stress test that the CPU can clock to 4.8 GHz on all cores via turbo boost. This'll ensure the server is running on the best hardware we have available.

With that said, I'll be spawning a bunch of bots onto the server to make sure there isn't anything immediately noticeable regarding performance. Obviously, this is unrealistic since they're bots, but making sure we get good performance with just bots is a good first step in my opinion.

I'm not sure what my first game was, but it was probably something on the Nintendo 64 back in the early 2000s or late 90s (my brother used to bring that over when he'd visit). I will have to say the most memorable game would be Halo from 2004 though. I also remember playing a lot of James Bond 007: Agent Under Fire on my older brother's PS2 at the time:

Fun times! I miss them  

I'd have to say COVID-19, but not because it was resulting in me staying indoors (I'm at home pretty much all day anyways since I work remote and do programming/networking outside of work). I'd just have to say the job situation is what I'm impacted by the most, it just sucks trying to find a job right now

The Office and The Punisher. I love them both  

I'd have to say The Punisher is my favorite though. But sadly it was cancelled after two seasons