[C] Unofficial Code To Capture All Packets And Forward Via IPIP. Help Needed?

~~Roy~~ · March 12, 2020

Hey everyone,

I am releasing some code of a test program I am making along with asking for some help from experienced C/networking programmers (if any). The program does the following:

Captures all packets coming into a specific interface (specified in the command line).
Forwards all TCP, UDP, and ICMP packets to a destination (specified in the command line) via IPIP.
Since we forward the traffic using IPIP packets, this gives us the ability to perform NAT on the destination machine. The NAT IP (the IPIP tunnel's internal IP) is specified in the command line.
When the destination machine sends packets back (via IPIP), the program removes the outer IP header, changes the inner header's source address to the server's IP, and sends it back to the client (this completes the forwarding).
Supports multiple threads via pthreads and uses epoll() on socket descriptors. Thread count is specified in command line.

Please note that this code is sloppy and not documented very well. With that said, the program isn't complete. For example, TCP packets aren't working properly. I'm unsure what the issue is, but I figured I'd release the code and maybe an experienced C/networking programmer in GFL may know the issue.

The issue with TCP packets is for some reason, it's sending the RST flag (reset) each time I attempt to forward the traffic. At first, I was suspecting the three-way handshake wasn't finishing. However, data is still sent and received by the destination application most of the time, which is weird. If the three-way handshake wasn't completing, I doubt any data would be sent and received by the destination application.

For example, I have a CS:S server running on the IPIP tunnel (endpoint). When I attempt to submit RCON commands, I get this message:

] rcon status
Lost RCON connection, please retry command.
Lost RCON connection, please retry command (WSAENOTSOCK)

Here are the packets via tcpdump -i any ... -nne -XX -vv:

1:49:23.187921 Out xx:xx:xx:xx:xx:xx ethertype IPv4 (0x0800), length 68: (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 52)
    10.50.0.3.27015 > xxx.xxx.xxx.xxx.53026: Flags [S.], cksum 0xcbf3 (correct), seq 2751408096, ack 2928991913, win 64800, options [mss 1440,nop,nop,sackOK,nop,wscale 7], length 0
        0x0000:  0004 0001 0006 1ac4 df70 d8a6 0000 0800  .........p......
        0x0010:  4500 0034 0000 4000 4006 268c 0a32 0003  E..4..@.@.&..2..
        0x0020:  0a01 0003 6987 cf22 a3ff 27e0 ae94 dea9  ....i.."..'.....
        0x0030:  8012 fd20 cbf3 0000 0204 05a0 0101 0402  ................
        0x0040:  0103 0307                                ....
21:49:23.189173 Out xx:xx:xx:xx:xx:xx ethertype IPv4 (0x0800), length 56: (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 40)
    10.50.0.3.27015 > xxx.xxx.xxx.xxx.53026: Flags [R], cksum 0x971d (correct), seq 2751408097, win 0, length 0
        0x0000:  0004 0001 0006 1ac4 df70 d8a6 0000 0800  .........p......
        0x0010:  4500 0028 0000 4000 4006 2698 0a32 0003  E..(..@.@.&..2..
        0x0020:  0a01 0003 6987 cf22 a3ff 27e1 0000 0000  ....i.."..'.....
        0x0030:  5004 0000 971d 0000                      P.......
21:49:23.189200 Out xx:xx:xx:xx:xx:xx ethertype IPv4 (0x0800), length 56: (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 40)
    10.50.0.3.27015 > xxx.xxx.xxx.xxx.53026: Flags [R], cksum 0x971d (correct), seq 2751408097, win 0, length 0
        0x0000:  0004 0001 0006 1ac4 df70 d8a6 0000 0800  .........p......
        0x0010:  4500 0028 0000 4000 4006 2698 0a32 0003  E..(..@.@.&..2..
        0x0020:  0a01 0003 6987 cf22 a3ff 27e1 0000 0000  ....i.."..'.....
        0x0030:  5004 0000 971d 0000                      P.......
21:49:23.189785 Out xx:xx:xx:xx:xx:xx ethertype IPv4 (0x0800), length 56: (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 40)
    10.50.0.3.27015 > xxx.xxx.xxx.xxx.53026: Flags [R], cksum 0x971d (correct), seq 2751408097, win 0, length 0
        0x0000:  0004 0001 0006 1ac4 df70 d8a6 0000 0800  .........p......
        0x0010:  4500 0028 0000 4000 4006 2698 0a32 0003  E..(..@.@.&..2..
        0x0020:  0a01 0003 6987 cf22 a3ff 27e1 0000 0000  ....i.."..'.....
        0x0030:  5004 0000 971d 0000                      P.......
21:49:23.189809 Out xx:xx:xx:xx:xx:xx ethertype IPv4 (0x0800), length 56: (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 40)
    10.50.0.3.27015 > xxx.xxx.xxx.xxx.53026: Flags [R], cksum 0x971d (correct), seq 2751408097, win 0, length 0
        0x0000:  0004 0001 0006 1ac4 df70 d8a6 0000 0800  .........p......
        0x0010:  4500 0028 0000 4000 4006 2698 0a32 0003  E..(..@.@.&..2..
        0x0020:  0a01 0003 6987 cf22 a3ff 27e1 0000 0000  ....i.."..'.....
        0x0030:  5004 0000 971d 0000                      P.......

This is the forwarding server sending packets back out to my client. As you can see, it's sending packets with the R flag (RST) set. This is resetting the TCP connection and I'm unsure why this is the case. With that said, the game server itself appears to get every single attempt, but it only sometimes actually registers the command I send via RCON. For example:

The "autokick is disabled" message happens each time I attempt an RCON command. However, as you can see, only 1/4 of the time does it actually send the command I submitted (in this case, status).

All checksums are calculated properly for the IP header(s) + protocol headers (TCP/UDP). I'm still having issues with the ICMP header, but I'll be able to figure that out (not a priority right now, I'm more focused on TCP).

Also, fun note, since UDP traffic works fine, I was able to connect to my CS:S server, add bots, test the program's performance by using my UDP Sender (a program that can send DoS attacks), and observe what happened. I launched the UDP Sender tool from two servers, therefore, it technically can be considered an internal (D)DoS attack, lol. It had some weird results, but I did confirm that using four threads vs one thread did make a huge performance difference.

Anyways, here's the code, finally:

/* Testing forwarding packets to destination. */
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/socket.h>
#include <sys/types.h>
#include <netinet/ip.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <linux/if_packet.h>
#include <linux/tcp.h>
#include <linux/udp.h>
#include <linux/icmp.h>
#include <net/ethernet.h>
#include <net/if.h>
#include <string.h>
#include <error.h>
#include <errno.h>
#include <inttypes.h>
#include <pthread.h>
#include <sys/epoll.h>
#include <sys/sysinfo.h>
#include <signal.h>
#include <fcntl.h>
#include <ctype.h>

#define REDIRECT_HEADER

#include "csum.h"

#define MAX_PCKT_LENGTH 65507
#define MAX_EPOLL_EVENTS 64

extern int errno;
int cont = 1;

int packetCount = 0;
int sentPacketCount = 0;

struct stuff
{
    char *lIP;
    uint16_t lPort;
    char *dIP;
    char *nIP;
    uint16_t dPort;
    int sockfd;
    int udpsockfd;
    int tcpsockfd;
    char *interface;
    struct sockaddr_in *sockin;
};

void sigHndl(int tmp)
{
    cont = 0;
}

void shiftChar(char *arr, int size, int dataLen)
{
    for (int i = (dataLen - 1); i >= 0; i--)
    {
        memmove(arr + i + size, arr + i, 1);
    }

    for (int i = 0; i < size; i++)
    {
        memcpy(arr + i, "0", 1);
    }
}

void* threadHndl(void * data)
{
    struct stuff *stuff = data;

    // Get thread ID.
    pthread_t threadID = pthread_self();

    // Headers.
    unsigned char buffer[MAX_PCKT_LENGTH];

    //struct ethhdr *ethhdr = (struct ethhdr *) buffer;
    struct iphdr *iphdr = (struct iphdr *) (buffer + sizeof(struct ethhdr));

    memset(buffer, 0, MAX_PCKT_LENGTH);

    struct sockaddr_ll din;

    socklen_t dinLen = sizeof(din);

    int8_t epoll;

    epoll = epoll_create1(0);

    if (epoll < 0)
    {
        fprintf(stderr, "EPoll() :: Error Creating - %s\n", strerror(errno));

        pthread_exit(NULL);
    }

    struct epoll_event event;
    event.events = EPOLLIN;
    event.data.fd = stuff->sockfd;

    if (epoll_ctl(epoll, EPOLL_CTL_ADD, stuff->sockfd, &event) < 0)
    {
        fprintf(stderr, "EPoll_ctl() :: Error - %s\n", strerror(errno));

        pthread_exit(NULL);
    }

    while (cont)
    {
        struct epoll_event event;

        epoll_wait(epoll, &event, 1, -1);

        if (event.data.fd == stuff->sockfd)
        {
            uint16_t received = recvfrom(event.data.fd, &buffer, MAX_PCKT_LENGTH, 0, (struct sockaddr *)&din, &dinLen);

            if (iphdr->protocol == IPPROTO_TCP)
            {
                // Handle TCP packets.
                struct tcphdr *tcphdr = (struct tcphdr *) (buffer + sizeof(struct ethhdr) + (iphdr->ihl * 4));
                unsigned char *data = (unsigned char *) (buffer + sizeof(struct ethhdr) + (iphdr->ihl * 4) + (tcphdr->doff * 4));
                uint16_t payloadLength = received - (sizeof(struct ethhdr) + (iphdr->ihl * 4) + (tcphdr->doff * 4));

                // Check if destination is listen IP and port.
                if (iphdr->daddr == inet_addr(stuff->lIP) && ntohs(tcphdr->dest) > 1024)
                {
                    unsigned char newData[MAX_PCKT_LENGTH];
                    memcpy(newData, buffer + sizeof(struct ethhdr), MAX_PCKT_LENGTH - sizeof(struct ethhdr));

                    // Add outter IP frame.
                    shiftChar(newData, sizeof(struct iphdr), received - sizeof(struct ethhdr));

                    // Initialize outter IP header and shift other headers.
                    struct iphdr *oiphdr = (struct iphdr *) (newData);
                    struct iphdr *Niphdr = (struct iphdr *) (newData +  sizeof(struct iphdr));
                    struct tcphdr *Ntcphdr = (struct tcphdr *) (newData  + sizeof(struct iphdr) + (iphdr->ihl * 4));
                    
                    oiphdr->version = 4;
                    oiphdr->ihl = 5;
                    oiphdr->protocol = IPPROTO_IPIP;
                    oiphdr->frag_off = 0;
                    oiphdr->check = 0;
                    oiphdr->ttl = 64;
                    oiphdr->tos = 0x0;
                    oiphdr->daddr = inet_addr(stuff->dIP);
                    oiphdr->saddr = inet_addr(stuff->lIP);
                    oiphdr->tot_len = ntohs(Niphdr->tot_len) + sizeof(struct iphdr);

                    oiphdr->check = ip_fast_csum(oiphdr, oiphdr->ihl);

                    uint32_t oldDaddr = Niphdr->daddr;
                    Niphdr->daddr = inet_addr(stuff->nIP);

                    Niphdr->check = csum_diff4(oldDaddr, Niphdr->daddr, Niphdr->check);
                    Ntcphdr->check = csum_diff4(oldDaddr, Niphdr->daddr, Ntcphdr->check);
        
                    struct sockaddr_in a;
                    a.sin_family = AF_INET;
                    a.sin_addr.s_addr = inet_addr(stuff->dIP);
                    memset(&a.sin_zero, 0, sizeof(a.sin_zero));

                    uint16_t sent;

                    if ((sent = sendto(stuff->tcpsockfd, newData, oiphdr->tot_len, 0, (struct sockaddr *)&a, sizeof(a))) < 0)
                    {
                        perror("sentto");
                    }

                    //fprintf(stdout, "Forwarded %d (%d) bytes to proxy.\n", sent, oiphdr->tot_len);
                }

                packetCount++;
            }
            else if (iphdr->protocol == IPPROTO_UDP)
            {
                // Handle UDP packets.
                struct udphdr *udphdr = (struct udphdr *) (buffer + sizeof(struct ethhdr) + (iphdr->ihl * 4));
                unsigned char *data = (unsigned char *) (buffer + sizeof(struct ethhdr) + (iphdr->ihl * 4) + sizeof(struct udphdr));
                uint16_t payloadLength = received - (sizeof(struct ethhdr) + (iphdr->ihl * 4) + sizeof(struct udphdr));

                // Check if destination is listen IP and port.
                if (iphdr->daddr == inet_addr(stuff->lIP) && ntohs(udphdr->dest) > 1024)
                {
                    unsigned char newData[MAX_PCKT_LENGTH];
                    memcpy(newData, buffer + sizeof(struct ethhdr), MAX_PCKT_LENGTH - sizeof(struct ethhdr));

                    // Add outter IP frame.
                    shiftChar(newData, sizeof(struct iphdr), received - sizeof(struct ethhdr));

                    // Initialize outter IP header and shift other headers.
                    struct iphdr *Niphdr = (struct iphdr *) (newData +  sizeof(struct iphdr));
                    struct udphdr *Nudphdr = (struct udphdr *) (newData  + sizeof(struct iphdr) + (iphdr->ihl * 4));

                    struct iphdr *oiphdr = (struct iphdr *) (newData);
                    
                    oiphdr->version = 4;
                    oiphdr->ihl = 5;
                    oiphdr->protocol = IPPROTO_IPIP;
                    oiphdr->frag_off = 0;
                    oiphdr->check = 0;
                    oiphdr->ttl = 64;
                    oiphdr->tos = 0x50;
                    oiphdr->daddr = inet_addr(stuff->dIP);
                    oiphdr->saddr = inet_addr(stuff->lIP);
                    oiphdr->tot_len = ntohs(Niphdr->tot_len) + sizeof(struct iphdr);

                    oiphdr->check = ip_fast_csum(oiphdr, oiphdr->ihl);

                    uint32_t oldDaddr = Niphdr->daddr;
                    Niphdr->daddr = inet_addr(stuff->nIP);

                    Niphdr->check = csum_diff4(oldDaddr, Niphdr->daddr, Niphdr->check);
                    Nudphdr->check = csum_diff4(oldDaddr, Niphdr->daddr, Nudphdr->check);

                    struct sockaddr_in a;
                    a.sin_family = AF_INET;
                    a.sin_addr.s_addr = inet_addr(stuff->dIP);
                    memset(&a.sin_zero, 0, sizeof(a.sin_zero));

                    uint16_t sent;

                    if ((sent = sendto(stuff->udpsockfd, newData, oiphdr->tot_len, 0, (struct sockaddr *)&a, sizeof(a))) < 0)
                    {
                        perror("sentto");
                    }

                    //fprintf(stdout, "Forwarded %d (%d) (%d is payload data) UDP bytes to proxy.\n", sent, oiphdr->tot_len, payloadLength);

                    if (payloadLength == 14)
                    {
                        //fprintf(stdout, "Found suspect packet. %lu is udphdr + payload length. %d is Niphdr->total_length and %d is Nudphdr->len. Also %lu. %lu is size of ethhdr\n", (sizeof(struct udphdr) + payloadLength), ntohs(Niphdr->tot_len), ntohs(Nudphdr->len), ntohs(Nudphdr->len) - sizeof(struct udphdr), sizeof(struct ethhdr));
                    }
                }

                packetCount++;
            }
            else if (iphdr->protocol == IPPROTO_ICMP)
            {
                // Handle TCP packets.
                struct icmphdr *icmphdr = (struct icmphdr *) (buffer + sizeof(struct ethhdr) + (iphdr->ihl * 4));
                unsigned char *data = (unsigned char *) (buffer + sizeof(struct ethhdr) + (iphdr->ihl * 4) + sizeof(struct icmphdr));
                uint16_t payloadLength = received - (sizeof(struct ethhdr) + (iphdr->ihl * 4) + sizeof(struct icmphdr));

                // Check if destination is listen IP and port.
                if (iphdr->daddr == inet_addr(stuff->lIP))
                {
                    unsigned char newData[MAX_PCKT_LENGTH];
                    memcpy(newData, buffer + sizeof(struct ethhdr), MAX_PCKT_LENGTH - sizeof(struct ethhdr));

                    // Add outter IP frame.
                    shiftChar(newData, sizeof(struct iphdr), received - sizeof(struct ethhdr));

                    // Initialize outter IP header and shift other headers.
                    struct iphdr *Niphdr = (struct iphdr *) (newData +  sizeof(struct iphdr));
                    struct icmphdr *Nicmphdr = (struct icmphdr *) (newData  + sizeof(struct iphdr) + (iphdr->ihl * 4));

                    struct iphdr *oiphdr = (struct iphdr *) (newData);
                    
                    oiphdr->version = 4;
                    oiphdr->ihl = 5;
                    oiphdr->protocol = IPPROTO_IPIP;
                    oiphdr->frag_off = 0;
                    oiphdr->check = 0;
                    oiphdr->ttl = 64;
                    oiphdr->tos = 0x50;
                    oiphdr->daddr = inet_addr(stuff->dIP);
                    oiphdr->saddr = inet_addr(stuff->lIP);
                    oiphdr->tot_len = ntohs(Niphdr->tot_len) + sizeof(struct iphdr);

                    oiphdr->check = ip_fast_csum(oiphdr, oiphdr->ihl);

                    uint32_t oldDaddr = Niphdr->daddr;
                    Niphdr->daddr = inet_addr(stuff->nIP);

                    Niphdr->check = csum_diff4(oldDaddr, Niphdr->daddr, Niphdr->check);

                    Nicmphdr->checksum = 0;
                    Nicmphdr->checksum = csum_tcpudp_magic(Niphdr->saddr, Niphdr->daddr, sizeof(struct icmphdr), IPPROTO_TCP, csum_partial(Nicmphdr, sizeof(struct icmphdr), 0));

                    struct sockaddr_in a;
                    a.sin_family = AF_INET;
                    a.sin_addr.s_addr = inet_addr(stuff->dIP);
                    memset(&a.sin_zero, 0, sizeof(a.sin_zero));

                    uint16_t sent;

                    if ((sent = sendto(stuff->tcpsockfd, newData, oiphdr->tot_len, 0, (struct sockaddr *)&a, sizeof(a))) < 0)
                    {
                        perror("sentto");
                    }
                    //fprintf(stdout, "Forwarded %d (%d) bytes to proxy.\n", sent, oiphdr->tot_len);
                }

                packetCount++;
            }
            else if (iphdr->protocol == IPPROTO_IPIP)
            {
                // Send the packet back after checking.
                struct iphdr *Iiphdr = (struct iphdr *) (buffer + sizeof(struct ethhdr) + (iphdr->ihl * 4));

                if (Iiphdr->protocol == IPPROTO_UDP)
                {
                    // Let's send back the UDP packet.

                    // Start new data from inner header.
                    struct udphdr *udphdr = (struct udphdr *) (buffer + sizeof(struct ethhdr) + (iphdr->ihl * 4) + (Iiphdr->ihl * 4));
                    unsigned char *newData = (unsigned char *)(buffer + sizeof(struct ethhdr) + (iphdr->ihl * 4));
                    uint16_t payloadLength = received - sizeof(struct ethhdr) - (iphdr->ihl * 4) - (Iiphdr->ihl * 4) - sizeof(struct udphdr);

                    // Fill in necessary IP header data.
                    uint32_t oldAddr = iphdr->saddr;
                    Iiphdr->saddr = iphdr->daddr;
                    Iiphdr->tot_len = (Iiphdr->ihl * 4) + sizeof(struct udphdr) + payloadLength;

                    // Recalculate checksums.
                    Iiphdr->check = ip_fast_csum(Iiphdr, Iiphdr->ihl);
                    udphdr->check = 0;
                    udphdr->check = csum_tcpudp_magic(Iiphdr->saddr, Iiphdr->daddr, sizeof(struct udphdr) + payloadLength, IPPROTO_UDP, csum_partial(udphdr, sizeof(struct udphdr) + payloadLength, 0));

                    struct sockaddr_in a;
                    a.sin_family = AF_INET;
                    a.sin_addr.s_addr = Iiphdr->daddr;
                    memset(&a.sin_zero, 0, sizeof(a.sin_zero));

                    //fprintf(stdout, "Sending %d bytes to %s:%d from %d:%d\n", Iiphdr->tot_len, inet_ntoa(a.sin_addr), ntohs(udphdr->dest), Iiphdr->saddr, ntohs(udphdr->source));

                    uint16_t sent;

                    if ((sent = sendto(stuff->udpsockfd, newData, Iiphdr->tot_len, 0, (struct sockaddr *)&a, sizeof(a))) < 0)
                    {
                        perror("sentto");
                    }
                    else
                    {
                        sentPacketCount++;
                    }
                    
                }
                else if (Iiphdr->protocol == IPPROTO_TCP)
                {
                    // Let's send back the TCP packet.

                    // Start new data from inner header.
                    struct tcphdr *tcphdr = (struct tcphdr *) (buffer + sizeof(struct ethhdr) + (iphdr->ihl * 4) + (Iiphdr->ihl * 4));
                    unsigned char *newData = (unsigned char *)(buffer + sizeof(struct ethhdr) + (iphdr->ihl * 4));
                    uint16_t payloadLength = received - sizeof(struct ethhdr) - (iphdr->ihl * 4) - (Iiphdr->ihl * 4) - (tcphdr->doff * 4);

                    // Fill in necessary IP header data.
                    uint32_t oldAddr = Iiphdr->saddr;
                    Iiphdr->saddr = iphdr->daddr;
                    Iiphdr->tot_len = (Iiphdr->ihl * 4) + (tcphdr->doff * 4) + payloadLength;

                    fprintf(stdout, "%d is the tcphdr->doff\n", tcphdr->doff);

                    // Recalculate checksums.
                    Iiphdr->check = ip_fast_csum(Iiphdr, Iiphdr->ihl);
                    tcphdr->check = csum_diff4(oldAddr, Iiphdr->saddr, tcphdr->check);

                    struct sockaddr_in a;
                    a.sin_family = AF_INET;
                    a.sin_addr.s_addr = Iiphdr->daddr;
                    memset(&a.sin_zero, 0, sizeof(a.sin_zero));

                    //fprintf(stdout, "Sending %d bytes to %s\n", Iiphdr->tot_len, inet_ntoa(a.sin_addr));

                    uint16_t sent;

                    if ((sent = sendto(stuff->tcpsockfd, newData, Iiphdr->tot_len, 0, (struct sockaddr *)&a, sizeof(a))) < 0)
                    {
                        perror("sentto");
                    }
                    else
                    {
                        sentPacketCount++;
                    }
                    
                }
                else if (Iiphdr->protocol == IPPROTO_ICMP)
                {
                    // Let's send back the ICMP packet.

                    // Start new data from inner header.
                    struct icmphdr *icmphdr = (struct icmphdr *) (buffer + sizeof(struct ethhdr) + sizeof(struct iphdr) + sizeof(struct iphdr));
                    unsigned char *newData = (unsigned char *)(buffer + sizeof(struct ethhdr) + sizeof(struct iphdr));

                    // Fill in necessary IP header data.
                    Iiphdr->saddr = iphdr->daddr;
                    Iiphdr->tot_len = (Iiphdr->ihl * 4) + sizeof(struct icmphdr);

                    // Recalculate checksums.
                    Iiphdr->check = ip_fast_csum(Iiphdr, Iiphdr->ihl);

                    icmphdr->checksum = 0;
                    icmphdr->checksum = csum_tcpudp_magic(Iiphdr->saddr, Iiphdr->daddr, sizeof(struct icmphdr), IPPROTO_ICMP, csum_partial(icmphdr, sizeof(struct icmphdr), 0));

                    struct sockaddr_in a;
                    a.sin_family = AF_INET;
                    a.sin_addr.s_addr = Iiphdr->daddr;
                    memset(&a.sin_zero, 0, sizeof(a.sin_zero));

                    //fprintf(stdout, "Sending %d bytes to %s\n", Iiphdr->tot_len, inet_ntoa(a.sin_addr));

                    uint16_t sent;

                    if ((sent = sendto(stuff->tcpsockfd, newData, Iiphdr->tot_len, 0, (struct sockaddr *)&a, sizeof(a))) < 0)
                    {
                        perror("sentto");
                    }
                    else
                    {
                        sentPacketCount++;
                    }
                    
                }
            }
        }
    }

    pthread_exit(NULL);
}

int main(int argc, char *argv[])
{
    if (argc < 7)
    {
        fprintf(stderr, "Usage: %s <Listen IP> <Listen Port> <Destination IP> <Nat IP> <Nat Port> <Interface> [<Threads>]\n", argv[0]);

        exit(1);
    }

    time_t startingTime = time(NULL);

    uint8_t threads = get_nprocs_conf();

    if (argc > 7)
    {
        threads = atoi(argv[7]);
    }

    int sockfd, udpsockfd, tcpsockfd;

    // Set up receiving socket that processes all packets on a certain interface :)
    sockfd = socket(AF_PACKET, SOCK_RAW, htons(ETH_P_IP));

    if (sockfd == -1)
    {
        fprintf(stderr, "Socket() :: Error - %s\n", strerror(errno));
        perror("socket");

        exit(1);
    }

    if (setsockopt(sockfd, SOL_SOCKET, SO_BINDTODEVICE, argv[6], strlen(argv[6])) < 0)
    {
        fprintf(stderr, "SetSockOpt() :: Error %s\n", strerror(errno));
        perror("setsockopt");

        exit(1);
    }

    // Set up UDP sending socket.
    udpsockfd = socket(PF_INET, SOCK_RAW, IPPROTO_UDP);

    if (udpsockfd <= 0)
    {
        fprintf(stderr, "Socket() :: Error with UDP sending socket - %s\n", strerror(errno));
        perror("socket");

        exit(1);
    }

    fprintf(stdout, "Created sending socket (UDP) %" PRIu8 "\n", udpsockfd);

    // Pass our own headers.
    int one = 1;
    if (setsockopt(udpsockfd, IPPROTO_IP, IP_HDRINCL, &one, sizeof(one)) < 0)
    {
        fprintf(stderr, "SetSockOpt() :: Error setting IP Hdr for UDP socket - %s\n", strerror(errno));
        perror("setsockopt");

        exit(1);
    }

    // Set up TCP sending socket.
    tcpsockfd = socket(PF_INET, SOCK_RAW, IPPROTO_TCP);

    if (tcpsockfd <= 0)
    {
        fprintf(stderr, "Socket() :: Error with TCP sending socket - %s\n", strerror(errno));
        perror("socket");

        exit(1);
    }

    fprintf(stdout, "Created sending socket (TCP) %" PRIu8 "\n", tcpsockfd);

    // Pass our own headers.
    if (setsockopt(tcpsockfd, IPPROTO_IP, IP_HDRINCL, &one, sizeof(one)) < 0)
    {
        fprintf(stderr, "SetSockOpt() :: Error setting IP Hdr for TCP socket - %s\n", strerror(errno));
        perror("setsockopt");

        exit(1);
    }

    // Setup sockaddr_in structure.
    struct sockaddr_in sin;
    sin.sin_family = AF_INET;
    sin.sin_addr.s_addr = inet_addr(argv[1]);
    sin.sin_port = 0;
    memset(&sin.sin_zero, 0, sizeof(sin.sin_zero));

    fprintf(stdout, "Binding to %s:%u and redirecting to %s (%s:%u) with %" PRIu8 " threads and on interface %s.\n\n", argv[1], atoi(argv[2]), argv[3], argv[4], atoi(argv[5]), threads, argv[6]);

    struct stuff stuff;
    stuff.lIP = argv[1];
    stuff.lPort = atoi(argv[2]);
    stuff.dIP = argv[3];
    stuff.nIP = argv[4];
    stuff.dPort = atoi(argv[5]);
    stuff.interface = argv[6];
    stuff.sockfd = sockfd;
    stuff.udpsockfd = udpsockfd;
    stuff.tcpsockfd = tcpsockfd;
    stuff.sockin = &sin;

    for (uint8_t i = 0; i < threads; i++)
    {
        fprintf(stdout, "Starting thread #%" PRIu8 "\n", i);

        // Create new thread.
        pthread_t pid;

        if (pthread_create(&pid, NULL, threadHndl, (void *)&stuff) != 0)
        {
            fprintf(stderr, "Error creating thread #%lu\n", pid);
        }
    }

    signal(SIGINT, sigHndl);

    while(cont)
    {
        // Allow the program to stay up until signaled to shutdown.
        sleep(1);
    }

    close(sockfd);
    close(udpsockfd);
    close(tcpsockfd);

    time_t stoppingTime = time(NULL);

    time_t timeE = stoppingTime - startingTime;

    fprintf(stdout, "%d received packets and %d sent sockets in %jd seconds\n\n\n", packetCount, sentPacketCount, timeE);

    exit(0);
}

If you see anything that might be causing these issues, please let me know! I code will be highly simplified when I make it official (e.g. using switch and functions for processing the packets instead of code for each protocol, etc).

Thanks!

~~Roy~~ · March 12, 2020

Also wanted to give a shoutout to Stack Overflow. I got a lot of advice by making the following threads:

https://stackoverflow.com/questions/60551530/best-way-to-shift-characters-to-the-right-in-char-array

https://stackoverflow.com/questions/60471858/best-way-to-receive-process-high-amounts-of-packets-traffic-via-af-packet-socket

https://stackoverflow.com/questions/60405116/packet-fragmentation-with-raw-sockets-ip-udp-headers

I'll probably make another question there if I can't figure out this issue and nobody in GFL knows it.

Thanks!

**Joshy** · March 13, 2020

It makes me feel queasy looking at some of this stuff and quite honestly I don't understand it... I'm just going to start tossing some darts.

Shouldn't it be in your shift function?

memcpy(arr + i, "0", 2);

Line numbers might be a nice feature.

~~Roy~~ · March 14, 2020

23 hours ago, Joshy said:
It makes me feel queasy looking at some of this stuff and quite honestly I don't understand it... I'm just going to start tossing some darts.

Shouldn't it be in your shift function?
memcpy(arr + i, "0", 2);
Line numbers might be a nice feature.

The function is copying one byte of data (in this case, '0') to the arr char starting at the i index. It shouldn't be 2 in this case since we're only copying one byte. I also agree with the line numbers suggestion, it'd be cool if IPS 4 could add that

In regards to my issue, I found out the cause, but finding a proper solution is a pain. The issue is outlined in this post on Stack Overflow towards the end of the reply. Basically, since we're using raw sockets to forward the TCP traffic, the kernel doesn't know that the TCP traffic is being forwarded to the destination machine. Therefore, it sees the TCP traffic coming through, doesn't see a valid TCP handshake, and sends a TCP packet with the RST flag set which resets the TCP connection. There's a solution with IPTables which is basically just adding the following rule to the OUTPUT chain on the filter table:

iptables -A OUTPUT -p tcp --tcp-flags RST RST -j DROP

* I tested this and it does confirm work when this IPTables rule is added.

However, this blocks all outbound TCP traffic with the RST flag set. I don't really consider this a valid solution since:

We can't predict which TCP destination ports will be used. Therefore, we can't filter the rule further and make it only affect traffic being forwarded in our program.
Due to item 1, it is also impacting packets outside of the program. There are many cases where it's valid to send TCP traffic with the RST flag set.

We could filter it further by making it so it only rejects those type of packets when going to the destination machine (e.g. adding -d <Destination IP> to the command). I still don't consider this a solution since forwarded traffic can also have valid cases with using the RST flag.

The two options I have right now are:

Create a dummy TCP socket and bind it to the IP and port before forwarding traffic so the kernel sees there's some sort of connection on the IP:Port. The socket doesn't have to do anything and should be closed after forwarding the traffic.
Use a AF_PACKET/PF_PACKET raw socket to send the TCP traffic. I would need to send my own Ethernet header for this, but that's easy since the socket that receives the packets uses AF_PACKET and includes an Ethernet header already. Additionally, I believe I'll have to change the source MAC address to a MAC address of an IP on the local network that isn't used by other interfaces on the machine and then handle ARP requests to this MAC address myself.

At first, I was thinking I'd need to go with option 2 since I feel creating a dummy TCP socket for every single incoming TCP request is a bit hectic and would cause performance issues, etc. However, I'm considering just going with option 1 since option 2 is a pain to setup. I've been having a lot of trouble getting the AF_PACKET/PF_PACKET raw socket to send the traffic.

We'll have to see. I'm still going to try to get option 2 to work because I really feel option 1 would result in other issues and possible attack vulnerabilities (e.g. an attacker can start sending a ton of TCP packets with all different ports and fill up the max file limit for sockets).

Thanks!

Edited March 14, 2020 by Roy

JGuary55 · March 14, 2020

Ima start throwing up if you keep posting more my brain cant handle this Roy, Google translate doesnt work for this xD

~~Roy~~ · March 14, 2020

22 minutes ago, JGuary551 said:

Ima start throwing up if you keep posting more my brain cant handle this Roy, Google translate doesnt work for this xD

Pretty interesting stuff lol. I'm going to teach you all of this

Also, so I'm not sure what I'm doing wrong here. Here's the code that initializes the AF_PACKET/PF_PACKET socket:

// Set up TCP sending socket.
tcpsockfd = socket(PF_PACKET, SOCK_RAW, IPPROTO_RAW);

if (tcpsockfd <= 0)
{
  fprintf(stderr, "Socket() :: Error with TCP sending socket - %s\n", strerror(errno));
  perror("socket");

  exit(1);
}

fprintf(stdout, "Created sending socket (TCP) %" PRIu8 "\n", tcpsockfd);

if (setsockopt(tcpsockfd, SOL_SOCKET, SO_BINDTODEVICE, argv[6], strlen(argv[6])) < 0)
{
  fprintf(stderr, "SetSockOpt() :: Error (TCP Socket) - %s\n", strerror(errno));
  perror("setsockopt");

  exit(1);
}

Here's the code that creates the IPIP packet and attempts to forward it via the AF_PACKET/PF_PACKET socket:

// Create new char and copy buffer to it.
unsigned char newData[MAX_PCKT_LENGTH];
memcpy(newData, buffer, MAX_PCKT_LENGTH);

// Add outter IP frame after the Ethernet header.
shiftChar(newData + sizeof(struct ethhdr), sizeof(struct iphdr), received - sizeof(struct ethhdr));

// Setup headers including the outer IP header.
struct ethhdr *ethhdr = (struct ethhdr *) (newData);
struct iphdr *oiphdr = (struct iphdr *) (newData + sizeof(struct ethhdr));
struct iphdr *Niphdr = (struct iphdr *) (newData + sizeof(struct ethhdr) + sizeof(struct iphdr));
struct tcphdr *Ntcphdr = (struct tcphdr *) (newData + sizeof(struct ethhdr) + sizeof(struct iphdr) + (iphdr->ihl * 4));

// Fill out the outer IP header.
oiphdr->version = 4;
oiphdr->ihl = 5;
oiphdr->protocol = IPPROTO_IPIP;
oiphdr->frag_off = 0;
oiphdr->check = 0;
oiphdr->ttl = 64;
oiphdr->tos = 0x0;
oiphdr->daddr = inet_addr(stuff->dIP);
oiphdr->saddr = inet_addr(stuff->lIP);
oiphdr->tot_len = ntohs(Niphdr->tot_len) + sizeof(struct iphdr);

// Check... SUM!
oiphdr->check = ip_fast_csum(oiphdr, oiphdr->ihl);

// Save old destination address of inner IP header for checksum recalculation and set the inner IP header's destination address to the NAT IP (with IPIP, it performs NAT to the tunnel's internal IP after forwarded).
uint32_t oldDaddr = Niphdr->daddr;
Niphdr->daddr = inet_addr(stuff->nIP);

// More checksums :P
Niphdr->check = csum_diff4(oldDaddr, Niphdr->daddr, Niphdr->check);
Ntcphdr->check = csum_diff4(oldDaddr, Niphdr->daddr, Ntcphdr->check);

// Aight, this is where it gets interesting. We initialize the a sockaddr_ll structure. This is what's used for AF_PACKET/PF_PACKET sockets.
struct sockaddr_ll a;

// Set the structure to all 0's.
memset(&a, 0, sizeof(a));

// Change the family to AF_PACKET.
a.sll_family = AF_PACKET;

// Set the IF Index to the index of the interface specified in the command line (I made sure this was valid).
a.sll_ifindex = if_nametoindex(stuff->interface);

// I don't think this is needed, but this is setting the protocol to the Ethernet protocol. Should be the same regardless :PPPP
a.sll_protocol = ethhdr->h_proto;

// Okay, so normally we'd set a.sll_addr => ethhdr->h_source (the source MAC address of the incoming packet). But this isn't what we want since we want the source MAC address to be the MAC address of the interface specified in the command line. Since the AF_PACKET socket that receives traffic operates on the proper correct interface, we can just copy the destination MAC address of the ethernet header to the source MAC address of the a structure.
memcpy(a.sll_addr, ethhdr->h_dest, 6 * sizeof (uint8_t));

// 6 byte MAC address (IPv4).
a.sll_halen = 6;

// Debug information.
fprintf(stdout, "%d is tcphdr->doff and %d is Niphdr->saddr. %d is oiphdr->saddr. %d is Niphdr->daddr. %d is oiphdr->version. %d is Niphdr->version. %d is oiphdr->ttl. %d is Niphdr->ttl. %d is ntohs(tcphdr->dest). %d is ethhdr->h_proto..\n", tcphdr->doff, Niphdr->saddr, oiphdr->saddr, Niphdr->daddr, oiphdr->version, Niphdr->version, oiphdr->ttl, Niphdr->ttl, ntohs(tcphdr->dest), ethhdr->h_proto);
fprintf(stdout, "%d is the index.\n", a.sll_ifindex);
fprintf(stdout, "Mac adress => ");

for (int i = 0; i < 6; i++)
{
	fprintf(stdout, "%02x:", a.sll_addr[i]);
}

fprintf(stdout, "\n");

uint16_t sent;

// I tried sendto() and write(). All same results.
if ((sent = sendto(stuff->tcpsockfd, newData, oiphdr->tot_len + sizeof(struct ethhdr), 0, (struct sockaddr *)&a, sizeof(a))) < 0)
//if ((sent = write(stuff->tcpsockfd, newData, oiphdr->tot_len)) < 0)
{
	perror("sentto");
}

// This actually returns that we're forwarding the correct amount of traffic.
fprintf(stdout, "Forwarded %d (%lu) bytes to proxy.\n\n", sent, oiphdr->tot_len + sizeof(struct ethhdr));

I added a bunch of comments to try to explain what I was doing along with the results. I am going to try some more things (e.g. using the bind() function instead of setsockopt() function when binding the sending socket to the interface).

I am not seeing any of the packets go anywhere via tcpdump.

Thanks!

Edited March 14, 2020 by Roy

JGuary55 · March 14, 2020

obviously i dont know what the fuck is going on but i do get what your trying to do, and it is quite interesting I just wish i know wtf that all meant lmaooo

TokenDude · March 14, 2020

Well JGuary let me explain a shit ton of codes everywhere. That’s how people make cool games. :benthinking:

~~Roy~~ · March 15, 2020

Just an update on this, I am trying to send the TCP packets using an AF_PACKET socket with SOCK_RAW. From what I've read, this will make it so the kernel doesn't do anything to the packet (e.g. set the RST flag). There is a possibility I will need to send ARP requests to get the destination's MAC address.

I created a test program that sends an empty UDP packet using the above socket from one of my home VMs to the other. I am doing this while knowing the MAC address of the destination server (since it's a VM on my home server). However, this program isn't working. I made a Stack Overflow thread here and hopefully will get a helpful response.

Here's the test program's code:

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/socket.h>
#include <sys/types.h>
#include <netinet/ip.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <net/if.h>
#include <linux/if.h>
#include <linux/if_packet.h>
#include <linux/udp.h>
#include <net/ethernet.h>
#include <string.h>
#include <error.h>
#include <errno.h>
#include <inttypes.h>

#define REDIRECT_HEADER

#include "csum.h"

#define MAX_PCKT_LENGTH 65535

int main()
{
    int sockfd;
    struct sockaddr_ll dst;
    char pckt[MAX_PCKT_LENGTH];

    sockfd = socket(AF_PACKET, SOCK_RAW, IPPROTO_RAW);

    if (sockfd <= 0)
    {
        perror("socket");

        exit(1);
    }

    dst.sll_family = AF_PACKET;
    dst.sll_protocol = ETH_P_IP;

    if ((dst.sll_ifindex = if_nametoindex("ens18")) == 0)
    {
        fprintf(stdout, "Interface 'ens18' not found.\n");

        exit(1);
    }

    // Do destination ethernet MAC (ae:21:14:4b:3a:6d).
    dst.sll_addr[0] = 0xAE;
    dst.sll_addr[1] = 0x21;
    dst.sll_addr[2] = 0x14;
    dst.sll_addr[3] = 0x4B;
    dst.sll_addr[4] = 0x3A;
    dst.sll_addr[5] = 0x6D;
    dst.sll_halen = 6;

    // I tried doing this with and without bind. Still not working.
    if (bind(sockfd, (struct sockaddr *)&dst, sizeof(dst)) < 0)
    {
        perror("bind");

        exit(1);
    }

    struct ethhdr *ethhdr = (struct ethhdr *) (pckt);
    struct iphdr *iphdr = (struct iphdr *) (pckt + sizeof(struct ethhdr));
    struct udphdr *udphdr = (struct udphdr *) (pckt + sizeof(struct ethhdr) + sizeof(struct iphdr));
    unsigned char *data = (unsigned char *) (pckt + sizeof(struct ethhdr) + sizeof(struct iphdr) + sizeof(struct udphdr));

    // Do source ethernet MAC (1a:c4:df:70:d8:a6).
    ethhdr->h_source[0] = 0x1A;
    ethhdr->h_source[1] = 0xC4;
    ethhdr->h_source[2] = 0xDF;
    ethhdr->h_source[3] = 0x70;
    ethhdr->h_source[4] = 0xD8;
    ethhdr->h_source[5] = 0xA6;

    // Copy destination MAC to sockaddr_ll.
    memcpy(ethhdr->h_dest, dst.sll_addr, 6);

    // Protocol.
    ethhdr->h_proto = ETH_P_IP;

    // Fill out ip header.
    iphdr->ihl = 5;
    iphdr->version = 4;
    iphdr->frag_off = 0;
    iphdr->id = rand();
    iphdr->protocol = IPPROTO_UDP;
    iphdr->tos = 0x0;
    iphdr->ttl = 64;
    iphdr->saddr = inet_addr("10.50.0.3");
    iphdr->daddr = inet_addr("10.50.0.4");
    iphdr->tot_len = sizeof(struct iphdr) + sizeof(struct udphdr);
    iphdr->check = 0;
    iphdr->check = ip_fast_csum(iphdr, iphdr->ihl);

    // Fill out UDP header.
    udphdr->source = htons(27000);
    udphdr->dest = htons(27015);
    udphdr->len = htons(sizeof(struct udphdr));
    udphdr->check = 0;
    udphdr->check = csum_tcpudp_magic(iphdr->saddr, iphdr->daddr, sizeof(struct udphdr), IPPROTO_UDP, csum_partial(udphdr, sizeof(struct udphdr), 0));

    // Send packet
    uint16_t sent;

    if ((sent = sendto(sockfd, pckt, iphdr->tot_len + sizeof(struct ethhdr), 0, (struct sockaddr *)&dst, sizeof(dst))) < 0)
    {
        perror("sendto");
    }

    fprintf(stdout, "Sent %d of data.\n", sent);

    close(sockfd);

    exit(0);
}

I'm honestly very stumped on what the issue could be. The program returns that it has sent data. However, I see no packets on the source and destination servers via tcpdump.

Thanks!

~~Roy~~ · March 15, 2020

I got this working! I wasn't converting the Ethernet protocol (ETH_P_IP) to network byte order via htons() since it's a big endian. With that said, I had to convert the iphdr->total_len to network byte order as well. In my other programs, I didn't do this, but the kernel would convert it automatically. Since I'm using completely raw sockets, I need to make sure I'm doing everything the kernel would normally do.

Here's a packet capture when not converting the IP header's total length to network byte order:

14:56:26.938672 Out 1a:c4:df:70:d8:a6 ethertype IPv4 (0x0800), length 104: truncated-ip - 14760 bytes missing! (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 14848)
    10.50.0.3.27000 > 10.50.0.4.27015: [udp sum ok] UDP, length 30
        0x0000:  0004 0001 0006 1ac4 df70 d8a6 0000 0800  .........p......
        0x0010:  4500 3a00 0000 0000 4011 2c83 0a32 0003  E.:.....@.,..2..
        0x0020:  0a32 0004 6978 6987 0026 5474 6262 6262  .2..ixi..&Ttbbbb
        0x0030:  6262 6262 6262 6262 6262 6262 6262 6262  bbbbbbbbbbbbbbbb
        0x0040:  6262 6262 6262 6262 6262 0000 0000 0000  bbbbbbbbbb......
        0x0050:  0000 0000 0000 0000 0000 0000 0000 0000  ................
        0x0060:  0000 0000 0000 0000                      ........

You see can it states the IP header length is 14848 bytes, but the packet itself is 104 bytes. Therefore, it complains data is missing. Here's a capture where everything is working as normal:

15:06:14.746686 Out 1a:c4:df:70:d8:a6 ethertype IPv4 (0x0800), length 104: (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 58)
    10.50.0.3.27000 > 10.50.0.4.27015: [udp sum ok] UDP, length 30
        0x0000:  0004 0001 0006 1ac4 df70 d8a6 0000 0800  .........p......
        0x0010:  4500 003a 0000 0000 4011 6649 0a32 0003  E..:[email protected]..
        0x0020:  0a32 0004 6978 6987 0026 5474 6262 6262  .2..ixi..&Ttbbbb
        0x0030:  6262 6262 6262 6262 6262 6262 6262 6262  bbbbbbbbbbbbbbbb
        0x0040:  6262 6262 6262 6262 6262 0000 0000 0000  bbbbbbbbbb......
        0x0050:  0000 0000 0000 0000 0000 0000 0000 0000  ................
        0x0060:  0000 0000 0000 0000                      ........

Here's the final code:

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/socket.h>
#include <sys/types.h>
#include <netinet/ip.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <net/if.h>
#include <linux/if.h>
#include <linux/if_packet.h>
#include <linux/udp.h>
#include <net/ethernet.h>
#include <string.h>
#include <error.h>
#include <errno.h>
#include <inttypes.h>

#define REDIRECT_HEADER

#include "csum.h"

#define MAX_PCKT_LENGTH 65535

int main()
{
    int sockfd;
    struct sockaddr_ll dst;
    char pckt[MAX_PCKT_LENGTH];

    sockfd = socket(AF_PACKET, SOCK_RAW, IPPROTO_RAW);

    if (sockfd <= 0)
    {
        perror("socket");

        exit(1);
    }

    dst.sll_family = PF_PACKET;
    dst.sll_protocol = htons(ETH_P_IP);

    if ((dst.sll_ifindex = if_nametoindex("ens18")) == 0)
    {
        fprintf(stdout, "Interface 'ens18' not found.\n");

        exit(1);
    }

    // Do destination ethernet MAC (ae:21:14:4b:3a:6d).
    dst.sll_addr[0] = 0xAE;
    dst.sll_addr[1] = 0x21;
    dst.sll_addr[2] = 0x14;
    dst.sll_addr[3] = 0x4B;
    dst.sll_addr[4] = 0x3A;
    dst.sll_addr[5] = 0x6D;
    dst.sll_halen = ETH_ALEN;

    // I tried doing this with and without bind. Still not working.
    if (bind(sockfd, (struct sockaddr *)&dst, sizeof(dst)) < 0)
    {
        perror("bind");

        exit(1);
    }

    struct ethhdr *ethhdr = (struct ethhdr *) (pckt);
    struct iphdr *iphdr = (struct iphdr *) (pckt + sizeof(struct ethhdr));
    struct udphdr *udphdr = (struct udphdr *) (pckt + sizeof(struct ethhdr) + sizeof(struct iphdr));
    unsigned char *data = (unsigned char *) (pckt + sizeof(struct ethhdr) + sizeof(struct iphdr) + sizeof(struct udphdr));

    // Do source ethernet MAC (1a:c4:df:70:d8:a6).
    ethhdr->h_source[0] = 0x1A;
    ethhdr->h_source[1] = 0xC4;
    ethhdr->h_source[2] = 0xDF;
    ethhdr->h_source[3] = 0x70;
    ethhdr->h_source[4] = 0xD8;
    ethhdr->h_source[5] = 0xA6;
    
    for (int i = 0; i < 30; i++)
    {
        memcpy(data + i, "b", 1);
    }

    // Copy destination MAC to sockaddr_ll.
    memcpy(ethhdr->h_dest, dst.sll_addr, ETH_ALEN);

    // Protocol.
    ethhdr->h_proto = htons(ETH_P_IP);

    // Fill out ip header.
    iphdr->ihl = 5;
    iphdr->version = 4;
    iphdr->frag_off = 0;
    iphdr->id = htons(0);
    iphdr->protocol = IPPROTO_UDP;
    iphdr->tos = 0x0;
    iphdr->ttl = 64;
    iphdr->saddr = inet_addr("10.50.0.3");
    iphdr->daddr = inet_addr("10.50.0.4");
    iphdr->tot_len = htons(sizeof(struct iphdr) + sizeof(struct udphdr) + 30);
    iphdr->check = 0;
    iphdr->check = ip_fast_csum(iphdr, iphdr->ihl);

    // Fill out UDP header.
    udphdr->source = htons(27000);
    udphdr->dest = htons(27015);
    udphdr->len = htons(sizeof(struct udphdr) + 30);
    udphdr->check = 0;
    udphdr->check = csum_tcpudp_magic(iphdr->saddr, iphdr->daddr, sizeof(struct udphdr) + 30, IPPROTO_UDP, csum_partial(udphdr, sizeof(struct udphdr) + 30, 0));

    // Send packet
    uint16_t sent;
    int len = ntohs(iphdr->tot_len) + sizeof(struct ethhdr) + 30;

    if ((sent = sendto(sockfd, pckt, len, 0, (struct sockaddr *)&dst, sizeof(dst))) < 0)
    //if ((sent = write(sockfd, pckt, len)) < 0)
    {
        perror("sendto");
    }

    fprintf(stdout, "Sent %d of data. %d is IPHdr len. %d is len.\n", sent, iphdr->tot_len, len);

    close(sockfd);

    exit(0);
}

I made some other small changes such as replacing the MAC address length with ETH_ALEN (which is 6 bytes). That said, both write() and sendto() work when sending packets via AF_PACKET sockets. However, since I made struct sockaddr_ll already, I figured I'd use it via sendto().

Thanks!

**Joshy** · March 15, 2020

It may not be the cause of the problem, but maybe this is compiler dependent? If it's a problem, then it could propagate into other parts of the code or create errors in the long run unless everything you are overwriting is exactly the same size? When I ran it on mine if you don't include that extra byte it ignores the null so you still have the old garbage in there.

#include <stdio.h>
#include <string.h>

int main () 
{
   const char src[20] = "0";
   char buffer[20];
   
   strcpy(buffer,"Overwrite this");
   printf("Before memcpy, buffer = %s\n", buffer);
   
   // src is "0"
   memcpy(buffer, src, 1);
   printf("After memcpy, buffer = %s\n", buffer);
   
   return 0;
}

If you change the size to 2, then it'll completely overwrite it.

~~Roy~~ · March 16, 2020

1 hour ago, Joshy said:
It may not be the cause of the problem, but maybe this is compiler dependent? If it's a problem, then it could propagate into other parts of the code or create errors in the long run unless everything you are overwriting is exactly the same size? When I ran it on mine if you don't include that extra byte it ignores the null so you still have the old garbage in there.
#include <stdio.h>
#include <string.h>

int main () 
{
   const char src[20] = "0";
   char buffer[20];
   
   strcpy(buffer,"Overwrite this");
   printf("Before memcpy, buffer = %s\n", buffer);
   
   // src is "0"
   memcpy(buffer, src, 1);
   printf("After memcpy, buffer = %s\n", buffer);
   
   return 0;
}
If you change the size to 2, then it'll completely overwrite it.

Interesting. So I'd assume it overwrites the entire string because src had only one byte of valid data. I'd assume this is some sort of bug though since it should be only copying two bytes of data max regardless of non-valid data. This also happens when I try to compile the program on Linux.

I think the best way to prevent something like this is to copy the data with a loop (copying one byte at a time and ensuring the source string isn't garbage) which is what I do in the networking program. I copied the above code you had and made some changes:

#include <stdio.h>
#include <string.h>

void cpy(int size)
{
    printf("Starting replace with %d size\n", size);
    printf("--------------------------------------------------\n\n");

    const char src[20] = "0";
    char buffer[20];

    strcpy(buffer,"Overwrite this");

    printf("Before memcpy, buffer = %s\n", buffer);

    // src is "0"
    memcpy(buffer, src, size);
    printf("After memcpy, buffer = %s\n", buffer);

    printf("--------------------------------------------------\n\n");
}

void cpyv2(int size)
{
    printf("Starting replace with %d size (V2)\n", size);
    printf("--------------------------------------------------\n\n");

    const char src[20] = "0";
    char buffer[20];

    strcpy(buffer,"Overwrite this");

    printf("Before memcpy, buffer = %s\n", buffer);

    // src is "0"
    for (int i = 0; i < size; i++)
    {
        memcpy(buffer + i , src, 1);
    }
    
    printf("After memcpy, buffer = %s\n", buffer);

    printf("--------------------------------------------------\n\n");
}

int main() 
{
    cpy(1);
    cpy(2);
    cpy(3);

    cpyv2(1);
    cpyv2(2);
    cpyv2(3);
   
   return 0;
}

Output was:

Starting replace with 1 size
--------------------------------------------------

Before memcpy, buffer = Overwrite this
After memcpy, buffer = 0verwrite this
--------------------------------------------------

Starting replace with 2 size
--------------------------------------------------

Before memcpy, buffer = Overwrite this
After memcpy, buffer = 0
--------------------------------------------------

Starting replace with 3 size
--------------------------------------------------

Before memcpy, buffer = Overwrite this
After memcpy, buffer = 0
--------------------------------------------------

Starting replace with 1 size (V2)
--------------------------------------------------

Before memcpy, buffer = Overwrite this
After memcpy, buffer = 0verwrite this
--------------------------------------------------

Starting replace with 2 size (V2)
--------------------------------------------------

Before memcpy, buffer = Overwrite this
After memcpy, buffer = 00erwrite this
--------------------------------------------------

Starting replace with 3 size (V2)
--------------------------------------------------

Before memcpy, buffer = Overwrite this
After memcpy, buffer = 000rwrite this
--------------------------------------------------

Was the cpyv2() function more so what you were trying to accomplish?

Also, in regards to the networking program. I pretty much got everything working. So, the issue I ran into after my last reply was I didn't know how to get the destination's MAC address to see in the Ethernet header. This resulted in traffic not sending properly. I also learned that you cannot send and receive valid ARP requests to a destination outside of your network. Or well, perhaps it is possible, but inconvenient and I'm not sure since it's something I'll have to test later on (it'd also require creating my own ARP cache table which is a pain). Therefore, I wasn't sure what MAC address to set the destination MAC to. After researching further, I found a thread where somebody suggested setting the destination MAC address to the next hop's MAC address. In our case, this would be the router/gateway's MAC address. The router/gateway should be then resolving the MAC address of the destination (or the next hop's MAC address after the gateway/router I'd assume and it'd repeat with each hop until the ending destination).

I ended up creating a function that gets the gateway's MAC address based off of Linux commands. I don't know if this is the best way to receive the gateway's MAC address and it relies on the ip package being installed (to be fair, most Linux OS's should have this installed, lol). However, C doesn't have built-in functionality for this. Therefore, we need to use OS-specific functions to gather this information. This worked for me:

void GetGatewayMAC()
{
    char cmd[] = "ip neigh | grep \"$(ip -4 route list 0/0|cut -d' ' -f3) \"|cut -d' ' -f5|tr '[a-f]' '[A-F]'";

    FILE *fp =  popen(cmd, "r");

    if (fp != NULL)
    {
        char line[18];

        if (fgets(line, sizeof(line), fp) != NULL)
        {
            sscanf(line, "%hhx:%hhx:%hhx:%hhx:%hhx:%hhx", &routerMac[0], &routerMac[1], &routerMac[2], &routerMac[3], &routerMac[4], &routerMac[5]);
        }

        pclose(fp);
    }
}

This copies the MAC address to the routerMac variable which is a global-level variable. I then just copy the Ethernet header's destination MAC address before sending the packet:

memcpy(ethhdr->h_dest, routerMac, ETH_ALEN);

The program forwards and sends back TCP packets successfully with these changes and both internal and external connections work (I tested external connections with my NJ VM running HLSW for RCON/TCP).

I just need to change the rest of the protocols to use the AF_PACKET socket (e.g. the UDP packets). Apart from that, I also want to look into optimizing the AF_PACKET sending socket and perhaps using multiple sending sockets (perhaps one socket per thread spawned). I think in order to do this, I'll need to use something like this from the packet.7 manual page:

Here's the final code for the test program. There's still a lot of useless code in the program since I was testing things, but once I begin writing the official software (that'll run on GFL's Anycast POPs), I'll organize/optimize the program a lot further:

/* Testing forwarding packets to destination. */
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/socket.h>
#include <sys/types.h>
#include <netinet/ip.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <net/if.h>
#include <linux/if.h>
#include <linux/if_packet.h>
#include <linux/tcp.h>
#include <linux/udp.h>
#include <linux/icmp.h>
#include <net/ethernet.h>
#include <string.h>
#include <error.h>
#include <errno.h>
#include <inttypes.h>
#include <pthread.h>
#include <sys/epoll.h>
#include <sys/sysinfo.h>
#include <sys/ioctl.h>
#include <signal.h>
#include <fcntl.h>
#include <ctype.h>

#define REDIRECT_HEADER

#include "csum.h"

#define MAX_PCKT_LENGTH 65535
#define MAX_EPOLL_EVENTS 64
#define MAX_BINDS 16

extern int errno;
int cont = 1;

int packetCount = 0;
int sentPacketCount = 0;
unsigned char routerMac[ETH_ALEN];
unsigned char srcMac[ETH_ALEN];

struct stuff
{
    char *lIP;
    uint16_t lPort;
    char *dIP;
    char *nIP;
    uint16_t dPort;
    int sockfd;
    int udpsockfd;
    int tcpsockfd;
    char *interface;
};

void sigHndl(int tmp)
{
    cont = 0;
}

int FindMAC(const char IP)
{

}

void GetGatewayIP(char *IP) 
{
    char cmd[] = "ip route | grep default | awk '{print $3}'";

    FILE* fp = popen(cmd, "r");

    if (fp != NULL)
    {
        char line[11];

        if(fgets(line, sizeof(line), fp) != NULL)
        {
            strncpy(IP, line, strlen(line));
        }

        pclose(fp);
    }
}

void GetGatewayMAC()
{
    char cmd[] = "ip neigh | grep \"$(ip -4 route list 0/0|cut -d' ' -f3) \"|cut -d' ' -f5|tr '[a-f]' '[A-F]'";

    FILE *fp =  popen(cmd, "r");

    if (fp != NULL)
    {
        char line[18];

        if (fgets(line, sizeof(line), fp) != NULL)
        {
            sscanf(line, "%hhx:%hhx:%hhx:%hhx:%hhx:%hhx", &routerMac[0], &routerMac[1], &routerMac[2], &routerMac[3], &routerMac[4], &routerMac[5]);
        }

        pclose(fp);
    }
}

void shiftChar(char *arr, int size, int dataLen)
{
    for (int i = (dataLen - 1); i >= 0; i--)
    {
        memmove(arr + i + size, arr + i, 1);
    }

    for (int i = 0; i < size; i++)
    {
        memcpy(arr + i, "0", 1);
    }
}

int setPromisc(char *enterface,int *sock)
{
    struct ifreq ifr;
    strcpy(ifr.ifr_name, enterface);
    ifr.ifr_flags=IFF_UP|IFF_PROMISC|IFF_BROADCAST|IFF_RUNNING;
	//ifr.ifr_flags |= IFF_PROMISC;      // this is wrong code
    if(ioctl(*sock,SIOCSIFFLAGS,&ifr)==-1)
    {
        perror("set 'eth' to promisc model failed\n"); //cant write  '%s',enterface  why?
        exit(1);
    }
    printf("set '%s' to promisc successed!\n",enterface);
    return 1;
}

void* threadHndl(void * data)
{
    struct stuff *stuff = data;

    // Get thread ID.
    pthread_t threadID = pthread_self();

    // Headers.
    unsigned char buffer[MAX_PCKT_LENGTH];

    //struct ethhdr *ethhdr = (struct ethhdr *) buffer;
    struct iphdr *iphdr = (struct iphdr *) (buffer + sizeof(struct ethhdr));

    memset(buffer, 0, MAX_PCKT_LENGTH);

    struct sockaddr_ll din;

    socklen_t dinLen = sizeof(din);

    int8_t epoll;

    epoll = epoll_create1(0);

    if (epoll < 0)
    {
        fprintf(stderr, "EPoll() :: Error Creating - %s\n", strerror(errno));

        pthread_exit(NULL);
    }

    struct epoll_event event;
    event.events = EPOLLIN;
    event.data.fd = stuff->sockfd;

    if (epoll_ctl(epoll, EPOLL_CTL_ADD, stuff->sockfd, &event) < 0)
    {
        fprintf(stderr, "EPoll_ctl() :: Error - %s\n", strerror(errno));

        pthread_exit(NULL);
    }

    while (cont)
    {
        struct epoll_event event;

        epoll_wait(epoll, &event, 1, -1);

        if (event.data.fd == stuff->sockfd)
        {
            uint16_t received = recvfrom(event.data.fd, &buffer, MAX_PCKT_LENGTH, 0, (struct sockaddr *)&din, &dinLen);

            if (iphdr->protocol == IPPROTO_TCP)
            {
                // Handle TCP packets.
                struct tcphdr *tcphdr = (struct tcphdr *) (buffer + sizeof(struct ethhdr) + (iphdr->ihl * 4));
                unsigned char *data = (unsigned char *) (buffer + sizeof(struct ethhdr) + (iphdr->ihl * 4) + (tcphdr->doff * 4));
                uint16_t payloadLength = received - (sizeof(struct ethhdr) + (iphdr->ihl * 4) + (tcphdr->doff * 4));

                // Check if destination is listen IP and port.
                if (iphdr->daddr == inet_addr(stuff->lIP) && ntohs(tcphdr->dest) > 1024)
                {
                    unsigned char newData[MAX_PCKT_LENGTH];
                    memcpy(newData, buffer, MAX_PCKT_LENGTH);

                    // Add outter IP frame.
                    shiftChar(newData + sizeof(struct ethhdr), sizeof(struct iphdr), received - sizeof(struct ethhdr));

                    // Initialize outter IP header and shift other headers.
                    struct ethhdr *ethhdr = (struct ethhdr *) (newData);
                    struct iphdr *oiphdr = (struct iphdr *) (newData + sizeof(struct ethhdr));
                    struct iphdr *Niphdr = (struct iphdr *) (newData + sizeof(struct ethhdr) + sizeof(struct iphdr));
                    struct tcphdr *Ntcphdr = (struct tcphdr *) (newData + sizeof(struct ethhdr) + sizeof(struct iphdr) + (iphdr->ihl * 4));
                    
                    oiphdr->version = 4;
                    oiphdr->ihl = 5;
                    oiphdr->protocol = IPPROTO_IPIP;
                    oiphdr->frag_off = 0;
                    oiphdr->check = 0;
                    oiphdr->ttl = 64;
                    oiphdr->tos = 0x0;
                    oiphdr->daddr = inet_addr(stuff->dIP);
                    oiphdr->saddr = inet_addr(stuff->lIP);
                    oiphdr->tot_len = htons(ntohs(Niphdr->tot_len) + sizeof(struct iphdr));

                    oiphdr->check = ip_fast_csum(oiphdr, oiphdr->ihl);

                    uint32_t oldDaddr = Niphdr->daddr;
                    Niphdr->daddr = inet_addr(stuff->nIP);

                    Niphdr->check = csum_diff4(oldDaddr, Niphdr->daddr, Niphdr->check);
                    Ntcphdr->check = csum_diff4(oldDaddr, Niphdr->daddr, Ntcphdr->check);

                    memcpy(ethhdr->h_source, srcMac, ETH_ALEN);
                    ethhdr->h_proto = htons(ETH_P_IP);
                    
                    // Set destination MAC.
                    memcpy(ethhdr->h_dest, routerMac, ETH_ALEN);

                    uint16_t sent;

                    if ((sent = write(stuff->tcpsockfd, newData, ntohs(oiphdr->tot_len) + sizeof(struct ethhdr))) < 0)
                    {
                        perror("sentto");
                    }

                    fprintf(stdout, "Forwarded %d (%lu) bytes to proxy.\n\n", sent, ntohs(oiphdr->tot_len) + sizeof(struct ethhdr));
                }

                packetCount++;
            }
            else if (iphdr->protocol == IPPROTO_UDP)
            {
                // Handle UDP packets.
                struct udphdr *udphdr = (struct udphdr *) (buffer + sizeof(struct ethhdr) + (iphdr->ihl * 4));
                unsigned char *data = (unsigned char *) (buffer + sizeof(struct ethhdr) + (iphdr->ihl * 4) + sizeof(struct udphdr));
                uint16_t payloadLength = received - (sizeof(struct ethhdr) + (iphdr->ihl * 4) + sizeof(struct udphdr));

                // Check if destination is listen IP and port.
                if (iphdr->daddr == inet_addr(stuff->lIP) && ntohs(udphdr->dest) > 1024)
                {
                    unsigned char newData[MAX_PCKT_LENGTH];
                    memcpy(newData, buffer + sizeof(struct ethhdr), MAX_PCKT_LENGTH - sizeof(struct ethhdr));

                    // Add outter IP frame.
                    shiftChar(newData, sizeof(struct iphdr), received - sizeof(struct ethhdr));

                    // Initialize outter IP header and shift other headers.
                    struct iphdr *Niphdr = (struct iphdr *) (newData +  sizeof(struct iphdr));
                    struct udphdr *Nudphdr = (struct udphdr *) (newData  + sizeof(struct iphdr) + (iphdr->ihl * 4));

                    struct iphdr *oiphdr = (struct iphdr *) (newData);
                    
                    oiphdr->version = 4;
                    oiphdr->ihl = 5;
                    oiphdr->protocol = IPPROTO_IPIP;
                    oiphdr->frag_off = 0;
                    oiphdr->check = 0;
                    oiphdr->ttl = 64;
                    oiphdr->tos = 0x50;
                    oiphdr->daddr = inet_addr(stuff->dIP);
                    oiphdr->saddr = inet_addr(stuff->lIP);
                    oiphdr->tot_len = ntohs(Niphdr->tot_len) + sizeof(struct iphdr);

                    oiphdr->check = ip_fast_csum(oiphdr, oiphdr->ihl);

                    uint32_t oldDaddr = Niphdr->daddr;
                    Niphdr->daddr = inet_addr(stuff->nIP);

                    Niphdr->check = csum_diff4(oldDaddr, Niphdr->daddr, Niphdr->check);
                    Nudphdr->check = csum_diff4(oldDaddr, Niphdr->daddr, Nudphdr->check);

                    struct sockaddr_in a;
                    a.sin_family = AF_INET;
                    a.sin_addr.s_addr = inet_addr(stuff->dIP);
                    memset(&a.sin_zero, 0, sizeof(a.sin_zero));

                    uint16_t sent;

                    if ((sent = sendto(stuff->udpsockfd, newData, oiphdr->tot_len, 0, (struct sockaddr *)&a, sizeof(a))) < 0)
                    {
                        perror("sentto");
                    }
                }

                packetCount++;
            }
            else if (iphdr->protocol == IPPROTO_ICMP)
            {
                // Handle TCP packets.
                struct icmphdr *icmphdr = (struct icmphdr *) (buffer + sizeof(struct ethhdr) + (iphdr->ihl * 4));
                unsigned char *data = (unsigned char *) (buffer + sizeof(struct ethhdr) + (iphdr->ihl * 4) + sizeof(struct icmphdr));
                uint16_t payloadLength = received - (sizeof(struct ethhdr) + (iphdr->ihl * 4) + sizeof(struct icmphdr));

                // Check if destination is listen IP and port.
                if (iphdr->daddr == inet_addr(stuff->lIP))
                {
                    unsigned char newData[MAX_PCKT_LENGTH];
                    memcpy(newData, buffer + sizeof(struct ethhdr), MAX_PCKT_LENGTH - sizeof(struct ethhdr));

                    // Add outter IP frame.
                    shiftChar(newData, sizeof(struct iphdr), received - sizeof(struct ethhdr));

                    // Initialize outter IP header and shift other headers.
                    struct iphdr *Niphdr = (struct iphdr *) (newData +  sizeof(struct iphdr));
                    struct icmphdr *Nicmphdr = (struct icmphdr *) (newData  + sizeof(struct iphdr) + (iphdr->ihl * 4));

                    struct iphdr *oiphdr = (struct iphdr *) (newData);
                    
                    oiphdr->version = 4;
                    oiphdr->ihl = 5;
                    oiphdr->protocol = IPPROTO_IPIP;
                    oiphdr->frag_off = 0;
                    oiphdr->check = 0;
                    oiphdr->ttl = 64;
                    oiphdr->tos = 0x50;
                    oiphdr->daddr = inet_addr(stuff->dIP);
                    oiphdr->saddr = inet_addr(stuff->lIP);
                    oiphdr->tot_len = ntohs(Niphdr->tot_len) + sizeof(struct iphdr);

                    oiphdr->check = ip_fast_csum(oiphdr, oiphdr->ihl);

                    uint32_t oldDaddr = Niphdr->daddr;
                    Niphdr->daddr = inet_addr(stuff->nIP);

                    Niphdr->check = csum_diff4(oldDaddr, Niphdr->daddr, Niphdr->check);

                    Nicmphdr->checksum = 0;
                    Nicmphdr->checksum = csum_tcpudp_magic(Niphdr->saddr, Niphdr->daddr, sizeof(struct icmphdr), IPPROTO_TCP, csum_partial(Nicmphdr, sizeof(struct icmphdr), 0));

                    struct sockaddr_in a;
                    a.sin_family = AF_INET;
                    a.sin_addr.s_addr = inet_addr(stuff->dIP);
                    memset(&a.sin_zero, 0, sizeof(a.sin_zero));

                    uint16_t sent;

                    if ((sent = sendto(stuff->tcpsockfd, newData, oiphdr->tot_len, 0, (struct sockaddr *)&a, sizeof(a))) < 0)
                    {
                        perror("sentto");
                    }
                    //fprintf(stdout, "Forwarded %d (%d) bytes to proxy.\n", sent, oiphdr->tot_len);
                }

                packetCount++;
            }
            else if (iphdr->protocol == IPPROTO_IPIP)
            {
                // Send the packet back after checking.
                struct iphdr *Iiphdr = (struct iphdr *) (buffer + sizeof(struct ethhdr) + (iphdr->ihl * 4));

                if (Iiphdr->protocol == IPPROTO_UDP)
                {
                    // Let's send back the UDP packet.

                    // Start new data from inner header.
                    struct udphdr *udphdr = (struct udphdr *) (buffer + sizeof(struct ethhdr) + (iphdr->ihl * 4) + (Iiphdr->ihl * 4));
                    unsigned char *newData = (unsigned char *)(buffer + sizeof(struct ethhdr) + (iphdr->ihl * 4));
                    uint16_t payloadLength = received - sizeof(struct ethhdr) - (iphdr->ihl * 4) - (Iiphdr->ihl * 4) - sizeof(struct udphdr);

                    // Fill in necessary IP header data.
                    uint32_t oldAddr = iphdr->saddr;
                    Iiphdr->saddr = iphdr->daddr;
                    Iiphdr->tot_len = (Iiphdr->ihl * 4) + sizeof(struct udphdr) + payloadLength;

                    // Recalculate checksums.
                    Iiphdr->check = ip_fast_csum(Iiphdr, Iiphdr->ihl);
                    udphdr->check = 0;
                    udphdr->check = csum_tcpudp_magic(Iiphdr->saddr, Iiphdr->daddr, sizeof(struct udphdr) + payloadLength, IPPROTO_UDP, csum_partial(udphdr, sizeof(struct udphdr) + payloadLength, 0));

                    struct sockaddr_in a;
                    a.sin_family = AF_INET;
                    a.sin_addr.s_addr = Iiphdr->daddr;
                    memset(&a.sin_zero, 0, sizeof(a.sin_zero));

                    //fprintf(stdout, "Sending %d bytes to %s:%d from %d:%d\n", Iiphdr->tot_len, inet_ntoa(a.sin_addr), ntohs(udphdr->dest), Iiphdr->saddr, ntohs(udphdr->source));

                    uint16_t sent;

                    if ((sent = sendto(stuff->udpsockfd, newData, Iiphdr->tot_len, 0, (struct sockaddr *)&a, sizeof(a))) < 0)
                    {
                        perror("sentto");
                    }
                    else
                    {
                        sentPacketCount++;
                    }
                    
                }
                else if (Iiphdr->protocol == IPPROTO_TCP)
                {
                    // Let's send back the TCP packet.

                    // Start new data from inner header.
                    struct ethhdr *ethhdr = (struct ethhdr *) (buffer);
                    struct tcphdr *tcphdr = (struct tcphdr *) (buffer + sizeof(struct ethhdr) + (iphdr->ihl * 4) + (Iiphdr->ihl * 4));
                    unsigned char *data = (unsigned char *) (buffer + sizeof(struct ethhdr) + (iphdr->ihl * 4) + (Iiphdr->ihl * 4) + (tcphdr->doff * 4));
                    uint16_t payloadLength = received - sizeof(struct ethhdr) - (iphdr->ihl * 4) - (Iiphdr->ihl * 4) - (tcphdr->doff * 4);
                    
                    char newData[MAX_PCKT_LENGTH];
                    struct ethhdr *Nethhdr = (struct ethhdr *) (newData);
                    struct iphdr *Niphdr = (struct iphdr *) (newData + sizeof(struct ethhdr));
                    struct tcphdr *Ntcphdr = (struct tcphdr *) (newData + sizeof(struct ethhdr) + (Iiphdr->ihl * 4));
                    unsigned char *Ndata = (unsigned char *) (newData + sizeof(struct ethhdr) + (Iiphdr->ihl * 4) + (tcphdr->doff * 4));

                    // Copy existing.
                    memcpy(Nethhdr, ethhdr, sizeof(struct ethhdr));
                    memcpy(Niphdr, Iiphdr, Iiphdr->ihl * 4);
                    memcpy(Ntcphdr, tcphdr, tcphdr->doff * 4);
                    memcpy(Ndata, data, payloadLength);

                    memcpy(Nethhdr->h_source, srcMac, ETH_ALEN);

                    // Set destination MAC.
                    memcpy(Nethhdr->h_dest, routerMac, ETH_ALEN);
                     
                    // Fill in necessary IP header data.
                    uint32_t oldAddr = Niphdr->saddr;
                    Niphdr->saddr = iphdr->daddr;
                    Niphdr->tot_len = htons((Niphdr->ihl * 4) + (Ntcphdr->doff * 4) + payloadLength);

                    fprintf(stdout, "%d is the Ntcphdr->doff. %d is source port. \n", Ntcphdr->doff, ntohs(Ntcphdr->source));

                    // Recalculate checksums.
                    Niphdr->check = 0;
                    Niphdr->check = ip_fast_csum(Niphdr, Niphdr->ihl);
                    Ntcphdr->check = csum_diff4(oldAddr, Niphdr->saddr, Ntcphdr->check);

                    uint16_t sent;

                    if ((sent = write(stuff->tcpsockfd, newData, ntohs(Niphdr->tot_len) + sizeof(struct ethhdr))) < 0)
                    {
                        perror("sentto");
                    }
                    else
                    {
                        sentPacketCount++;
                    }
                    
                }
                else if (Iiphdr->protocol == IPPROTO_ICMP)
                {
                    // Let's send back the ICMP packet.

                    // Start new data from inner header.
                    struct icmphdr *icmphdr = (struct icmphdr *) (buffer + sizeof(struct ethhdr) + sizeof(struct iphdr) + sizeof(struct iphdr));
                    unsigned char *newData = (unsigned char *)(buffer + sizeof(struct ethhdr) + sizeof(struct iphdr));

                    // Fill in necessary IP header data.
                    Iiphdr->saddr = iphdr->daddr;
                    Iiphdr->tot_len = (Iiphdr->ihl * 4) + sizeof(struct icmphdr);

                    // Recalculate checksums.
                    Iiphdr->check = ip_fast_csum(Iiphdr, Iiphdr->ihl);

                    icmphdr->checksum = 0;
                    icmphdr->checksum = csum_tcpudp_magic(Iiphdr->saddr, Iiphdr->daddr, sizeof(struct icmphdr), IPPROTO_ICMP, csum_partial(icmphdr, sizeof(struct icmphdr), 0));

                    struct sockaddr_in a;
                    a.sin_family = AF_INET;
                    a.sin_addr.s_addr = Iiphdr->daddr;
                    memset(&a.sin_zero, 0, sizeof(a.sin_zero));

                    //fprintf(stdout, "Sending %d bytes to %s\n", Iiphdr->tot_len, inet_ntoa(a.sin_addr));

                    uint16_t sent;

                    if ((sent = sendto(stuff->tcpsockfd, newData, Iiphdr->tot_len, 0, (struct sockaddr *)&a, sizeof(a))) < 0)
                    {
                        perror("sentto");
                    }
                    else
                    {
                        sentPacketCount++;
                    }
                    
                }
            }
        }
    }

    pthread_exit(NULL);
}

int main(int argc, char *argv[])
{
    if (argc < 7)
    {
        fprintf(stderr, "Usage: %s <Listen IP> <Listen Port> <Destination IP> <Nat IP> <Nat Port> <Interface> [<Threads>]\n", argv[0]);

        exit(1);
    }

    time_t startingTime = time(NULL);

    uint8_t threads = get_nprocs_conf();

    if (argc > 7)
    {
        threads = atoi(argv[7]);
    }

    struct ifreq ifr;
    strcpy(ifr.ifr_name, argv[6]);

    int sockfd, udpsockfd, tcpsockfd;

    // Set up receiving socket that processes all packets on a certain interface :)
    sockfd = socket(AF_PACKET, SOCK_RAW, htons(ETH_P_IP));

    if (sockfd == -1)
    {
        fprintf(stderr, "Socket() :: Error - %s\n", strerror(errno));
        perror("socket");

        exit(1);
    }

    if (setsockopt(sockfd, SOL_SOCKET, SO_BINDTODEVICE, argv[6], strlen(argv[6])) < 0)
    {
        fprintf(stderr, "SetSockOpt() :: Error %s\n", strerror(errno));
        perror("setsockopt");

        exit(1);
    }

    if (ioctl(sockfd, SIOCGIFHWADDR, &ifr) != 0)
    {
        perror("ioctl");

        exit(1);
    }

    for (uint8_t i = 0; i < ETH_ALEN; i++)
    {
        memcpy(srcMac + i, ifr.ifr_addr.sa_data + i, 1);
    }

    // Set up UDP sending socket.
    udpsockfd = socket(PF_INET, SOCK_RAW, IPPROTO_UDP);

    if (udpsockfd <= 0)
    {
        fprintf(stderr, "Socket() :: Error with UDP sending socket - %s\n", strerror(errno));
        perror("socket");

        exit(1);
    }

    fprintf(stdout, "Created sending socket (UDP) %" PRIu8 "\n", udpsockfd);

    // Pass our own headers.
    int one = 1;
    if (setsockopt(udpsockfd, IPPROTO_IP, IP_HDRINCL, &one, sizeof(one)) < 0)
    {
        fprintf(stderr, "SetSockOpt() :: Error setting IP Hdr for UDP socket - %s\n", strerror(errno));
        perror("setsockopt");

        exit(1);
    }

    // Gateway addr?
    char *gw = malloc(11);
    GetGatewayIP(gw);
    GetGatewayMAC(gw);

    fprintf(stdout, "Gateway Address => %s\n", gw);
    fprintf(stdout, "Gateway MAC => ");

    for (uint8_t i = 0; i < ETH_ALEN; i++)
    {
        fprintf(stdout, "%02x:", routerMac[i]);
    }

    fprintf(stdout, "\n");
    free(gw);

    // Set up TCP sending socket.
    struct sockaddr_ll a;
    a.sll_family = PF_PACKET;
    a.sll_ifindex = if_nametoindex(argv[6]);
    a.sll_protocol = htons(ETH_P_IP);
    memcpy(a.sll_addr, srcMac, ETH_ALEN);
    a.sll_halen = ETH_ALEN;

    tcpsockfd = socket(AF_PACKET, SOCK_RAW, IPPROTO_RAW);

    if (tcpsockfd < 0)
    {
        fprintf(stderr, "Socket() :: Error with TCP sending socket - %s\n", strerror(errno));
        perror("socket");

        exit(1);
    }

    if (bind(tcpsockfd, (struct sockaddr *)&a, sizeof(a)) < 0)
    {
        fprintf(stderr, "Bind() :: Error (TCP Socket) - %s\n", strerror(errno));
        perror("bind");

        exit(1);
    }

    fprintf(stdout, "Binding to %s:%u and redirecting to %s (%s:%u) with %" PRIu8 " threads and on interface %s.\n\n", argv[1], atoi(argv[2]), argv[3], argv[4], atoi(argv[5]), threads, argv[6]);

    struct stuff stuff;
    stuff.lIP = argv[1];
    stuff.lPort = atoi(argv[2]);
    stuff.dIP = argv[3];
    stuff.nIP = argv[4];
    stuff.dPort = atoi(argv[5]);
    stuff.interface = argv[6];
    stuff.sockfd = sockfd;
    stuff.udpsockfd = udpsockfd;
    stuff.tcpsockfd = tcpsockfd;

    for (uint8_t i = 0; i < threads; i++)
    {
        fprintf(stdout, "Starting thread #%" PRIu8 "\n", i);

        // Create new thread.
        pthread_t pid;

        if (pthread_create(&pid, NULL, threadHndl, (void *)&stuff) != 0)
        {
            fprintf(stderr, "Error creating thread #%lu\n", pid);
        }
    }

    signal(SIGINT, sigHndl);

    while(cont)
    {
        // Allow the program to stay up until signaled to shutdown.
        sleep(1);
    }

    close(sockfd);
    close(udpsockfd);
    close(tcpsockfd);

    time_t stoppingTime = time(NULL);

    time_t timeE = stoppingTime - startingTime;

    fprintf(stdout, "%d received packets and %d sent sockets in %jd seconds\n\n\n", packetCount, sentPacketCount, timeE);

    exit(0);
}

This is probably a very long code block, lol. But hopefully somebody will find it helpful either in GFL or maybe somebody finding this thread on the Search Engine.

Thanks!

**Joshy** · March 16, 2020

I think the best approach is to pull all the functions and run them individually. Force it to pass; force it to fail, and try very odd outlier numbers intending to break it. Can you run it without all of those functions that have exit in it first and see if at least the rest of it works if you have an intended input?

~~Roy~~ · March 17, 2020

On 3/15/2020 at 7:22 PM, Joshy said:

I think the best approach is to pull all the functions and run them individually. Force it to pass; force it to fail, and try very odd outlier numbers intending to break it. Can you run it without all of those functions that have exit in it first and see if at least the rest of it works if you have an intended input?

I will test this out tomorrow hopefully or whenever I have time

Sign In

[C] Unofficial Code To Capture All Packets And Forward Via IPIP. Help Needed?

Recommended Posts

Roy 10,832 / 0

Share this post

Link to post

Share on other sites

Roy 10,832 / 0

Share this post

Link to post

Share on other sites

Joshy 4,364 / 44,927

Share this post

Link to post

Roy 10,832 / 0

Share this post

Link to post

Share on other sites

JGuary55 707 / 14,712

Share this post

Link to post

Roy 10,832 / 0

Share this post

Link to post

Share on other sites

JGuary55 707 / 14,712

Share this post

Link to post

TokenDude 74 / 4,265

Share this post

Link to post

Roy 10,832 / 0

Share this post

Link to post

Share on other sites

Roy 10,832 / 0

Share this post

Link to post

Share on other sites

Joshy 4,364 / 44,927

Share this post

Link to post

Roy 10,832 / 0

Share this post

Link to post

Share on other sites

Joshy 4,364 / 44,927

Share this post

Link to post

Roy 10,832 / 0

Share this post

Link to post

Share on other sites

Recently Browsing 0 members

Latest Topics

Latest Posts on Topics

Recent Milestones

Recent Ranks