Overview

In this article you can find short summary of my network performance research in linux:

• Notes and hints where to look and what to check
• Final sysctl parameters applied

Notes and hints where to look and what to check

Find out how your network works now with iperf3

Run iperf3 on the server

iperf3 -s

Run iperf3 on the client (to check UPLOAD to the server throughput):

iperf3 -c <server IP> -P 4 -t 30 -i 2 -N

Short description of parameters

-c server_ip: Run iperf in client mode, connecting to the specified server IP.
-P 4: Use 4 parallel streams.
-w 1M: Set the socket buffer size to 1 megabyte.
-t 30: Run the test for 30 seconds.
-i 2: Report results every 2 seconds.
-N: Enable the "no delay" option to disable Nagle's algorithm.

Run iperf3 on the client (to check DOWNLOAD from the server throughput):

iperf3 -c <server IP> -P 4 -t 30 -i 2 -N -R

How to find the optimal MTU

Try:

ping -c 5 -M do -s 1472 [server_ip]

periodically increasing mtu at -s parameter to find the maximum

Check congestion control (must be BBR)

# sysctl -a | grep tcp_congestion_control
net.ipv4.tcp_congestion_control = bbr

and set in up if it is not BBR

echo "net.ipv4.tcp_congestion_control=bbr" >> /etc/sysctl.conf
sysctl -p

Final sysctl parameters applied

My current sysctl settings for high network performance

net.ipv4.tcp_max_tw_buckets = 65536
net.ipv4.tcp_tw_reuse = 0
net.ipv4.tcp_max_syn_backlog = 131072
net.ipv4.tcp_syn_retries = 3
net.ipv4.tcp_synack_retries = 3
net.ipv4.tcp_retries1 = 3
net.ipv4.tcp_retries2 = 8
net.ipv4.tcp_rmem = 4096 87380 6291456
net.ipv4.tcp_wmem = 4096 87380 6291456
net.ipv4.tcp_mem = 4096 87380 6291456
net.core.rmem_max = 6291456
net.core.wmem_max = 6291456
net.ipv4.tcp_max_orphans = 65536
net.ipv4.tcp_fin_timeout = 10
net.ipv4.tcp_low_latency = 1
net.ipv4.tcp_syncookies = 0
net.netfilter.nf_conntrack_max = 1048576
net.ipv4.tcp_fastopen = 3

How to calculate maximum rmem and wmem

Calculating optimal max value:
Use the bandwidth-delay product (BDP) formula:
BDP = bandwidth * round-trip-time
Example: For a 1Gbps link with 50ms RTT:
BDP = (1000 Mbps * 0.05s) / 8 = 6.25 MB

ex: net.ipv4.tcp_rmem = 4096 87380 6291456