Steve's Ramblings

FreeBSD Router/Latency Generator: A How-To

The goal was to put a client end box behind some sort of router/gateway device that would introduce latency and if possible bandwidth restrictions and random packet loss.

Research indicated the best option is to use dummynet. Dummynet is a flexible tool originally designed for testing network protocols. It can also be used for bandwidth management (although the owner claims it is misused).

It is a built in feature of FreeBSD so that is the route I took.

The first step is to download FreeBSD which is now version 7. I download the torrent file and used the linux command line client "rtorrent" to pull down the ISOs. I only really needed Disk 1.

The server I found to do this task is a typical older server most IT shops will have laying around. It is a Dell 1750 with a 2.8GHz processor and 1G of RAM. That will be plenty of horsepower to do the routing required.

Boot the new router with FreeBSD Disk 1 and begin the install. When it asked about the partitions I used a single FreeBSD partition (type 165) made of the whole disk. For the boot manager I selected the FreeBSD BootMgr. When it came to file system layout, I used the "auto defaults". It created enough swap and user space for me (even on a 20 Gb drive). When I selected which version to install I used the "developer + ports" version. I made sure not to install any GUI features (X, Gnome, and the like) because this will be a headless server sitting in a data center. Give the primary NIC an address on your LAN and a reasonable name. I named my server "cloud" because it will represent the "Internet Cloud" when it is fully functional. The first time I did the install, I did not do the developer version which includes all of the source tree. When it is time to update the system it will take a lot longer to update if it is not put down on the initial update. Also, be sure to install the ports tree. It will prove to be really helpful.

Once your system is up and running it is time to install some basic packages and update the source tree.

I installed the following packages: bash, cvsup-without-gui, and isc-dhcp3-server. One of the wonderful things about FreeBSD is the ports system. From within /usr/ports you have access to over 18000 third party packages with all dependencies already worked out. To install bash for example you use the following commands: # cd /usr/ports # cd shells/bash # make install clean

That's it. Simple. I changed root's default shell to bash because I am more comfortable working with bash (chsh -s /usr/local/bin/bash). The other two packages I used are found in /usr/ports/net/cvsup-without-gui and /usr/ports/net/isc-dhcp3-server.

The next step is to compile a new kernel and world. This is required because we need to make dummynet a known option of the kernel you are running. Before we even get close to configuring the kernel, it is a good idea to update the source tree. That is why we installed cvsup above.

When cvsup was installed a couple of example cvsup config files were put on the system is /usr/share/example/cvsup. The one we are interested in is "stable-supfile". Copy that file to a working name such as "active-stable-supfile". That way it will not get overwritten with the updates. Edit the new file and change the host=CHANGE_THIS.FreeBSD.org to a real cvsup host. Here is a list of cvsup mirrors. I do recomend not using the Central Servers and using one of the Primary Mirrors. Then you just need to run cvsup. # cvsup -g -L 2 active-stable-supfile

This will probably take quite some time. If you make the mistake I made of not installing the source initially, this will take 2 or 3 hours depending on your network connection. When the cvsup is complete it is time to begin the buildworld process. It is a simple command but will take quite some time. # cd /usr/src # make buildworld

When that is complete you then need to build your custom kernel. Why is it custom? Because we need to add a few options to the kernel configuration file. To make this config file copy the generic config file. I used CLOUD because that is my machine name. # cd /usr/src/sys/i386/conf # cp GENERIC CLOUD

Edit the "CLOUD" file and add these three options. I put them at the end of the options section but above the devices section: options IPFIREWALL options DUMMYNET options HZ=1000

Now you are ready to compile your kernel. It is also pretty simple: # cd /usr/src # make buildkernel KERNCONF=CLOUD

When that is complete, it is time to install your new custom kernel. Simple: # make installkernel KERNCONF=CLOUD

Now you need to reboot into single user mode. I have found the easiest way to do this is to just issue the "reboot" command and sit at console. When the FreeBSD boot menu appears select option "4" which is boot into single user mode.

When you are in single user mode you will want to do the following commands: # mergemaster -p # make installworld # mergemaster # reboot

It sounds simple, but mergemaster can be a bit of a pain. Since this is a brand new system you can almost always choose "i" to install the temporary version of the file. If you had done any custom work prior to this merge you will need to make sure your changes do not get undone.

When you boot to full multi-user and login run this: # uname -a FreeBSD cloud.qa.example.com 7.0-STABLE FreeBSD 7.0-STABLE #2: Tue Jun 17 17:17:47 CDT 2008 [email protected]:/usr/obj/usr/src/sys/CLOUD i386

Now you are ready to make the router. FreeBSD really shines here because you just need to make a couple of config changes.

First, set up your second network adapter. On my 1750 this is bge1. In /etc/rc.conf I added these three lines: ifconfig_bge1="inet 10.250.15.1 netmask 255.255.255.0" gateway_enable="YES" ipnat_enable="YES"

Then I created a file called "ipnat.rules" with these two lines: map bge0 10.250.15.0/24 -> 10.10.10.5/32 portmap tcp/udp 40000:65000 map bge0 10.250.15.0/24 -> 10.10.10.5/32

The "10.10.10.5" address is my network address to the outside world, adjust that as necessary.

Now you need to setup DNS forwarding. Named is already installed on your system you just need to enable and configure it. In /etc/rc.conf add this line: named_enable="YES"

Then edit /etc/namedb/named.conf. Comment out the "listen-on" entry so it looks like this: // listen-on { 127.0.0.1; };

That will enable named to listed for anything on the network. Then un-comment the "forward only" line. You also need to un-comment the forwarders section and enter your upstream DNS server. Mine looks like: forwarders { 10.10.10.71; };

To make it easier to have clients on your new internal network you may want to configure the dhcpd server installed above. The file is located at /usr/local/etc/dhcpd.conf. Mine looks like this: option domain-name "example.com"; option domain-name-servers 10.250.15.1; option subnet-mask 255.255.255.0; default-lease-time 86400; max-lease-time 86400; ddns-update-style none; subnet 10.250.15.0 netmask 255.255.255.0 { range 10.250.15.150 10.250.15.200; option routers 10.250.15.1; }

You also need to enable the dhcpd server by adding this line to your rc.conf file: dhcpd_enable="YES"

There a few other options in rc.conf you should add now because we will need them for dummynet specifically: firewall_enable="YES" firewall_type="OPEN" dummynet_enable="YES"

Now make sure your 2nd NIC is plugged into a hub/switch/vlan segment and reboot.

When your system comes up an ifconfig should reveal that you have both NICs up and the network configured. The command "ipnat list" should show your two NAT rules from above.

Hook up a client box to the internal network segment. Use DHCP and get a lease. When you have a lease, do a nslookup of a known host (like www.cnn.com). Fire up a web browser and make sure you can surf the web. Life should be good at this point.

Now we are at the point of this computing adventure. Time to add latency!

On your client box set up a continuous ping. For windows this would look like: C:> ping -t www.google.com Pinging www.l.google.com [74.125.47.147] with 32 bytes of data: Reply from 74.125.47.147: bytes=32 time=36ms TTL=241 .... Reply from 74.125.47.147: bytes=32 time=37ms TTL=241

Now on your new network device issue these two commands: # ipfw pipe 10 config delay 50 # ipfw add 1000 pipe 10 all from 10.250.15.0/24 to any

Look back at your client box. The ping times should jump 100ms. The 50 from the first command is actually done twice, once on the way out and once on the way back in. You have just added 100ms latency to your connection.

Great, now it is time to remove that latency. These two commands will do it: # ipfw delete 1000 # ipfw delete pipe 10

Poof. Latency gone.

You now have a router with a high cool factor. Why cool? Because you have a lot more control over your internal network than your normal plug-n-play routers.