Vyos Policy Based Routing

• What is Policy Based Routing?
• Vyos

What is Policy Based Routing?

Normal ip packet routing only looks at the destination ip (v4 or v6) address to decide which outbound interface should send that packet. Policy based routing allows that interface selection process to use information other than the destination ip address. For example, you might want all TCP packets with destination port number 80 to exit via a particular interface.

In this particular case, we have two interfaces running a full BGP (IPv4) feed from a provider, and we also have a higher-speed consumer grade DSL line configured via DHCP. As long as the DSL link is up, we can use it for web browsing. If the link is down, we want to automatically fall back to the main data links.

Vyos

We add some stock Vyos commands to setup the policy based routing. Two interfaces and their BGP configuration are omitted here to simplify this, and to concentrate on the policy based issues.

The bulk-data port-group defines the TCP ports that we want to route over the DSL link. The source-route policy selects the packets to be routed via table 1. That policy is applied to the local ethernet interface, not the outgoing DSL link. The static protocol section defines the content of routing table 1, which forces those packets out thru the DSL link, rather than via the default BGP configured links.

firewall {
    group {
        port-group bulk-data {
            port 22
            port 143
            port 443
            port 8000
            port 80
        }
    }
}

policy {
    route source-route {
        rule 10 {
            destination {
                group {
                    port-group bulk-data
                }
            }
            protocol tcp
            set {
                table 1
            }
            source {
                address 0.0.0.0/0
            }
        }
    }
}

interfaces {
    ethernet eth0 {
        description "local network"
        policy {
            route source-route
        }
    }
    ethernet eth1 {
        address dhcp
        description "dsl link"
    }
}

protocols {
    static {
        table 1 {
            interface-route 0.0.0.0/0 {
                next-hop-interface eth1 {
                }
            }
        }
    }
}

This all works as advertised, with the small exception of DHCP renewals. There are times when the DSL link may die (as in no traffic), but the Vyos router won't notice that until it tries to do a DHCP renewal many hours later. So we monitor the next hop by periodic pings, and when the link state changes, we force a 'renew dhcp interface eth1' command. If that renewal attempt fails, the default route in table 1 is removed, even though the Vyos configuration thinks it is still there. When the link comes back up, and the DHCP renewal succeeds, we need to reinstate that default route in table 1.

We also need to ensure that traffic from the vyos itself is properly natted out over that dhcp configured eth1. This is required to allow outside systems (IPv6 tunnelbroker.net) to ping the local IPv4 endpoint.

The following script in /etc/dhcp3/dhclient-exit-hooks.d/bgp handles those issues.

#!/bin/bash

(
if [ -z "$reason" ]; then
    reason="NONE"
fi

case $reason in
    BOUND|RENEW|REBIND|REBOOT|NONE|EXPIRE|FAIL|RELEASE|STOP)

    # check if dhcp to verizon is up
    #
    dev=eth1
    table=1
    cmd="show ip route 0.0.0.0/0
         exit
        "
    gwip=$( (echo "$cmd" | ssh -t -t vyos@localhost 2>/dev/null) | grep "via $dev" | rev | awk '{print $3}' | rev | cut -d, -f1 )
    if [ "$gwip" != "" ]; then
        # link is up, replace the mangle table
        ip=$(ip addr list $dev | grep 'inet ' | awk '{print $2}' | cut -d/ -f1)
        iptables -t mangle -F OUTPUT
        # vyos source eth1 natted out eth1
        iptables -t mangle -A OUTPUT -s $ip/32 -j MARK --set-mark 0x80000000
        iptables -n -t mangle -L OUTPUT

        # link is up, check if table $table contains default route
        echo "dhcp $dev link is up at $(date)"
        cmd="ip route list table $table
             exit
             exit
        "
        ok=$( (echo "$cmd" | ssh -t -t vyos@localhost 2>/dev/null) | grep "default dev $dev")
        if [ "$ok" == "" ]; then
            # table $table does not contain the default route
            echo "route table $table has no default route at $(date)"
            cmd="configure
                 delete protocols static table $table
                 commit
                 set protocols static table $table interface-route 0.0.0.0/0 next-hop-interface $dev
                 commit
                 save
                 exit
                 exit
                "
            echo "$cmd" | ssh -t -t vyos@localhost 2>/dev/null
            echo "remove/re-add protocols static table $table interface-route 0.0.0.0/0 next-hop-interface $dev $(date)"
        fi
    fi
    ;;

    PREINIT)
    ;;
esac

) </dev/null >>/tmp/xxx 2>&1