hsu@Amath-Client00:~$ more /etc/network/interfaces # This file describes the network interfaces available on your system # and how to activate them. For more information, see interfaces(5). # The loopback network interface auto lo iface lo inet loopback address 127.0.0.1 netmask 255.0.0.0 auto eth0 iface eth0 inet static address 192.168.0.33 netmask 255.255.255.0 gateway 192.168.0.1 auto eth1 iface eth1 inet static address 192.168.1.33 netmask 255.255.255.0 # gateway 192.168.1.1 amd-6:~$ more /etc/network/interfaces auto lo iface lo inet loopback address 127.0.0.1 netmask 255.0.0.0 auto eth0 iface eth0 inet static address 192.168.0.32 netmask 255.255.255.0 gateway 192.168.0.1 auto eth1 iface eth1 inet static address 192.168.1.32 netmask 255.255.255.0 # gateway 192.168.1.1
Download VC6.5 from: vyatta-livecd_VC6.5R1_amd64.iso
hsu@amd-6:/src4/ceph/Router$ dd if=/dev/zero of=MyRouter-Template.img bs=1M count=4000
hsu@amd-6:/src3/KVM/bin$ start-Gparted-6-efs /src4/ceph/Router/MyRouter-Template.img
Gparted:~$ sudo fdisk -l /dev/sdb Disk /dev/sdb: 4194 MB, 4194304000 bytes 255 heads, 63 sectors/track, 509 cylinders, total 8192000 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x000962db Device Boot Start End Blocks Id System /dev/sdb1 2048 999423 498688 83 Linux /dev/sdb2 999424 7143423 3072000 83 Linux /dev/sdb3 7143424 8191999 524288 82 Linux swap / Solaris
After booting, login as "vyatta", passwd "yvatta". When seeing system prompt, type "install system" without the double quotes. When asking for partitioning hard disk, choose skip and specify installing the root on sda2. For everything else, just accept default. Otherwise, vyatta just ignore whatever you specify. When seeing "Done!", at system prompt, type "poweroff"
$ kvm -no-kvm -cdrom vyatta-livecd_VC6.5R1_amd64.iso -hda MyRouter-Template.img -boot d $ poweroff
Check whether installation is successful by booting image. Also, we need
ssh and eth0, eth1 interfaces. Hardcode their MAC addresses:
1c:6f:65:4f:cc:8f
and 1c:6f:65:e5:2f:3d
. The rest
settings should be done via configure command, which is a function defined
in /etc/bash_completion.d/configure
file?
hsu@MyRouter:~$ type configure configure is aliased to `_vyatta_op_run configure' ######################################################################################### # From the typeset -f output in the bash shell, we see the definition of _vyatta_op_run, # apparently, defined in /opt/vyatta/share/vyatta-op/functions/interpreter/vyatta-op-run. ######################################################################################### # _vyatta_op_run () # { # local -i estat; # local tpath=$vyatta_op_templates; # local restore_shopts=$( shopt -p extglob nullglob | tr \\n \; ); # shopt -s extglob nullglob; # _vyatta_op_last_comp=${_vyatta_op_last_comp_init}; # false; # estat=$?; # stty echo 2> /dev/null; # i=1; # declare -a args; # for arg in "$@"; # do # local orig_arg=$arg; # if [[ $arg == "*" ]]; then # arg="*"; # else # arg=($(_vyatta_op_conv_node_path $tpath $arg)); # fi; # if [[ "${arg[1]}" == "ambiguous" ]]; then # echo -ne "\n Ambiguous command: ${args[@]} [$arg]\n" 1>&2; # local -a cmds=($(compgen -d $tpath/$arg)); # _vyatta_op_node_path=$tpath; # local comps=$(_vyatta_op_help $arg ${cmds[@]##*/}); # echo -e "$comps\n" | sed -e 's/^P/ P/'; # eval $restore_shopts; # return 1; # else # if [[ "${arg[1]}" == "invalid" ]]; then # echo -ne "\n Invalid command: ${args[@]} [$arg]\n\n" 1>&2; # eval $restore_shopts; # return 1; # fi; # fi; # if [ -f "$tpath/$arg/node.def" ]; then # tpath+=/$arg; # else # if [ -f $tpath/node.tag/node.def ]; then # tpath+=/node.tag; # else # echo -ne "\n Invalid command: ${args[@]} [$arg]\n\n" 1>&2; # eval $restore_shopts; # return 1; # fi; # fi; # if [[ "$arg" == "node.tag" ]]; then # args[$i]=$orig_arg; # else # args[$i]=$arg; # fi; # let "i+=1"; # done; # local run_cmd=$(_vyatta_op_get_node_def_field $tpath/node.def run); # run_cmd=$(_vyatta_op_conv_run_cmd "$run_cmd"); # local ret=0; # local cmd_regex="^(LESSOPEN=|less|pager|tail|/opt/vyatta/bin/vyatta-tshark-interface-port.pl).*"; # if [ -n "$run_cmd" ]; then # eval $restore_shopts; # if [[ -t 1 && "${args[1]}" == "show" && ! $run_cmd =~ $cmd_regex ]]; then # eval "($run_cmd) | ${VYATTA_PAGER:-cat}"; # else # eval "$run_cmd"; # fi; # else # echo -ne "\n Incomplete command: ${args[@]}\n\n" 1>&2; # eval $restore_shopts; # ret=1; # fi; # return $ret # }
Set eth0, eth1, and ssh so that we have network and can remote login via ssh.
$ kvm -hda MyRouter-Template.img # After login with newly specified passwd $ df $ more /etc/mtab # We found /opt/vyatta/etc/config is mounted in /config directory $ cp /config/config.boot /config/config.boot.orig $ nano /config/config.boot $ diff /config/config.boot /config/config.boot.orig 1,23d0 < interfaces { < ethernet eth0 { < address 192.168.0.2/24 < duplex auto < hw-id 1c:6f:65:4f:cc:8f < smp_affinity auto < speed auto < } < ethernet eth1 { < address 192.168.1.1/24 < duplex auto < hw-id 1c:6f:65:e5:2f:3d < smp_affinity auto < speed auto < } < loopback lo { < } < } < service { < ssh { < port 22 < } < } . . . . .
Also modify the PATH variable in /etc/profile
as follow:
vyatta@MyRouter:~$ diff /etc/profile /etc/profile.orig
7c7 < PATH="/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin:." --- > PATH="/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games"
Add two lines to /etc/rc.local
$ diff /etc/rc.local /etc/rc.local.orig 18,21d17 < < ifconfig eth1 192.168.1.1 < route add default gw 192.168.1.32 <
Poweroff vyatta. At this stage, we move previous vyatta image to bkp and cp new template image to MyRouter.img. And start MyRouter using previous script, we then can have legal interface eth1 and our network is ready and we can remote login via the ssh command. And after login, the next three commands successfully add user hsu, passwd hsu, and let hsu be administrator.
# In amd-6:hsu@amd-6:/src4/ceph/Router $ mv MyRouter.img MyRouter-VC6.4.img $ cp MyRouter-Template.img MyRouter.img # In amd-6:hsu@amd-6:/src4/ceph/bin $ start-MyRouter-13 $ ssh -X vyatta@192.168.1.1 vyatta@MyRouter:~$ configure [edit] vyatta@MyRouter# set system login user hsu [edit] vyatta@MyRouter# set system login user hsu authentication plaintext-password ----[edit] vyatta@MyRouter# set system login user hsu level admin [edit] vyatta@MyRouter# commit [edit] vyatta@MyRouter# save Saving configuration to '/config/config.boot'... Done [edit] vyatta@MyRouter# exit exit
Vyatta must be setup via configure command. It is of no use to hand editing
the related configuration files. It seems, everytime, we boot the vyatta system, the
system setting is read from /opt/vyatta/etc/config/config.boot
and the
results are shown in the /opt/vyatta/config/active
directory.
$ configure # set system host-name MyRouter # set interfaces ethernet eth0 address 192.168.0.2/24 # set interfaces ethernet eth1 address 192.168.1.1/24 ;; Setting eth1 failed # set system gateway-address 192.168.1.1 # set system name-server 168.95.192.1 # set system name-server 168.95.1.1 # show system name-server +name-server 168.95.192.1 +name-server 168.95.1.1 # set system login user hsu # set system login user hsu authentication plaintext-password ---- ;; Setting passwd failed # set system login user hsu level admin # commit # save # exit ;; Cannot exit: configuration modified. ;; Use 'exit discard' to discard the changes and exit. # exit discard
The difference of config.boot file is:
$ diff /config/config.boot /config/config.boot.orig >/tmp/config.boot.diff $ cat /tmp/config.boot.diff 1,23d0 < interfaces { < ethernet eth0 { < address 192.168.0.2/24 < duplex auto < hw-id 1c:6f:65:4f:cc:8f < smp_affinity auto < speed auto < } < ethernet eth1 { < address 192.168.1.1/24 < duplex auto < hw-id 1c:6f:65:e5:2f:3d < smp_affinity auto < speed auto < } < loopback lo { < } < } < service { < ssh { < port 22 < } < } 25,34d1 < config-management { < commit-revisions 20 < } < console { < device ttyS0 { < speed 9600 < } < } < gateway-address 192.168.1.1 < host-name MyRouter 36,42d2 < user hsu { < authentication { < encrypted-password $1$7x/zrIyo$/dnPj2A9RXN0m9u6e5Yup0 < plaintext-password "" < } < level admin < } 45c5 < encrypted-password $1$o96GVKA8$iN/9gfglFjlWxprGsohgn0 --- > encrypted-password "$1$o96GVKA8$iN/9gfglFjlWxprGsohgn0" 50,59d9 < name-server 168.95.192.1 < name-server 168.95.1.1 < ntp { < server 0.vyatta.pool.ntp.org { < } < server 1.vyatta.pool.ntp.org { < } < server 2.vyatta.pool.ntp.org { < } < } 61d10 < auto-sync 1 63,67c12,14 < components main < distribution stable < password "" < url http://packages.vyatta.com/vyatta < username "" --- > distribution "stable" > components "main" > url "http://packages.vyatta.com/vyatta" 80c27,45 < time-zone GMT --- > ntp { > server "0.vyatta.pool.ntp.org" > server "1.vyatta.pool.ntp.org" > server "2.vyatta.pool.ntp.org" > } > console { > device ttyS0 { > speed 9600 > } > } > config-management { > commit-revisions 20 > } > } > interfaces { > loopback lo > ethernet eth0 { > hw-id 52:54:00:12:34:56 > } 82,83d46 < < 85c48 < /* === vyatta-config-version: "webproxy@1:ipsec@4:webgui@1:wanloadbalance@3:conntrack@1:firewall@5:qos@1:dhcp-server@4:cluster@1:system@6:nat@4:conntrack-sync@1:zone-policy@1:config-management@1:dhcp-relay@1:vrrp@1:quagga@2" === */ --- > /* === vyatta-config-version: "zone-policy@1:ipsec@4:config-management@1:wanloadbalance@3:cluster@1:dhcp-relay@1:nat@4:webproxy@1:qos@1:system@6:conntrack@1:conntrack-sync@1:vrrp@1:firewall@5:webgui@1:quagga@2:dhcp-server@4" === */
Note: (12/30/2012) The followings are kept for reference purpose. VC6.5 is rather stable, now!
We choose Vyatta to create our virtual router/gateway. So far so good. Download its iso from vyatta-livecd_VC6.5R1_amd64.iso.
Note: (11/24/2012) Vyatta, a Debian-based software-based virtual router, and claimed to be similar to Juniper JUNOS or Cisco IOS. It has two editions: (1) subscription and (2) open sourced editions. Subscription edition provides web-based management interface, i.e. user friendlier. We take the second approach. Almost all the setups you did based on your Debian experience are in vain, i.e. after rebooting, setups are gone. Only can be done via configure, a command no where to be found. Worst of all, you won't be able to upgrade your software packages using Debian mirror. And you can't install additional packages, such as synaptic and emacs. nano is the only text editor (similar to emacs) available. There is no X GUI interface, everything is based on command line interface (CLI). It dose not offer any upgrade path. You only can reinstall newer version via ISO image. Its documentaion web page: Vyatta Docdl, zip download: VC65.zip
Note: (10/08/2012) MyRouter is OK, now, I think. To test it, bring up Test-Eth1 (on ac00), a VM with only IP 192.168.1.254, edit its /etc/rc.local so that its default gateway is 192.168.1.1, not 192.168.1.33, the second IP address of ac00. Reboot it. And on amd-6, booting MyRouter and ceph-client1. Login Test-Eth1, from it we can successfully login 192.168.0.33 (ac00), 192.168.0.32 (amd-6), 192.168.0.130 (ceph-client1), but not machines on the 140.120 network. But, I think this is OK, since 192.168.1.0/24 is our own private lan. (Originally, Test-Eth1 with 192.168.1.33 default gateway can reach anywhere.)
Note: (10/08/2012) Our router should
route 192.168.1.0/24 subnet to other subnet. For consistency, we use eth1 (if possible
at all, for virtual machines with only one 192.168.1.* IP address, it only has (virtual)
eth0 card,) to connect our 192.168.1.0/24 subnet. For Setting up Kvm with 2 Nics and 2
Taps, you may consult the Kvm with 2 Nics. The correct way
to setup MyRouter is as follows: Notice that the MAC addresses of the two nics
must be the same as the MAC addresses for ethernet eth0 and ethernet eth1 recorded in
the /opt/vyatta/etc/config/config.boot
file.
$ Config-Kvm ../Router/MyRouter.img MyRouter 192.168.1.1 eth1 13 # Edit start-MyRouter-13, start-MyRouter-13-AsDaemon, stop-MyRouter-restore-lan-13 # As follows: $ diff start-MyRouter-13 start-MyRouter-13.orig 17,22d16 < ################################################################################ < sudo tunctl -u hsu -t tap103 < sudo ifconfig tap103 192.168.0.32 netmask 255.255.255.255 up < sudo iptables --table nat -A POSTROUTING --out-interface eth0 -j MASQUERADE < sudo iptables -A FORWARD --in-interface tap103 -j ACCEPT < ################################################################################ 28,32d21 < ################################################################################ < sudo sysctl net.ipv4.conf.tap103.proxy_arp=1 < sudo arp -Ds 192.168.0.2 eth0 pub < sudo route add -host 192.168.0.2 dev tap103 < ################################################################################ 35,37d23 < ################################################################################ < vde_switch -tap tap103 -mod 644 -sock=/src4/ceph/network-3039 -mgmt /src4/ceph/network-3039/vde_switch.mgmt -daemon /dev/null < ################################################################################ 39,43c25 < ################################################################################ < # The MAC addresses for eth0 and eth1 are inscribed in config.boot file, can't be < # changed arbitrarily. < ################################################################################ < kvm -net vde,vlan=0,sock=/src4/ceph/network-3039 -net nic,vlan=0,macaddr=1c:6f:65:4f:cc:8f -net vde,vlan=1,sock=/src4/ceph/network-3049 -net nic,vlan=1,macaddr=1c:6f:65:e5:2f:3d -m 512M -monitor unix:/src4/ceph/network-3049/MonSock,server,nowait -hda ../Router/MyRouter.img & --- > kvm -net vde,vlan=0,sock=/src4/ceph/network-3049 -net nic,vlan=0,macaddr=1c:6f:65:e5:2f:3d -net nic,vlan=0,macaddr=1c:6f:65:4f:cc:8f -m 512M -monitor unix:/src4/ceph/network-3049/MonSock,server,nowait -hda ../Router/MyRouter.img & ############################################################################ # For start-MyRouter-13-AsDaemon script, it is almost identical to start-MyRouter-13, # We only need to pay attention to the "-net" options for the kvm command. And the # eth0 and eth1 MAC addresses are hard-coded in its /opt/vyatta/etc/config/config.boot # file. We also use vlan0 and vlan1 as different (virtual) switches for two different # subnets. It seems OK, now. Surely, we need more testing!! The difference of the # last line in the start-MyRouter-13-AsDaemon and start-MyRouter-13-AsDaemon.orig shell # scripts is kept, the rest differences are the same as above. ############################################################################ < screen -S MyRouter -d -m kvm -net vde,vlan=0,sock=/src4/ceph/network-3039 -net nic,vlan=0,macaddr=1c:6f:65:4f:cc:8f -net vde,vlan=1,sock=/src4/ceph/network-3049 -net nic,vlan=1,macaddr=1c:6f:65:e5:2f:3d -m 512M -monitor unix:/src4/ceph/network-3049/MonSock,server,nowait -curses -hda ../Router/MyRouter.img & --- > screen -S MyRouter -d -m kvm -net vde,vlan=0,sock=/src4/ceph/network-3049 -net nic,vlan=0,macaddr=1c:6f:65:e5:2f:3d -net nic,vlan=0,macaddr=1c:6f:65:4f:cc:8f -m 512M -monitor unix:/src4/ceph/network-3049/MonSock,server,nowait -curses -hda ../Router/MyRouter.img & $ diff stop-MyRouter-restore-lan-13 stop-MyRouter-restore-lan-13.orig 45,47d44 < ################################################################################ < sudo pkill -f "vde_switch -tap tap103 -mod 644 -sock=/src4/ceph/network-3039 -mgmt /src4/ceph/network-3039/vde_switch.mgmt" < ################################################################################ 52,56d48 < ################################################################################ < if [ -S /src4/ceph/network-3039/ctl ]; then rm /src4/ceph/network-3039/ctl; fi < if [ -S /src4/ceph/network-3039/vde_switch.mgmt ]; then rm /src4/ceph/network-3039/vde_switch.mgmt; fi < if [ -d /src4/ceph/network-3039 ]; then rm -rf /src4/ceph/network-3039; fi < ################################################################################ 65,71d56 < ################################################################################ < sudo sysctl net.ipv4.conf.tap103.proxy_arp=0 < sudo ifconfig tap103 192.168.0.32 down < # sudo iptables --table nat -D POSTROUTING --out-interface eth1 -j MASQUERADE < sudo iptables -D FORWARD --in-interface tap103 -j ACCEPT < sudo tunctl -d tap103 < ################################################################################
The mkpartfs command provided by qemu-kvm ends up with "/dev/sda
unrecognized disk label". We can use start-Gparted-6-efs
(in /src3/KVM/bin
) and specify /src4/ceph/Router/MyRouter.img
as its argument and use gparted command to partition /dev/sdb
(1)
First partition 488M, ext2, (2) second partition 3096M, ext4, (3) third partition 512M,
swap. I always got 513M for 3rd partition. Also turn on the boot flag for the first
partition. Apparently, first 1MB is reserved for MBR, not used. I asked for 488M,
only got 487M and the second partition (/) started at sector 999424, the correct offset
to use Config-Kvm shell script. The first partition is totally wasted, but we
need it to get the right offset for Config-Kvm shellscript to be functional.
$ mkdir /src4/ceph/Router $ mv *iso /src4/ceph/Router $ cd /src4/ceph/Router $ qemu-img create MyRouter.img 4G############################################################
Well-Known failure: When booting vyatta for kvm-image installation, we need to use the gparted command to partition the image. When asking for partitioning hard disk, choose gparted:
# When seeing system prompt, type "install system" without the double quotes. # print ;; print info about hard disk. # mkpartfs primary ext2 1 512 ;; in the unit of MBs. # set 1 boot on ;; enable boot option on partition 1. # print # mkpartfs primary ext4 512############################################################
$ kvm -no-kvm -cdrom vyatta-livecd-virt_VC6.4-2012.05.31_amd64.iso -hda MyRouter.img -boot d
When asking for partitioning hard disk, choose skip and specify installing the root on sda2. For everything else, just accept default. Otherwise, vyatta just ignore whatever you specify. When seeing "Done!", at system prompt, type
$ poweroff
Booting image:
$ kvm -hda MyRouter.img
# login via vyatta and enter your new password. $ df $ ls -l $ ls -l / $ ls -l /boot $ more /etc/mtab $ more /etc/fstab $ ls -l /dev/sda1 $ ls -l /opt/vyatta $ ls -l /opt/vyatta/bin $ /sbin/ifconfig -a $ ls -l /etc/network $ more /etc/network/interfaces . . . auto lo iface lo inet loopback $ poweroff
We then create a Router with two NICs
$ cp MyRouter.img MyRouter-Template.img $ kvm -hda MyRouter-Template.img # MyRouter-Template.img is OK, poweroff. It's a Template, we don't use it. # Configure MyRouter.img as described in here. # Need to edit /opt/vyatta/etc/config/config.boot file. The mac addresses of eth0 # and eth1 must match the ones we gave online when booting kvm -hda MyRouter.img.
For ethernet and ssh to work correctly in vyatta router, we use nano to edit its config.boot file:
vyatta@vyatta:~$ ls -l /opt/vyatta/etc/config/config* -rwxrwxr-x 1 root vyattacfg 1624 Sep 24 06:47 /opt/vyatta/etc/config/config.boot -rwxrwxr-x 1 root vyattacfg 1440 Sep 24 02:40 /opt/vyatta/etc/config/config.boot.orig $ diff /opt/vyatta/etc/config/config.boot /opt/vyatta/etc/config/config.boot.orig 41,46d40 < service { < ssh { < port 22 < protocol-version v2 < } < } 50c44 < hw-id 1c:6f:65:4f:cc:8f --- > hw-id 52:54:00:12:34:56 55,60c49,54 < /* ethernet eth2 { */ < /* hw-id 1c:6f:65:a8:8d:0f */ < /* } */ < /* ethernet eth3 { */ < /* hw-id 1c:6f:65:4f:cc:8f */ < /* } */ --- > ethernet eth2 { > hw-id 1c:6f:65:a8:8d:0f > } > ethernet eth3 { > hw-id 1c:6f:65:4f:cc:8f > } # To set correct root passwd $ sudo su root sudo: unable to resolve host vyatta # passwd root # As usual, enter password twice.
It seems we need to set almost everything via configure command. Otherwise, after reboot, we lost all the settings we had done in the previous session.
# Backup what we have done. $ sudo cp /opt/vyatta/etc/config/config.boot /opt/vyatta/etc/config/config.boot-09-24-2012 # PATH=.:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin $ which configure # No where to be found $ configure # Everybody can execute this command? vyatta@vyatta:~$ configure vyatta@vyatta:~$ su vyatta@vyatta# which configure vyatta@vyatta# configure vyatta@vyatta# set system host-name MyRouter [edit] vyatta@vyatta# set interfaces ethernet eth0 address 192.168.0.2/24 sh: 1: Syntax error: "(" unexpected sh: 1: Syntax error: "(" unexpected [edit] vyatta@vyatta# set interfaces ethernet eth0 address 192.168.0.2/24 sh: 1: Syntax error: "(" unexpected sh: 1: Syntax error: "(" unexpected Configuration path: [interfaces ethernet eth0 address 192.168.0.2/24] already exists [edit] vyatta@vyatta# set interfaces ethernet eth1 address 192.168.1.1/24 sh: 1: Syntax error: "(" unexpected sh: 1: Syntax error: "(" unexpected [edit] vyatta@vyatta# set system gateway-address 192.168.1.1 [edit] vyatta@vyatta# set system name-server 168.95.192.1 [edit] vyatta@vyatta# set system name-server 168.95.1.1 [edit] vyatta@vyatta# show system name-server +name-server 168.95.192.1 +name-server 168.95.1.1 [edit] vyatta@vyatta# set system login user hsu [edit] vyatta@vyatta# set system login user hsu authentication plaintext-password ---- [edit] vyatta@vyatta# set system login user hsu level admin sh: 1: Syntax error: "(" unexpected [edit] vyatta@vyatta# commit [ system host-name MyRouter ] sudo: unable to resolve host vyatta [ interfaces ethernet eth1 address 192.168.1.1/24 ] sudo: unable to resolve host MyRouter [ interfaces ethernet eth0 address 192.168.0.2/24 ] sudo: unable to resolve host MyRouter RTNETLINK answers: File exists [ system login ] sudo: unable to resolve host MyRouter [ system name-server 168.95.192.1 ] sudo: unable to resolve host MyRouter [ system name-server 168.95.1.1 ] sudo: unable to resolve host MyRouter Commit failed sudo: unable to resolve host MyRouter sudo: unable to resolve host MyRouter [edit] vyatta@vyatta# save Warning: you have uncommitted changes that will not be saved. sudo: unable to resolve host MyRouter Saving configuration to '/config/config.boot'... Done [edit] vyatta@vyatta# exit Cannot exit: configuration modified. Use 'exit discard' to discard the changes and exit. [edit] vyatta@vyatta# exit discard exit
The final outcome of config.boot
file is as follows:
$ diff /opt/vyatta/etc/config/config.boot /opt/vyatta/etc/config/config.boot.orig 1,24d0 < interfaces { < ethernet eth0 { < address 192.168.0.2/24 < duplex auto < hw-id 1c:6f:65:4f:cc:8f < smp_affinity auto < speed auto < } < ethernet eth1 { < address 192.168.1.1/24 < duplex auto < hw-id 1c:6f:65:e5:2f:3d < smp_affinity auto < speed auto < } < loopback lo { < } < } < service { < ssh { < port 22 < protocol-version v2 < } < } 26,30d1 < config-management { < commit-revisions 20 < } < gateway-address 192.168.1.1 < host-name MyRouter 32,38d2 < user hsu { < authentication { < encrypted-password $1$HqUAvE5Z$xvUSwx7JHivpFMsxi3u6C/ < plaintext-password "" < } < level admin < } 41c5 < encrypted-password $1$w4SHcSLk$EC3uunhRpoMQb0k3MWz4o1 --- > encrypted-password "$1$w4SHcSLk$EC3uunhRpoMQb0k3MWz4o1" 46,52d9 < name-server 168.95.192.1 < name-server 168.95.1.1 < ntp { < server 0.vyatta.pool.ntp.org < server 1.vyatta.pool.ntp.org < server 2.vyatta.pool.ntp.org < } 54d10 < auto-sync 1 56,60c12,14 < components main < distribution stable < password "" < url http://packages.vyatta.com/vyatta < username "" --- > distribution "stable" > components "main" > url "http://packages.vyatta.com/vyatta" 73c27,54 < time-zone Asia/Taipei --- > ntp { > server "0.vyatta.pool.ntp.org" > server "1.vyatta.pool.ntp.org" > server "2.vyatta.pool.ntp.org" > } > console { > device ttyS0 { > speed 9600 > } > } > config-management { > commit-revisions 20 > } > } > interfaces { > loopback lo > ethernet eth0 { > hw-id 52:54:00:12:34:56 > } > ethernet eth1 { > hw-id 1c:6f:65:e5:2f:3d > } > ethernet eth2 { > hw-id 1c:6f:65:a8:8d:0f > } > ethernet eth3 { > hw-id 1c:6f:65:4f:cc:8f > } 76c57 < /* === vyatta-config-version: "nat@4:system@5:webgui@1:cluster@1:conntrack@1:dhcp-relay@1:webproxy@1:config-management@1:conntrack-sync@1:quagga@2:wanloadbalance@3:zone-policy@1:dhcp-server@4:firewall@5:ipsec@3:qos@1:content-inspection@3:vrrp@1" === */ --- > /* === vyatta-config-version: "zone-policy@1:config-management@1:wanloadbalance@3:cluster@1:dhcp-relay@1:nat@4:ipsec@3:webproxy@1:qos@1:content-inspection@3:conntrack@1:conntrack-sync@1:system@5:vrrp@1:firewall@5:webgui@1:quagga@2:dhcp-server@4" === */
Prepare the rc.local file
#!/bin/sh -e # # rc.local # # This script is executed at the end of each multiuser runlevel. # Make sure that the script will "exit 0" on success or any other # value on error. # # In order to enable or disable this script just change the execution # bits. # # By default this script does nothing. # Mount /usr/local and /src2 host file systems to UML and # overwrite /etc/udev/rules.d/70-persistent-net.rules # mount -o ro /dev/sdb11 /src2 # mount -o ro /dev/sdb6 /usr/local ifconfig eth1 192.168.1.1 route add default gw 192.168.1.32 exit 0
After reboot, I got working account hsu, I can ping eth1, I even can remote login 192.168.1.1 (from 192.168.0.2, MyRouter, but not from 192.168.0.32, the amd-6 host), a new subnet. Also from 192.168.1.1, now I can ssh to amd-op. The unresolved problems are: Where is the configure command? There are quite a few syntax error messges during configuration session. We even failed on commit and save commands. Can I trust this configure command?
hsu@MyRouter:~$ ssh -X hsu@192.168.1.1 hsu@192.168.1.1's password: Linux MyRouter 3.0.23-1-amd64-vyatta-virt #1 SMP Fri Mar 23 19:38:13 PDT 2012 x86_64 Welcome to Vyatta. . . . hsu@MyRouter:~$ ping -c 3 192.168.0.2 . . . 64 bytes from 192.168.0.2: icmp_req=3 ttl=64 time=0.049 ms . 3 packets transmitted, 3 received, 0% packet loss, time 1999ms . hsu@MyRouter:~$ ping -c 3 192.168.0.32 . . . 64 bytes from 192.168.0.32: icmp_req=3 ttl=64 time=0.422 ms . 3 packets transmitted, 3 received, 0% packet loss, time 2002ms . hsu@MyRouter:~$ ssh -X hsu@140.120.7.41 . . . Last login: Mon Sep 24 21:43:17 2012 from 122-118-191-184.dynamic.hinet.net hsu@amd-op:~$
For the host 192.168.0.32 (amd-6) to be able to ssh to 192.168.1.1, we add 192.168.0.2 as its gateway to 192.168.1.0 sunnet
$ ssh -X hsu@192.168.1.1 ssh: connect to host 192.168.1.1 port 22: Connection timed out hsu@amd-6:~/inet$ man route hsu@amd-6:~/inet$ route -n # Original routing table Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 192.168.0.1 0.0.0.0 UG 0 0 0 eth0 192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 192.168.0.2 0.0.0.0 255.255.255.255 UH 0 0 0 tap11 ############################################################################ # In start-MyRouter-13* scripts, we add the next two lines for routing tap devices # sudo route add -host 192.168.1.1 dev tap13 # sudo route add -host 192.168.0.2 dev tap103 hsu@amd-6:~/inet$ route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 192.168.0.1 0.0.0.0 UG 0 0 0 eth0 192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 192.168.0.2 0.0.0.0 255.255.255.255 UH 0 0 0 tap103 192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1 192.168.1.1 0.0.0.0 255.255.255.255 UH 0 0 0 tap13 ############################################################################ hsu@amd-6:~/inet$ sudo route add -net 192.168.1.0 netmask 255.255.255.0 gw 192.168.0.2 [sudo] password for hsu: hsu@amd-6:~/inet$ route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 192.168.0.1 0.0.0.0 UG 0 0 0 eth0 192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 192.168.0.2 0.0.0.0 255.255.255.255 UH 0 0 0 tap11 192.168.1.0 192.168.0.2 255.255.255.0 UG 0 0 0 tap11 # The following command delete 192.168.0.2 as a router for 192.168.1.0/24 # $ sudo route del -net 192.168.1.0 netmask 255.255.255.0 gw 192.168.0.2 hsu@amd-6:~/inet$ ssh -X hsu@192.168.1.1 The authenticity of host '192.168.1.1 (192.168.1.1)' can't be established. RSA key fingerprint is 4e:6a:b5:5b:7e:56:af:15:28:3d:5f:e4:1f:f9:ff:90. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '192.168.1.1' (RSA) to the list of known hosts. hsu@192.168.1.1's password: X11 forwarding request failed on channel 0 Linux MyRouter 3.0.23-1-amd64-vyatta-virt #1 SMP Fri Mar 23 19:38:13 PDT 2012 x86_64 Welcome to Vyatta. This system is open-source software. The exact distribution terms for each module comprising the full system are described in the individual files in /usr/share/doc/*/copyright. Last login: Thu Sep 27 09:59:04 2012 from 192.168.1.1 hsu@MyRouter:~$
On another host 192.168.0.33 (ac00), we need to add 192.168.0.2 as its router/gateway to 192.168.1.0 subnet, too.
$ route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 192.168.0.1 0.0.0.0 UG 0 0 0 eth0 192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 hsu@Amath-Client00:/src4/ceph/Doc$ ping -c 3 192.168.1.1 PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data. --- 192.168.1.1 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2017ms hsu@Amath-Client00:/src4/ceph/Doc$ sudo route add -net 192.168.1.0 netmask 255.255.255.0 gw 192.168.0.2 [sudo] password for hsu: hsu@Amath-Client00:/src4/ceph/Doc$ route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 192.168.0.1 0.0.0.0 UG 0 0 0 eth0 192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 192.168.1.0 192.168.0.2 255.255.255.0 UG 0 0 0 eth0 hsu@Amath-Client00:/src4/ceph/Doc$ ping -c 3 192.168.1.1 PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data. 64 bytes from 192.168.1.1: icmp_req=1 ttl=63 time=451 ms 64 bytes from 192.168.1.1: icmp_req=2 ttl=63 time=0.732 ms 64 bytes from 192.168.1.1: icmp_req=3 ttl=63 time=0.744 ms --- 192.168.1.1 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.732/150.871/451.137/212.320 ms # The following command delete 192.168.0.2 as a router for 192.168.1.0/24 # $ sudo route del -net 192.168.1.0 netmask 255.255.255.0 gw 192.168.0.2 hsu@Amath-Client00:/src4/ceph/Doc$ ssh -X hsu@192.168.1.1 hsu@192.168.1.1's password: X11 forwarding request failed on channel 0 Linux MyRouter 3.0.23-1-amd64-vyatta-virt #1 SMP Fri Mar 23 19:38:13 PDT 2012 x86_64 Welcome to Vyatta. This system is open-source software. The exact distribution terms for each module comprising the full system are described in the individual files in /usr/share/doc/*/copyright. Last login: Thu Sep 27 05:24:57 2012 from 192.168.0.33 # We notice that "sudo dpkg-reconfigure tzdata" done this morning (09/27/2012) for # setting time zone lost effect after system rebooting. Need to set time zone via # configure command. hsu@MyRouter:~$ date Thu Sep 27 06:04:02 GMT 2012 hsu@MyRouter:~$ exit logout -bash: /vyatta-monitor-cleanup: No such file or directory Connection to 192.168.1.1 closed.
For ac00 (192.168.0.33) and amd-6 (192.168.0.32) to be able to reach the gateway (virtual) 192.168.1.1, we add 192.168.0.2 (vyatta virtual router) as the gateway for 192.168.1.0/24 subnet. But, this will route 192.168.1.32 via 192.168.0.2. The two subnets are no longer independent! By the way, without vyatta router, we can remote login amd6 from ac00 via 192.168.1.32 and then from amd-6 to amd-op. So, we did't gain anything by providing gateway for 192.168.1.0/24 subnet. Or, I did create a malfunctioning router? The following command helps us analyze packet traffics: tcpdump advanced filters
hsu@Amath-Client00:/src4/ceph/Doc$ ssh -X hsu@192.168.1.32 hsu@192.168.1.32's password: . . . Last login: Sat Oct 6 13:54:14 2012 from amath-client00 amd-6:~$ xs amd-op hsu@amd-op's password: . . . Last login: Sat Oct 6 13:52:02 2012 from 122-118-176-243.dynamic.hinet.net hsu@amd-op:~$
Unix based Open Source firewall router that sees itself in competition with Cisco. The firewall module is not on by default, so once interfaces are defined it will pass traffic freely. All configuration is done via the CLI [1].
Contents[hide] |
root / vyatta
install-system
vyatta / vyatta
configure
set system host-name router-name
set interfaces ethernet eth0 address 192.168.1.10/24
(repeat for other interfaces)
set system gateway-address 192.168.10.1
set service ssh
set system time-zone Europe/London
set system ntp server 192.168.1.50
delete system ntp server 0.vyatta.pool.ntp.org
delete system ntp server 1.vyatta.pool.ntp.org
delete system ntp server 2.vyatta.pool.ntp.org
commit
save
exit
set system name-server 172.16.0.34
set system domain-search domain mydomain.com
show system name-server
show system domain-search
To set up the SNMP service and allow a polling server to interact with (for example perform SNMP Gets, etc) with the router
configure
set service snmp community public
edit service snmp community mp-public
set client 192.168.1.25
commit
top
To set up SNMP trapping (assumes you're already in config mode)
set service snmp trap-target 192.168.10.199
commit
In similar fashion to Cisco IOS, configuration has to be entered in config mode, and the configuration can only be viewed in Operator mode.
Command | Description | |
---|---|---|
configure |
Enter configuration mode | |
commit |
Apply changes made since last commit | |
save |
Save changes since last save (otherwise lost at reboot) | |
discard |
Discard changes made since last commit |
Command | Description | |
---|---|---|
set system login user <user> |
Create user | |
set system login user <user> authentication plaintext-password <password> |
Change users password | |
set system login user <user> level admin |
Change users authorisation level | |
delete system login user <user> |
Delete user |
If an interface has no firewall config, then it passes all traffic. Once any firewall config is applied then that interface acts as a firewall.
Command | Description | |
---|---|---|
delete firewall name home_in rule 5 |
Delete firewall rule no 5 |
vyatta@vyatta:~$ configure vyatta@vyatta# set firewall name home_out rule 10 action accept vyatta@vyatta# set firewall name home_out rule 10 description "VC client access" vyatta@vyatta# set firewall name home_out rule 10 destination address 10.1.1.5 vyatta@vyatta# set firewall name home_out rule 10 destination port 80,443,8084,8443,9084 vyatta@vyatta# set firewall name home_out rule 10 protocol tcp vyatta@vyatta# set interfaces ethernet eth0 firewall in name home_out vyatta@vyatta# commit vyatta@vyatta# exit vyatta@vyatta:~$ show firewall home_out Active on (eth0,IN) State Codes: E - Established, I - Invalid, N - New, R - Related rule action source destination proto state ---- ------ ------ ----------- ----- ----- 10 ACCEPT 0.0.0.0/0 10.1.1.5 tcp any dst ports: 80,443,8084,8443,9084 1025 DROP 0.0.0.0/0 0.0.0.0/0 all any
The firewall filters in both directions, in and out of the interface. Traffic has to be configured in both directions to work! Therefore its necessary to config as shown...
vyatta@vyatta:~$ show firewall home_in Active on (eth0,OUT) State Codes: E - Established, I - Invalid, N - New, R - Related rule action source destination proto state ---- ------ ------ ----------- ----- ----- 5 ACCEPT 0.0.0.0/0 0.0.0.0/0 icmp any 10 ACCEPT 0.0.0.0/0 88.221.188.7 tcp any dst ports: 443 11 ACCEPT 0.0.0.0/0 92.123.36.7 tcp any dst ports: 443 1025 DROP 0.0.0.0/0 0.0.0.0/0 all any vyatta@vyatta:~$ show firewall home_out Active on (eth0,IN) State Codes: E - Established, I - Invalid, N - New, R - Related rule action source destination proto state ---- ------ ------ ----------- ----- ----- 5 ACCEPT 0.0.0.0/0 0.0.0.0/0 icmp any 110 ACCEPT 88.221.188.7/32 0.0.0.0/0 tcp any src ports: 443 111 ACCEPT 92.123.36.7 0.0.0.0/0 tcp any src ports: 443 1025 DROP 0.0.0.0/0 0.0.0.0/0 all any
For full details see http://www.vyatta.com/sites/vyatta.com/files/pdfs/Vyatta_NATRef_R6.2_v01.pdf or Vyatta_NATRe-f_R6.2_v0-1.pdf
To allow masquerade NAT, out through eth0, from multiple inside addresses out through the router's outside interface address
set service nat rule 10 type masquerade set service nat rule 10 source address 10.1.1.0/24 set service nat rule 10 outbound-interface eth0 commit
To allow destination NAT, into the router, presenting machines on the inside, to the outside world
set interfaces ethernet eth0 address 192.168.1.20/24 set service nat set service nat rule 20 description InsideServerName set service nat rule 20 type destination set service nat rule 20 inbound-interface eth0 set service nat rule 20 destination address 192.168.1.20 set service nat rule 20 inside-address address 10.1.1.20 set service nat rule 20 protocol all
To allow source NAT, going out from the router, so that machines on the inside, pick up external addresses as they communicate to the outside world
set service nat set service nat rule 1020 description InsideServerName set service nat rule 1020 type source set service nat rule 1020 inbound-interface eth0 set service nat rule 1020 source address 10.1.1.20 set service nat rule 1020 outside-address address 192.168.1.20 set service nat rule 1020 protocol all
To configure bidirectional NAT, whereby machines on the inside appear to the outside world by the same address for traffic initiated from either outside or inside the router just combine both source and destination NAT configuration.
To delete a NAT rule, use syntax similar to the following...
delete service nat rule 42
Command | Comments | |
---|---|---|
reboot |
||
shutdown |
||
show arp |
||
show interfaces |
||
show ip route |
||
show nat rules |
||
show configuration |
||
set system flow-accounting interface eth0 |
Enable flow accounting | |
delete system flow-accounting |
Disable flow accounting | |
show flow-accounting interface eth0 |
Show flow accounting for eth0
| |
show flow-accounting interface eth0 host 10.1.1.1 |
Show flow accounting for specific IP through eth0
|
When trying to commit interface changes you receive an error like...
admin@router# commit [ interfaces ethernet eth1 address ] RTNETLINK answers: File exists Commit failed
A show interfaces
shows the config to be correct, but a show
configuration
shows that the config hasn't been saved properly.
To resolve - restart the router (reboot
).
On restart the conflicted part of the configuration is lost (so can no longer be
seen via show interfaces
). However it can be reapplied, and shouldn't
generate an error this time around.
TCPdump can only be run as root, therefore you may need to set the root password first...
sudo passwd root
...then you'll be elevate to be the super user, from where you can run TCPdump...
su -
Some basic TCPdump examples...
Command | Comments | |
---|---|---|
tcpdump -i eth0 port 80 |
Anything on port 80 through eth0 | |
tcpdump -i eth0 dst 10.10.0.10 and port 80 |
Anything going to 10.10.0.10:80 through eth0 | |
tcpdump -w capture.pcap -i eth0 port 80 |
Write capture to capture.pcap |
For more info see the TCPdump man page - http://www.tcpdump.org/tcpdump_man.html
I find myself using the Vyatta virtual router (http://www.vyatta.org) for most everytime I need a router. It hasn't yet replaced my core enterprise routers, but it fits in nicely for smaller environments. This example is going to be a basic home configuration - The internet interface receives its address via DHCP, the internal interface is static at 10.0.0.1/24 and provides DHCP, DNS, and Proxy services. Additionally, an internal web server is published via HTTPS.
NOTE: I am using Vyatta version 6.4 which changed some of the
configuration commands. Confirm the version you are running to ensure
the commands are appropriate
Deploying the router is probably the easiest step to perform, especially if you are running VMware. If your running VMware, simply go to http://www.vyatta.com/downloads/vmware_ovf.php to get the link for the latest OVF available. Import this into VMware and your good to go.
If your running some other virtualization stack (Hyper-V, Xen, etc...), you will need to install from ISO. The latest stable version can be found at http://packages.vyatta.com/vyatta/iso/stable/, also Various Isos, just download the LiveCD, create a VM with 512MB RAM, 4GB disk, 2 NICs, and boot from the ISO. NOTE: Be careful the type of NIC chosen as not all adapters are supported by Vyatta. For Hyper-V, you have to use the Legacy Network Adapter. The default adapter type will not work
Once booted, log into the console with username/password of vyatta/vyatta. At the prompt type the following command, accept the default options, allow the install to overwrite the disk, and set the password.
$ install system
When installed, type the following command, and remove the ISO. Power back up and your up and running.
$ poweroff
NOTE: I find it a good step to write down the MAC addresses of the interfaces so I can easily determine which is internal and which is external.
Log into the console as the vyatta user and enter configuration mode by typing
$ configure
The first step needed is to determine which interface is which. We know that we will have 1 interface on the open internet, and the other interface on the trusted network - we obviously dont want to get these backwards.
While in configuration mode, type the following command and you will see something similar to below.
show interfaces
vyatta@vyatta# show interfaces ethernet eth0 { hw-id 00:15:5d:14:ed:2e } ethernet eth1 { hw-id 00:15:5d:14:ed:2f } loopback lo { } |
The router sees the interfaces as eth0 and eth1 and
provides the associated MAC addresses. Using the MAC addresses of the
interfaces, I can determine which interface is which, and even move them
based on need. In my case, eth0 is the external interface
Since our external interface will be receiving its IP address from our ISP, we configure it to use DHCP. To configure eth0 for DHCP, simply type
set interfaces ethernet eth0 address dhcp
Our internal network is owned/managed by us, so we can choose to use a private addressing scheme for our systems. To configure eth1 for a static address, simply type
set interfaces ethernet eth1 address 10.0.0.1/24
Whenever you make a change to the Vyatta configuration, it doesn't take effect until you commit them. Additionally, the changes aren't resilient (don't remain after reboot) until you save them.
To commit and save the changes, type
commit save
We want to give our router a descriptive name as well as create an internal domain name. In this case I am naming it intRtr for internet router, and giving it a domain of goad.local. This gives me a unique name and domain to identify the router and other systems.
set system host-name intRtr set system domain-name goad.local
Next we configure the DHCP server on the router. This involves creating a pool of addresses for DHCP to use, configuring the default gateway, DNS server and domain name.
set service dhcp-server shared-network-name ETH1_POOL subnet 10.0.0.0/24 start 10.0.0.65 stop 10.0.0.199 set service dhcp-server shared-network-name ETH1_POOL subnet 10.0.0.0/24 default-router 10.0.0.1 set service dhcp-server shared-network-name ETH1_POOL subnet 10.0.0.0/24 dns-server 10.0.0.1 set service dhcp-server shared-network-name ETH1_POOL subnet 10.0.0.0/24 domain-name goad.local set service dhcp-server shared-network-name ETH1_POOL authoritative enable
Now that clients have DHCP addresses, it is time to configure the DNS server. In this case we are creating a caching DNS server that receives requests, forwards them to the external DNS server, and caches them for future reference. This speeds up recurring requests, as well as contains the configuration for easy management.
set service dns forwarding dhcp eth0 set service dns forwarding listen-on eth1
Now we set the outbound proxy
set service webproxy listen-address 10.0.0.1 set service webproxy listen-address 10.0.0.1 disable-transparent
NOTE: This means that clients will have to configure their browsers as http://10.0.0.1:3128 to utilize the proxy
For anything other that web traffic (or web traffic we don't want to proxy), we enable Network Address Translation.
set nat source rule 10 source address 10.0.0.0/24 set nat source rule 10 outbound-interface eth0 set nat source rule 10 translation address masquerade
Finally, we want to publish the web server so that when someone browses to port 443 on the external interface, it is forwarded internally.
set nat destination rule 200 destination port https set nat destination rule 200 inbound-interface eth0 set nat destination rule 200 translation address 10.0.0.2 set nat destination rule 200 translation port https set nat destination rule 200 protocol tcp set interfaces ethernet eth0 firewall in name FROM-EXTERNAL set firewall name FROM-EXTERNAL description "Block Unwanted Internet Traffic" set firewall name FROM-EXTERNAL rule 10 description "Accept Established-Related Connections" set firewall name FROM-EXTERNAL rule 10 action accept set firewall name FROM-EXTERNAL rule 10 state established enable set firewall name FROM-EXTERNAL rule 10 state related enable set firewall name FROM-EXTERNAL rule 20 description "Allow https" set firewall name FROM-EXTERNAL rule 20 action accept set firewall name FROM-EXTERNAL rule 20 destination address 10.0.0.2 set firewall name FROM-EXTERNAL rule 20 destination port https set firewall name FROM-EXTERNAL rule 20 protocol tcp
Thats it,
commit saveand your golden