Linux Channel Bonding -- 實作合併網卡－joychen0517

在 Linux 上，允許把多個網路介面用一個叫 "bonding" 的 kernel module 以及 Channel bonding interface 來綁成一個 single channel，當然你可以將兩個網卡或更多的網卡綁成一個來用，簡單講就是增加頻寬，達到備援機制～

其實 Channel Bonding 的作法很簡單，只要編輯幾個檔案，再將網路重啟便可，這裡我們的環境是在一台 RHEL5 ES 的機器上要將 eth0 與 eth1 做成 bond0，那麼我們就需要編輯如下的檔案：

（可以在VM裡面測試）

[root@VMLINUX ~]# vi /etc/sysconfig/network-scripts/ifcfg-bond0
DEVICE=bond0
BOOTPROTO=none
ONBOOT=yes
NETWORK=10.15.25.0
NETMASK=255.255.255.0
IPADDR=10.15.25.13
USERCTL=no

P.S. 其中 bond0 的 0 視你的需求而定，這裡可以是 1, 2, 3....隨你高興。
接著編輯要被綁訂的網卡介面，這裡要綁訂的是 eth0 跟 eth1 所以就編輯這兩個檔囉：

[root@VMLINUX ~]# vi /etc/sysconfig/network-scripts/ifcfg-eth0
# Broadcom Corporation NetXtreme II BCM5708 Gigabit Ethernet
DEVICE=eth0
BOOTPROTO=none
HWADDR=00:22:19:50:BC:7E
ONBOOT=yes
MASTER=bond0
SLAVE=yes
USERCTL=no

[root@VMLINUX ~]# vi /etc/sysconfig/network-scripts/ifcfg-eth1
# Broadcom Corporation NetXtreme II BCM5708 Gigabit Ethernet
DEVICE=eth1
BOOTPROTO=none
HWADDR=00:22:19:50:BC:80
ONBOOT=yes
MASTER=bond0
SLAVE=yes
USERCTL=no

[root@VMLINUX ~]# cat /etc/sysconfig/network
NETWORKING=yes
NETWORKING_IPV6=no
HOSTNAME=KHCDNSS01
GATEWAY=10.15.25.254

再來是編輯 /etc/modprobe.conf 檔案：(這個檔案在 RHEL3 是在：/etc/modules.conf 而在 RHEL4 以後的版本則在：/etc/modprobe.conf)

[root@VMLINUX ~]# vi /etc/modprobe.conf
alias eth0 bnx2
alias eth1 bnx2
alias scsi_hostadapter megaraid_sas
alias scsi_hostadapter1 ata_piix
alias bond0 bonding
options bond0 miimon=100

P.S. 這裡要注意一下喔，假如你想綁兩個以上的話，那 option 這一行的後面要再多加一個選項喔：max_bonds=3，這代表我要將三個網卡綁成一個 Channel...像下面這樣：

options bond0 miimon=100 max_bonds=3

接著只要重啟一下網路就行了，在這之前我們先看一下目前的網路狀況：

[root@VMLINUX ~]# ifconfig -a
eth0 Link encap:Ethernet HWaddr 00:22:19:50:BC:7E
inet addr:10.15.25.13 Bcast:10.15.25.255 Mask:255.255.255.0
inet6 addr: fe80::222:19ff:fe50:bc7e/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:48374222 errors:0 dropped:0 overruns:0 frame:0
TX packets:4928117 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:3254586862 (3.0 GiB) TX bytes:561588414 (535.5 MiB)
Interrupt:169 Memory:f8000000-f8012100
eth1 Link encap:Ethernet HWaddr 00:22:19:50:BC:80
inet6 addr: fe80::222:19ff:fe50:bc80/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:2 errors:0 dropped:0 overruns:0 frame:0
TX packets:12 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:128 (128.0 b) TX bytes:3354 (3.2 KiB)
Interrupt:169 Memory:f4000000-f4012100
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:27284 errors:0 dropped:0 overruns:0 frame:0
TX packets:27284 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:19766775 (18.8 MiB) TX bytes:19766775 (18.8 MiB)
sit0 Link encap:IPv6-in-IPv4
NOARP MTU:1480 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)

接著我們就重啟一下網路吧：

[root@VMLINUX ~]# service network restart
Shutting down interface eth0: /etc/sysconfig/network-scripts/ifdown-eth: line 101: /sys/class/net/bond0/bonding/slaves: No such file or directory
[ OK ]
Shutting down interface eth1: /etc/sysconfig/network-scripts/ifdown-eth: line 101: /sys/class/net/bond0/bonding/slaves: No such file or directory
[ OK ]
Shutting down loopback interface: [ OK ]
Bringing up loopback interface: [ OK ]
Bringing up interface bond0: [ OK ]

剛設定完 Channel Bonding 後的第一次重啟看到的這些錯誤訊息是正常的，下次再重啟就不會在看到這些訊息了...我們先看一下 Bonding 的狀態吧：

[root@VMLINUX ~]# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.2.4 (January 28, 2008)
Bonding Mode: load balancing (round-robin)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
Slave Interface: eth0
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:22:19:50:bc:7e
Slave Interface: eth1
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:22:19:50:bc:80

我們順便看一下網路卡的狀況吧：

[root@VMLINUX ~]# mii-tool -v
eth0: negotiated 100baseTx-FD, link ok
product info: vendor 00:08:18, model 54 rev 6
basic mode: autonegotiation enabled
basic status: autonegotiation complete, link ok
capabilities: 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD
advertising: 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD flow-control
link partner: 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD
eth1: negotiated 100baseTx-FD, link ok
product info: vendor 00:08:18, model 54 rev 6
basic mode: autonegotiation enabled
basic status: autonegotiation complete, link ok
capabilities: 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD
advertising: 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD flow-control
link partner: 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD
[root@VMLINUX ~]# ethtool eth0
Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Advertised auto-negotiation: Yes
Speed: 1000Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 1
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: g
Wake-on: d
Link detected: yes

[root@VMLINUX ~]# ethtool eth1
Settings for eth1:
Supported ports: [ TP ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Advertised auto-negotiation: Yes
Speed: 1000Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 1
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: g
Wake-on: d
Link detected: yes

再來我們看一下目前網卡的 IP 為何：是不是多了一個 bond0 的介面了啊？

[root@VMLINUX ~]# ifconfig -a
bond0 Link encap:Ethernet HWaddr 00:22:19:50:BC:7E
inet addr:10.15.25.13 Bcast:10.15.25.255 Mask:255.255.255.0
inet6 addr: fe80::222:19ff:fe50:bc7e/64 Scope:Link
UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
RX packets:42 errors:0 dropped:0 overruns:0 frame:0
TX packets:76 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:8088 (7.8 KiB) TX bytes:20800 (20.3 KiB)
eth0 Link encap:Ethernet HWaddr 00:22:19:50:BC:7E
inet6 addr: fe80::222:19ff:fe50:bc7e/64 Scope:Link
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:23 errors:0 dropped:0 overruns:0 frame:0
TX packets:40 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:4548 (4.4 KiB) TX bytes:10601 (10.3 KiB)
Interrupt:169 Memory:f8000000-f8012100
eth1 Link encap:Ethernet HWaddr 00:22:19:50:BC:7E
inet6 addr: fe80::222:19ff:fe50:bc7e/64 Scope:Link
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:19 errors:0 dropped:0 overruns:0 frame:0
TX packets:36 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:3540 (3.4 KiB) TX bytes:10199 (9.9 KiB)
Interrupt:169 Memory:f4000000-f4012100
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:27284 errors:0 dropped:0 overruns:0 frame:0
TX packets:27284 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:19766775 (18.8 MiB) TX bytes:19766775 (18.8 MiB)
sit0 Link encap:IPv6-in-IPv4
NOARP MTU:1480 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)

接著我們做個小試驗，把 eth0 跟 eth1 的網路線拔插個幾次看看，我發現每次的拔插大概掉一個封包，所以兩條網路線各五次的拔插共掉了 10 個封包，不過 session 倒是不會斷：

[root@VMLINUX ~]# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.2.4 (January 28, 2008)
Bonding Mode: load balancing (round-robin)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
Slave Interface: eth0
MII Status: up
Link Failure Count: 5
Permanent HW addr: 00:22:19:50:bc:7e
Slave Interface: eth1
MII Status: up
Link Failure Count: 5
Permanent HW addr: 00:22:19:50:bc:80