wiki:wade/sr-iov

Version 13 (modified by wade, 13 years ago) (diff)

--

環境

  • 網卡:Intel 82599EB 10 Gigabit
    • module:
      • ixgbe
      • ixgbevf

測試指令

  • 載入網卡 module:
    • 僅載入網卡
      modprob ixgbevf
      modprob ixgbe
      
    • 載入網卡及 8 個 VF
      modprob ixgbe max_vfs=8
      
    • module 設定:
      vim /etc/modprobe.d/ixgbe.conf
      
    • 網卡設定:
      vim /etc/udev/rules.d/70-persistent-net.rules
      
  • 列出 ixgbe 網卡 module:
    lsmod | grep ixgbe
    
  • 列出 82599 網卡及 VF:
    lspci | grep 82599
    lspci | grep Virtual
    
  • 卸載網卡 module:
    • 卸載網卡前,要先檢查 VM 是否已經全部關閉:
      virsh list --all
      
    • 卸載網卡:
      modprob -r ixgbe
      

實驗

  • node 04、05
    • 實體 eth
      • eth2:右邊
      • eht4:左邊

1 實體機器 vs 1 實體機器

  • 設定
    • node04
      • eth2:192.168.100.42
      • eth4
    • node05
      • eth2:192.168.100.52
      • eth4
    • server:
      iperf -s server-ip
      
    • client:
       iperf -c server-ip -t 30 -i 1
      
  • node04,eth02,server → node05,eth02,client
    • 透過 switch、橙色網路線:
      • server
        iperf: ignoring extra argument -- 192.168.100.42
        ------------------------------------------------------------
        Server listening on TCP port 5001
        TCP window size: 85.3 KByte (default)
        ------------------------------------------------------------
        [  4] local 192.168.100.42 port 5001 connected with 192.168.100.52 port 44429
        [ ID] Interval       Transfer     Bandwidth
        [  4]  0.0-30.0 sec  11.8 GBytes  3.39 Gbits/sec
        [  5] local 192.168.100.42 port 5001 connected with 192.168.100.52 port 44430
        [  5]  0.0-30.0 sec  10.5 GBytes  3.01 Gbits/sec
        [  4] local 192.168.100.42 port 5001 connected with 192.168.100.52 port 47920
        [  4]  0.0-30.0 sec  12.0 GBytes  3.43 Gbits/sec
        
      • client:
        ex-01、ex-02、ex-03
        
  • node04,eth02,server → node05,eth02,client
    • 網路卡對接、橙色網路線:
      • server
        iperf: ignoring extra argument -- 192.168.100.42
        ------------------------------------------------------------
        Server listening on TCP port 5001
        TCP window size: 85.3 KByte (default)
        ------------------------------------------------------------
        [  4] local 192.168.100.42 port 5001 connected with 192.168.100.52 port 51644
        [ ID] Interval       Transfer     Bandwidth
        [  4]  0.0-30.0 sec  12.3 GBytes  3.53 Gbits/sec
        [  5] local 192.168.100.42 port 5001 connected with 192.168.100.52 port 51645
        [  5]  0.0-30.0 sec  11.4 GBytes  3.25 Gbits/sec
        [  4] local 192.168.100.42 port 5001 connected with 192.168.100.52 port 51646
        [  4]  0.0-30.0 sec  12.6 GBytes  3.61 Gbits/sec
        
      • client:
        ex-04、ex-05、ex-06
        
  • node04,eth02,server → node05,eth02,client
    • 透過 switch:
      • server
        iperf: ignoring extra argument -- 192.168.100.42
        ------------------------------------------------------------
        Server listening on TCP port 5001
        TCP window size: 85.3 KByte (default)
        ------------------------------------------------------------
        [  4]  0.0-100.0 sec  53.7 GBytes  4.61 Gbits/sec
        [  5] local 192.168.100.42 port 5001 connected with 192.168.100.52 port 55568
        [  5]  0.0-100.0 sec  54.0 GBytes  4.64 Gbits/sec
        [  4] local 192.168.100.42 port 5001 connected with 192.168.100.52 port 41919
        [  4]  0.0-100.0 sec  54.4 GBytes  4.67 Gbits/sec
        
      • client:
        ex-07、ex-08、ex-09
        
  • node04,eth03(VF),server → node05,eth02,client
    • 透過 switch:
      • server
        iperf: ignoring extra argument -- 192.168.100.43
        ------------------------------------------------------------
        Server listening on TCP port 5001
        TCP window size: 85.3 KByte (default)
        ------------------------------------------------------------
        [  4] local 192.168.100.43 port 5001 connected with 192.168.100.52 port 59617
        [ ID] Interval       Transfer     Bandwidth
        [  4]  0.0-30.0 sec  11.5 GBytes  3.31 Gbits/sec
        [  5] local 192.168.100.43 port 5001 connected with 192.168.100.52 port 59618
        [  5]  0.0-30.0 sec  11.5 GBytes  3.28 Gbits/sec
        [  4] local 192.168.100.43 port 5001 connected with 192.168.100.52 port 37373
        [  4]  0.0-30.0 sec  11.6 GBytes  3.32 Gbits/sec
        
      • client:
        ex-10、ex-11、ex-12
        

Reference