[[PageOutline]] = Build Lustre Storage Cluster for 3D Fly platform = == 0.Introduction == Some domain-applications usually need to computing with huge data. It usually meet a bottleneck of performance, the main problem is I/O performance to drag total performance. Thereforce, we use Lustre high performance stroage to improve I/O performance.[[BR]] Abiut Lustre, you can refer to Lustre web site http://wiki.lustre.org/ == 1.Machine information & Software == * Hardware ||Node ||8 nodes (1 server , 7 client provide disks)|| ||CPU ||Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz (each node)|| ||Memory ||2GB DDR2 667 (each node)|| ||Disk ||All nodes: 320G * 7 = 2.24T|| ||NIC ||Intel Corporation 82566DM Gigabit Network Connection|| ||Switch ||D-link 24 port GE switch|| * Software ||OS||OpenSuSE 10.2|| ||DRBL||1.9.0-35|| ||Amira||Testing Version|| ||Kernel||2.6.18|| * Architecture [[Image(fly_arch.png)]] == 2.Config OpenSuSE Environment == * Install packages for Lustre * Install packages for DRBL == 3.Install Lustre == * Build kernel {{{ $ sudo su $ cd /usr/src $ wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.18.tar.bz2 $ tar jxvf linux-2.6.18.tar.bz2 $ cd 2.6.18 $ cp /boot/config-xxx .config $ make oldconfig ; make menuconfig $ make -j 5 $ make modules $ make modules_install $ make install $ mkinitrd $ vim /boot/grub/menu.list }}} * recompiler NIC driver {{{ wget http://trac.nchc.org.tw/grid/attachment/wiki/Problem/e1000-7.6.15.4.tar.gz?format=raw tar e1000-7.6.15.4.tar.gz cd e1000-7.6.15.4/src make install }}} * Bonding NIC (bond0 -> eth1 & eth2) {{{ $ cd /etc/sysconfig/network $ vim ifcfg-eth1 $ vim ifeth-eth2 $ vim ifcfg-bond0 $ vim /etc/rc.d/after.local ifenslave bond0 eth1 eth2 modprobe drm modprobe ldiskfs modprobe lnet networks=tcp $ vim /etc/module/modules.conf.local alias bond0 bonding alias eth0 e1000 alias eth1 e1000 alias eth2 e1000 options bond0 mode=6 miimon=100 }}} * Build Lustre Modules {{{ $ configure --with-linux=/usr/src/2.6.18 $ make install $ depmod -a $ vim /etc/rc.d/after.local }}} == 4.Build DRBL Environment == {{{ $ zypper install drbl $ /opt/drbl/sbin/drblsrv -i $ /opt/drbl/sbin/drblpush -i = Check nodes booting setup = /tftpboot/nodes/192.168.1.X/etc/rc.d/after.local /tftpboot/nodes/192.168.1.X/etc/module/modules.conf.local /tftfpood/node_root/etc/sysconfig/network/ }}} == 5.Build Lustre Storage Cluster == * Setup autologin {{{ = fly = mkfs.lustre --fsname=flyfs --mdt --mgs /dev/sdb1 mount -t lustre /dev/sdb1 /home/mdt = fly01 ~ fly07 = mkfs.lustre --fsname=flyfs --ost --mgsnode=fly@tcp0 /dev/sdb mount -t lustre /dev/sdb /home/ost = fly = mkdir /home/flyfs mount -t lustre fly@tcp0:/flyfs /home/flyfs }}} == 6.Install TurboJPEG == rock@fly:~/turbojpeg> wget ftp://ftp.pbone.net/mirror/ftp.sourceforge.net/pub/sourceforge/v/vi/virtualgl/turbojpeg-1.10.3.x86_64.rpm [[BR]] rock@fly:~/turbojpeg> sudo rpm -i turbojpeg-1.10.3.x86_64.rpm [[BR]] rock@fly:~/turbojpeg> sudo rpm -qs turbojpeg [[BR]] {{{ normal /usr/bin/switchtjpeg normal /usr/include/turbojpeg.h normal /usr/lib/libturbojpeg-ipp.so normal /usr/lib/libturbojpeg-libjpeg.so normal /usr/lib/libturbojpeg.so normal /usr/lib64/libturbojpeg-ipp.so normal /usr/lib64/libturbojpeg-libjpeg.so normal /usr/lib64/libturbojpeg.so normal /usr/share/doc/turbojpeg-1.10.3 normal /usr/share/doc/turbojpeg-1.10.3/LGPL.txt normal /usr/share/doc/turbojpeg-1.10.3/LICENSE.txt normal /usr/share/doc/turbojpeg-1.10.3/README.txt }}} rock@fly:~/VirtualGL> switchtjpeg --help [[BR]] {{{ USAGE: /usr/bin/switchtjpeg < ipp | libjpeg > ipp = Use the accelerated version of TurboJPEG, which embeds functions from the Intel(R) Integrated Performance Primitives libjpeg = Use the unaccelerated version of TurboJPEG, which is GPL-compatible }}} == 7.Install VirtualGL == rock@fly:~/VirtualGL> wget ftp://ftp.pbone.net/mirror/ftp.sourceforge.net/pub/sourceforge/v/vi/virtualgl/VirtualGL-2.1.x86_64.rpm [[BR]] rock@fly:~/VirtualGL> sudo rpm -i VirtualGL-2.1.x86_64.rpm [[BR]] rock@fly:~/VirtualGL> sudo rpm -qs VirtualGL {{{ normal /opt/VirtualGL normal /opt/VirtualGL/bin normal /opt/VirtualGL/bin/cpustat normal /opt/VirtualGL/bin/glxinfo normal /opt/VirtualGL/bin/glxspheres normal /opt/VirtualGL/bin/glxspheres64 normal /opt/VirtualGL/bin/nettest normal /opt/VirtualGL/bin/tcbench normal /opt/VirtualGL/bin/vglclient normal /opt/VirtualGL/bin/vglconfig normal /opt/VirtualGL/bin/vglconfig64 normal /opt/VirtualGL/bin/vglconnect normal /opt/VirtualGL/bin/vglgenkey normal /opt/VirtualGL/bin/vgllogin normal /opt/VirtualGL/bin/vglrun normal /opt/VirtualGL/bin/vglserver_config normal /opt/VirtualGL/doc normal /opt/VirtualGL/fakelib normal /opt/VirtualGL/fakelib/64 normal /opt/VirtualGL/fakelib/64/libGL.so normal /opt/VirtualGL/fakelib/libGL.so normal /opt/VirtualGL/lib normal /opt/VirtualGL/lib64 normal /usr/bin/vglclient normal /usr/bin/vglconfig normal /usr/bin/vglconfig64 normal /usr/bin/vglconnect normal /usr/bin/vglgenkey normal /usr/bin/vgllogin normal /usr/bin/vglrun normal /usr/bin/vglserver_config normal /usr/lib/libdlfaker.so normal /usr/lib/libgefaker.so normal /usr/lib/librrfaker.so normal /usr/lib64/libdlfaker.so normal /usr/lib64/libgefaker.so normal /usr/lib64/librrfaker.so normal /usr/share/doc/VirtualGL-2.1 normal /usr/share/doc/VirtualGL-2.1/411.gif normal /usr/share/doc/VirtualGL-2.1/422.gif normal /usr/share/doc/VirtualGL-2.1/444.gif normal /usr/share/doc/VirtualGL-2.1/ChangeLog.txt normal /usr/share/doc/VirtualGL-2.1/LGPL.txt normal /usr/share/doc/VirtualGL-2.1/LICENSE-OpenSSL.txt normal /usr/share/doc/VirtualGL-2.1/LICENSE.txt normal /usr/share/doc/VirtualGL-2.1/chromium-displaywall.png normal /usr/share/doc/VirtualGL-2.1/chromium-sortfirst.png normal /usr/share/doc/VirtualGL-2.1/chromium-sortlast.png normal /usr/share/doc/VirtualGL-2.1/configdialog.gif normal /usr/share/doc/VirtualGL-2.1/exceed1.png normal /usr/share/doc/VirtualGL-2.1/exceed2.png normal /usr/share/doc/VirtualGL-2.1/exceed3.png normal /usr/share/doc/VirtualGL-2.1/exceed6.png normal /usr/share/doc/VirtualGL-2.1/index.html normal /usr/share/doc/VirtualGL-2.1/somerights20.png normal /usr/share/doc/VirtualGL-2.1/sshtunnel.png normal /usr/share/doc/VirtualGL-2.1/sunray.png normal /usr/share/doc/VirtualGL-2.1/turbovnc1.png normal /usr/share/doc/VirtualGL-2.1/turbovnc2.png normal /usr/share/doc/VirtualGL-2.1/turbovnc3.png normal /usr/share/doc/VirtualGL-2.1/turbovnc4.png normal /usr/share/doc/VirtualGL-2.1/vgltransport.png normal /usr/share/doc/VirtualGL-2.1/vgltransportglp.png normal /usr/share/doc/VirtualGL-2.1/vgltransportservernetwork.png normal /usr/share/doc/VirtualGL-2.1/virtualgl.css normal /usr/share/doc/VirtualGL-2.1/x11transport.png normal /usr/share/doc/VirtualGL-2.1/x11transportglp.png }}} rock@fly:~/VirtualGL> vglrun --help {{{ Usage: vglrun [options] [--] [OpenGL app arguments] VirtualGL options -c : proxy = Send 3D images uncompressed using X11 transport jpeg = Compress 3D images using JPEG/send using VGL transport rgb = Encode 3D images as RGB/send using VGL transport sr = Compress 3D images using DPCM/send using Sun Ray transport srrgb = Encode 3D images as RGB/send using Sun Ray transport -cl : = The hostname or IP address of the VirtualGL client (or Sun Ray server) -dl : Force apps that dlopen() libGL to load VirtualGL instead -d : = the X Display or GLP device to use for 3D rendering -fps : Limit client/server frame rate to frames/sec +/-g : Enable/disable gamma correction -gamma : Set gamma correction factor to (see docs) -ge : Fool application into thinking that LD_PRELOAD is unset -np : Use CPUs to perform image compression (VGL transport only) -p

: TCP port to use when connecting to vglclient (VGL transport only) +/-pr : Enable/disable performance profiling output -q : JPEG compression quality [1 <= <= 100] +/-s : Enable/disable SSL encryption of VGL image transport -samp : JPEG chrominance subsampling factor [ = gray, 1x, 2x, 4x, 8x, or 16x] +/-sp : Turn on/off frame spoiling -st : left = Send only left eye buffer right = Send only right eye buffer quad = Use quad-buffered stereo if available, otherwise use red/cyan (anaglyphic) stereo [default] rc = Always use red/cyan (anaglyphic) stereo +/-sync : Enable/disable strict 2D/3D synchronization +/-tr : Enable/disable function call tracing (generates a lot of output) +/-v : Enable/disable verbose VirtualGL messages -64 : (Solaris only) Preload VirtualGL only into 64-bit apps -32 : (Solaris only) Preload VirtualGL only into 32-bit apps }}} == 8.Install TurboVNC == rock@fly:~/turbovnc> wget http://downloads.sourceforge.net/virtualgl/turbovnc-0.4.i386.rpm?modtime=1205803835&big_mirror=0 [[BR]] rock@fly:~/turbovnc> sudo rpm -i turbovnc-0.4.i386.rpm [[BR]] rock@fly:~/turbovnc> rpm -qs turbovnc {{{ normal /etc/X11/applnk/Applications/tvncviewer.desktop normal /etc/init.d/tvncserver normal /etc/sysconfig/tvncservers normal /opt/TurboVNC normal /opt/TurboVNC/bin normal /opt/TurboVNC/bin/Xvnc normal /opt/TurboVNC/bin/vncconnect normal /opt/TurboVNC/bin/vncpasswd normal /opt/TurboVNC/bin/vncserver normal /opt/TurboVNC/bin/vncviewer normal /opt/TurboVNC/man normal /opt/TurboVNC/man/man1 normal /opt/TurboVNC/man/man1/Xvnc.1 normal /opt/TurboVNC/man/man1/vncconnect.1 normal /opt/TurboVNC/man/man1/vncpasswd.1 normal /opt/TurboVNC/man/man1/vncserver.1 normal /opt/TurboVNC/man/man1/vncviewer.1 normal /opt/TurboVNC/vnc normal /opt/TurboVNC/vnc/classes normal /opt/TurboVNC/vnc/classes/AuthPanel.class normal /opt/TurboVNC/vnc/classes/ButtonPanel.class normal /opt/TurboVNC/vnc/classes/CapabilityInfo.class normal /opt/TurboVNC/vnc/classes/CapsContainer.class normal /opt/TurboVNC/vnc/classes/ClipboardFrame.class normal /opt/TurboVNC/vnc/classes/DesCipher.class normal /opt/TurboVNC/vnc/classes/InStream.class normal /opt/TurboVNC/vnc/classes/MemInStream.class normal /opt/TurboVNC/vnc/classes/OptionsFrame.class normal /opt/TurboVNC/vnc/classes/RFBUpdateEvent.class normal /opt/TurboVNC/vnc/classes/RecordingFrame.class normal /opt/TurboVNC/vnc/classes/ReloginPanel.class normal /opt/TurboVNC/vnc/classes/RfbProto.class normal /opt/TurboVNC/vnc/classes/SessionRecorder.class normal /opt/TurboVNC/vnc/classes/SocketFactory.class normal /opt/TurboVNC/vnc/classes/VncCanvas.class normal /opt/TurboVNC/vnc/classes/VncCanvas2.class normal /opt/TurboVNC/vnc/classes/VncViewer.class normal /opt/TurboVNC/vnc/classes/VncViewer.jar normal /opt/TurboVNC/vnc/classes/ZlibInStream.class normal /opt/TurboVNC/vnc/classes/index.vnc normal /usr/share/doc/turbovnc-0.4 normal /usr/share/doc/turbovnc-0.4/LICENCE.TXT normal /usr/share/doc/turbovnc-0.4/TurboVNC-ChangeLog.txt normal /usr/share/doc/turbovnc-0.4/WhatsNew }}} rock@fly:~> sudo zypper install xorg-x11-fonts xorg-x11-fonts-core xorg-x11-fonts-devel [[BR]] rock@fly:~> sudo zypper install xorg-x11-devel xorg-x11-driver-input xorg-x11-driver-video [[BR]] rock@fly:~> sudo zypper install libdrm libdrm-devel Mesa Mesa-devel [[BR]] rock@fly:~> sudo zypper remove beagle beagle-evolution beagle-firefox beagle-gui beagle-index tomboy [[BR]] rock@fly:/usr/X11R6/lib> sudo /etc/init.d/xfs start [[BR]] rock@fly:~> cd /usr/X11R6/lib [[BR]] rock@fly:/usr/X11R6/lib> sudo mkdir X11 [[BR]] rock@fly:~> sudo cp rgb.txt /usr/X11R6/lib/X11/ [[BR]] rock@fly:~> sudo vim /etc/X11/xorg.conf [[BR]] {{{ Section "Module" Load "dbe" Load "type1" Load "freetype" Load "extmod" Load "glx" Load "v4l" Load "drm" Load "dri" }}} {{{ Section "DRI" Group "video" Mode 0666 EndSection }}} rock@fly:~> sudo vim /etc/X11/fs/config [[BR]] {{{ #no-listen = tcp }}} rock@fly:~> /opt/TurboVNC/bin/vncserver -geometry 1280x1024 :1 [[BR]] New 'X' desktop is fly:1 Creating default startup script /home/rock/.vnc/xstartup Starting applications specified in /home/rock/.vnc/xstartup Log file is /home/rock/.vnc/fly:1.log rock@fly:~> ps aux | grep X {{{ root 4216 0.0 0.9 75012 19760 tty7 Ss+ Jul03 0:04 /usr/bin/X :0 -audit 0 -br -auth /var/lib/gdm/:0.Xauth vt7 rock 4811 0.0 0.0 46628 988 ? Ss Jul03 0:00 /usr/bin/ssh-agent /bin/bash /etc/X11/xinit/xinitrc rock 20758 1.1 0.6 16760 12900 pts/9 S 00:56 0:00 /opt/TurboVNC/bin/Xvnc :1 -desktop X -httpd /opt/TurboVNC/bin/../vnc/classes -auth /home/rock/.Xauthority -dontdisconnect -geometry 1240x900 -depth 24 -rfbwait 120000 -rfbauth /home/rock/.vnc/passwd -rfbport 5901 -fp unix/:7100 -deferupdate 1 rock 22008 0.0 0.0 5004 808 pts/9 S+ 00:56 0:00 grep X }}} ### TurboVNC Initialization Bug fixed for ALSA Library ### [[BR]] rock@fly:~/alsa> wget http://gd.tuwien.ac.at/opsys/linux/alsa/lib/alsa-lib-1.0.13.tar.bz2 [[BR]] rock@fly:~/alsa> tar jxvf alsa-lib-1.0.13.tar.bz2 [[BR]] rock@fly:~/alsa> cd alsa-lib-1.0.13 [[BR]] rock@fly:~/alsa/alsa-lib-1.0.13> ./configure --enable-static [[BR]] rock@fly:~/alsa/alsa-lib-1.0.13> make -j 4 [[BR]] rock@fly:~/alsa/alsa-lib-1.0.13> sudo make install [[BR]] openSUSE(GUI)-> Computer -> Control Center -> System -> Sound -> Choose ALSA for each option. [[BR]] == 9.Install Amira == Necessary Library: libg2c.so.0 [[BR]] Download libg2c33(Fortran Compiler Runtime Library) for openSUSE 10.2 [[BR]] rock@fly:~/amira_source/download_lib> wget ftp://ftp.pbone.net/mirror/ftp5.gwdg.de/pub/opensuse/repositories/home:/algraf/openSUSE_10.2/x86_64/libg2c33-3.3.3-43.3.x86_64.rpm [[BR]] rock@fly:~/amira_source/download_lib> sudo rpm -i libg2c33-3.3.3-43.3.x86_64.rpm [[BR]] rock@fly:~/amira_source> chmod a+x Amira411-LinuxAMD64.bin [[BR]] rock@fly:~/amira_source> ./Amira411-LinuxAMD64.bin --console (console mode)[[BR]] Installation succeed !!! [[BR]] Installation path: /home/rock/Amira4.1.1 [[BR]] rock@fly:~/Amira4.1.1/bin> ./start --console (console mode)[[BR]] Finished ! It works... == 10.NVIDIA GeForce 7300 GT Installation == Change to root before following steps. [[BR]] fly:/home/rock/nvidia # wget http://tw.download.nvidia.com/XFree86/Linux-x86_64/173.14.09/NVIDIA-Linux-x86_64-173.14.09-pkg2.run [[BR]] fly:/home/rock/nvidia # sh NVIDIA-Linux-x86_64-173.14.09-pkg2.run [[BR]] Then follow extra steps to continue: Setup , kernel module , configuration ..etc [[BR]] Finished!!! Now, Check it out ~ fly:/home/rock/nvidia # modprobe -l | grep nv [[BR]] {{{ /lib/modules/2.6.18-default/kernel/drivers/scsi/sata_nv.ko /lib/modules/2.6.18-default/kernel/drivers/video/nvidia/nvidiafb.ko /lib/modules/2.6.18-default/kernel/drivers/video/nvidia.ko }}} We only need nvidia.ko, and it'll actually loaded after installation. [[BR]] fly:/home/rock/nvidia # lsmod | grep nv {{{ nvidia 8132208 24 i2c_core 41472 2 nvidia,i2c_i801 }}} fly:/home/rock # glxinfo | more {{{ direct rendering: Yes server glx vendor string: NVIDIA Corporation server glx version string: 1.4 server glx extensions: GLX_EXT_visual_info, GLX_EXT_visual_rating, GLX_SGIX_fbconfig, GLX_SGIX_pbuffer, GLX_SGI_video_sync, GLX_SGI_swap_control, GLX_EXT_texture_from_pixmap, GLX_ARB_multisample, GLX_NV_float_buffer, GLX_ARB_fbconfig_float client glx vendor string: NVIDIA Corporation client glx version string: 1.4 OpenGL vendor string: NVIDIA Corporation OpenGL renderer string: GeForce 7300 GT/PCI/SSE2 OpenGL version string: 2.1.2 NVIDIA 173.14.09 }}} fly:/home/rock/nvidia # vim /etc/X11/xorg.conf [[BR]] {{{ Section "Module" Load "dbe" Load "v4l" Load "freetype" Load "type1" Load "extmod" Load "glx" Load "vnc" # Load "dri" EndSection Section "Device" BoardName "GeForce 7300GT" Driver "nvidia" Identifier "Device[0]" # Option "alwaysshared" # Option "usevnc" "yes" # Option "httpdir" "/usr/share/vnc/classes" # Option "httpport" "5800" # Option "rfbauth" "/root/.vnc/passwd" Screen 0 VendorName "NVIDIA Corporation" EndSection # Section "DRI" # Group "video" # Mode 0666 # EndSection }}} fly:/home/rock/nvidia # less /var/log/Xorg.0.log [[BR]] {{{ (--) PCI:*(1:0:0) nVidia Corporation G70 [GeForce 7300 GT] rev 161, Mem @ 0xfd000000/24, 0xe0000000/ 28, 0xfc000000/24, I/O @ 0xcc00/7, BIOS @ 0xfe8e0000/17 (II) Module glx: vendor="NVIDIA Corporation" compiled for 4.0.2, module version = 1.0.0 Module class: X.Org Server Extension (II) NVIDIA GLX Module 173.14.09 Thu Jun 5 00:05:57 PDT 2008 (II) Loading extension GLX (II) LoadModule: "vnc" (II) Loading /usr/lib64/xorg/modules/extensions//libvnc.so (II) Module vnc: vendor="RealVNC Ltd" compiled for 4.3.99.902, module version = 1.0.0 Module class: X.Org Server Extension ABI class: X.Org Server Extension, version 0.3 (II) Loading extension VNC (II) LoadModule: "nvidia" (II) Loading /usr/lib64/xorg/modules/drivers//nvidia_drv.so (II) Module nvidia: vendor="NVIDIA Corporation" compiled for 4.0.2, module version = 1.0.0 Module class: X.Org Video Driver (II) NVIDIA dlloader X Driver 173.14.09 Wed Jun 4 23:48:23 PDT 2008 (II) NVIDIA Unified Driver for all Supported NVIDIA GPUs (II) Primary Device is: PCI 01:00:0 (--) Assigning device section with no busID to primary device (--) Chipset NVIDIA GPU found (**) NVIDIA(0): Depth 16, (--) framebuffer bpp 16 (==) NVIDIA(0): RGB weight 565 (==) NVIDIA(0): Default visual is TrueColor (==) NVIDIA(0): Using gamma correction (1.0, 1.0, 1.0) (**) NVIDIA(0): Enabling RENDER acceleration (II) NVIDIA(0): NVIDIA GPU GeForce 7300 GT (G73) at PCI:1:0:0 (GPU-0) (--) NVIDIA(0): Memory: 262144 kBytes (--) NVIDIA(0): VideoBIOS: 05.73.22.51.00 (II) NVIDIA(0): Detected PCI Express Link width: 16X (--) NVIDIA(0): Interlaced video modes are supported on this GPU (--) NVIDIA(0): Connected display device(s) on GeForce 7300 GT at PCI:1:0:0: (--) NVIDIA(0): Philips 190B (CRT-0) (--) NVIDIA(0): Philips 190B (CRT-0): 400.0 MHz maximum pixel clock (II) NVIDIA(0): Assigned Display Device: CRT-0 (II) NVIDIA(0): Validated modes: (II) NVIDIA(0): "1024x768" (II) NVIDIA(0): "1024x600" (II) NVIDIA(0): "800x600" (II) NVIDIA(0): "768x576" (II) NVIDIA(0): "640x480" (II) NVIDIA(0): Virtual screen size determined to be 1024 x 768 (--) NVIDIA(0): DPI set to (68, 65); computed from "UseEdidDpi" X config (--) NVIDIA(0): option (==) NVIDIA(0): Disabling 32-bit ARGB GLX visuals. }}} fly:/home/rock/nvidia # /etc/init.d/xdm restart [[BR]] === 10.1 NVIDIA 7300GT Performance === rock@fly:~> glxgears [[BR]] {{{ 37672 frames in 5.0 seconds = 7534.325 FPS 38320 frames in 5.0 seconds = 7663.902 FPS 38317 frames in 5.0 seconds = 7663.308 FPS 38320 frames in 5.0 seconds = 7663.936 FPS 38329 frames in 5.0 seconds = 7665.636 FPS 38316 frames in 5.0 seconds = 7663.152 FPS 38307 frames in 5.0 seconds = 7661.244 FPS 38321 frames in 5.0 seconds = 7664.146 FPS 38308 frames in 5.0 seconds = 7661.435 FPS 38310 frames in 5.0 seconds = 7661.956 FPS 38323 frames in 5.0 seconds = 7664.566 FPS 38311 frames in 5.0 seconds = 7662.112 FPS 38306 frames in 5.0 seconds = 7661.194 FPS 38318 frames in 5.0 seconds = 7663.445 FPS 38320 frames in 5.0 seconds = 7663.884 FPS 38310 frames in 5.0 seconds = 7661.989 FPS 38325 frames in 5.0 seconds = 7664.809 FPS 38317 frames in 5.0 seconds = 7663.257 FPS 38308 frames in 5.0 seconds = 7661.587 FPS 38318 frames in 5.0 seconds = 7663.428 FPS 38305 frames in 5.0 seconds = 7660.958 FPS 38317 frames in 5.0 seconds = 7663.250 FPS 38327 frames in 5.0 seconds = 7665.265 FPS 38313 frames in 5.0 seconds = 7662.488 FPS 38311 frames in 5.0 seconds = 7662.028 FPS 38317 frames in 5.0 seconds = 7663.277 FPS 38313 frames in 5.0 seconds = 7662.439 FPS 38314 frames in 5.0 seconds = 7662.786 FPS 38325 frames in 5.0 seconds = 7664.986 FPS 38311 frames in 5.0 seconds = 7662.035 FPS 38304 frames in 5.0 seconds = 7660.689 FPS 38333 frames in 5.0 seconds = 7666.563 FPS 38309 frames in 5.0 seconds = 7661.684 FPS 38311 frames in 5.0 seconds = 7662.048 FPS 38330 frames in 5.0 seconds = 7665.816 FPS 38316 frames in 5.0 seconds = 7663.065 FPS 38306 frames in 5.0 seconds = 7661.012 FPS 38022 frames in 5.0 seconds = 7604.208 FPS 38030 frames in 5.0 seconds = 7605.954 FPS 38020 frames in 5.0 seconds = 7603.871 FPS 38027 frames in 5.0 seconds = 7605.272 FPS 38023 frames in 5.0 seconds = 7604.425 FPS 38028 frames in 5.0 seconds = 7605.428 FPS 38032 frames in 5.0 seconds = 7606.298 FPS 38008 frames in 5.0 seconds = 7601.419 FPS 38032 frames in 5.0 seconds = 7606.386 FPS 38034 frames in 5.0 seconds = 7606.672 FPS 38029 frames in 5.0 seconds = 7605.690 FPS }}} == Reference == * Lustre, http://wiki.lustre.org/ * Amira, http://www.amiravis.com/ * NVIDIA, http://zh_tw.opensuse.org/NVIDIA * Glxgears, http://wiki.cchtml.com/index.php/Glxgears_is_not_a_Benchmark