[[PageOutline]] = 船舶機械模擬程式加速 = == 1.Roadmap == 1. Perpare DRBL-Kerrighed testbed 2. Config & tun Testbed 3. Choose OpenMP compiler 4. Test Stability of OpenMP Running in DRBL-Krg [[BR]] == 2.Schedule == || '''Task''' ||'''Complete'''||'''Ouput & Evaluation'''|| ||Perpare DRBL-Kerrighed Testbed ||2008-08-06|| 安裝說明文件和測試平台 || ||Config & tun Testbed || || || ||Choose OpenMP Compiler || || || ||Test Stability of CpenMP Running in DRBL-Krg|| || || [[BR]] == 3.Testbed information == * Software ||'''Name'''||'''Version'''|| ||debian ||etch !r3 || ||Kerrighed ||2.3.0 || ||DRBL || || * Compiler ||'''Compiler'''|| '''PATH''' || '''Comments''' || ||Intel gcc ||/opt/intel/cc/10.1.015||http://0rz.tw/8e4AE|| ||Intel Fortran ||/opt/intel/fc/10.1.015||http://0rz.tw/654xd|| * Features of the Intel OpenMP implementation * OpenMP 2.5 conforming * Nested parallelism * Workqueuing extension to OpenMP * Interoperability with POSIX and Windows threads * OMP_DYNAMIC support * Performance analysis via Intel Thread Profiler * Error Checking with the Intel Thread Checker (see below) [[BR]] == 4.Limitation of Running OpenMP in Kerrighed == * Features removed temporarily (Form 2.0.0 version) * KerFS * Distributed threads * Checkpoint * Dynamic streams. * Solution: * If we want to use OpenMP in Kerrighed, I think we must... [[BR]] == 5.Kerrighed modules - Source Code Structure Analysis == === 5.1 Kerrighed 0.70 Totally 7 Modules === * Kerrighed extension of Linux Kernel * Process & load balancing - '''Aragorn''' * Containers - '''Gandalf''' * Synchronization - '''Elrond''' * Ghosts - '''Nazgul''' * Communication - '''Gimli''', '''Gloin''' * Tools - '''lluvatar''' ==== 5.1.1 Probably the Key modules for this project ==== * '''Gandalf module''': Global memory management -> ('''For Container''') * /kerrighed-2.3.0/libs/include/kerrighed_tools.h * /kerrighed-2.3.0/modules/tools/krg_services.h * /kerrighed-2.3.0/modules/mm/mmap_handler.h[[BR]] '''/* Memory Part -> Distributed Shared Memory (DSM) , MPI&OpenMP */''' * '''Aragon module''': Global process management -> ('''For Migration''') * /kerrighed-2.3.0/modules/proc/distant_syscalls.c * /kerrighed-2.3.0/modules/scheduler/mosix_probe.c * /kerrighed-2.3.0/modules/tools/krg_arch.h * /kerrighed-2.3.0/modules/rpc/rpcid.h * /kerrighed-2.3.0/modules/epm/fork_delay.c * /kerrighed-2.3.0/modules/epm/migration.c * /kerrighed-2.3.0/modules/epm/procfs.c[[BR]] '''/* Process & Load balancing Part -> MPI , OpenMP */''' [[BR]] * '''Elrond module''': Synchronization tools for parallel applications -> ('''For Synchronization''') * /kerrighed-2.3.0/modules/epm/ghost_process_api.h[[BR]] '''/* Synchronization Part -> MPI , OpenMP */''' [[BR]] {{{ Elrond provides distributed locks, barriers, semaphores, and wait condition to processes in a implementation that supports process checkpointing as well as migration between nodes. }}} * '''Gimli''': * /kerrighed-2.3.0/libs/libkerrighed/libcomm.c * /kerrighed-2.3.0/libs/include/comm.h * /kerrighed-2.3.0/libs/include/kerrighed_tools.h * /kerrighed-2.3.0/modules/arch/um/krg_um.c * /kerrighed-2.3.0/tools/krg_services.h * /kerrighed-2.3.0/tools/krg_arch.h * /kerrighed-2.3.0/modules/fs/kerfs/dir_io_linker.c * /kerrighed-2.3.0/modules/fs/kerfs/inode_io_linker.c * /kerrighed-2.3.0/modules/fs/kerfs/file_io_linker.c [[BR]] {{{ All these distributed services are based on Gimli for their communications. Gimli is a service providing high performance communication inside the cluster and a complete abstraction of the communication system to kerrighed that makes it portable on various interconnection networks. }}} [[BR]] === 5.2 Kerrighed 2.3.0 Totally 14 Modules === [[Image(KerrighedArch.png)]] || RPC || || || Hotplug || || || KDDM || || || Capability || || || ProcFS || || || Sync || || || IPC || || || FS || || || MM || || || Dynamic Streams || || || Ghost || || || Proc || || || EPM || || || Global Scheduler || || * Load Kerrighed Moudles Messages {{{ $ dmesg Start loading Kerrighed... Init Kerrighed worker(s)...Init Kerrighed low-level framework... Init kerrighed syscall mechanism Kerrighed tools - init module TIPC: Started in network mode TIPC: Own node address <1.1.2>, network identity 1 Try to enable bearer on lo:<5>TIPC: Enabled bearer , discovery domain <1.1.0>, priority 10 ok Try to enable bearer on eth0:<5>TIPC: Enabled bearer , discovery domain <1.1.0>, priority 10 ok Try to enable bearer on eth1:<4>TIPC: Bearer priority adjustment required 10->9 TIPC: Enabled bearer , discovery domain <1.1.0>, priority 9 ok RPC initialisation done Init Kerrighed low-level framework (nodeid 1) : done Init Kerrighed distributed services... KDDM initialisation : start KDDM set init KDDM set init : done KDDM initialisation done KerMM initialisation : start KerMM initialisation done DVFS initialisation : start FAF: initialisation : start FAF: initialisation : done DVFS initialisation done KerIPC initialisation : start KerIPC initialisation done Proc initialisation: start Proc initialisation: done EPM initialisation: start EPM initialisation: done Init Kerrighed distributed services: done Kerrighed... loaded! TIPC: Established link <1.1.2:eth0-1.1.3:eth0> on network plane B Kerrighed is running on 2 nodes }}} [[BR]] === 5.3 Private Variables === * '''mmap''' (map pages of memory - by adding a MAP_LOCAL flag) * /kerrighed-2.3.0/libs/include/kerrighed_tools.h * /kerrighed-2.3.0/modules/tools/krg_services.h * /kerrighed-2.3.0/modules/mm/mman.h {{{ It is possible to ensure that memory will be allocated in the virtual address space of each thread of the process without linking it to a container, thus ensuring that no coherence will be maintained between the different copies of that memory segment across the different threads. }}} [[BR]] === 5.4 OpenMP on Kerrighed === * Standard '''Pthread''' library replaced by '''krgthread''' library proving the Posix interface on Kerrighed Clusters. * Containers to support shared variables in the cluster. * A thread using kerrighed synchronization primitives can migrate and be checkpointed at any time, and except when blocked in the OS. [[BR]] == Reference == * [http://openmp.org/wp/openmp-compilers/ OpenMP] * [http://www.compunity.org/resources/compilers/intel/index.php Intel Compiler] * INRIA: [ftp://ftp.inria.fr/INRIA/publication/publi-pdf/RR/RR-4947.pdf Kerrighed: A SSI Cluster OS Running OpenMP]