船舶機械模擬程式加速

1.Roadmap

1. Perpare DRBL-Kerrighed testbed
2. Config & tun Testbed
3. Choose OpenMP compiler
4. Test Stability of OpenMP Running in DRBL-Krg

2.Schedule

Task	Complete	Ouput & Evaluation
Perpare DRBL-Kerrighed Testbed	2008-08-06	安裝說明文件和測試平台
Config & tun Testbed
Choose OpenMP Compiler
Test Stability of CpenMP Running in DRBL-Krg

3.Testbed information

• Software

Name	Version
debian	etch r3
Kerrighed	2.3.0
DRBL

• Compiler

Compiler	PATH	Comments
Intel gcc	/opt/intel/cc/10.1.015	​http://0rz.tw/8e4AE
Intel Fortran	/opt/intel/fc/10.1.015	​http://0rz.tw/654xd

• Features of the Intel OpenMP implementation
  • OpenMP 2.5 conforming
  • Nested parallelism
  • Workqueuing extension to OpenMP
  • Interoperability with POSIX and Windows threads
  • OMP_DYNAMIC support
  • Performance analysis via Intel Thread Profiler
  • Error Checking with the Intel Thread Checker (see below)

4.Limitation of Running OpenMP in Kerrighed

• Features removed temporarily (Form 2.0.0 version)
  • KerFS
  • Distributed threads
  • Checkpoint
  • Dynamic streams.
• Solution:
  • If we want to use OpenMP in Kerrighed, I think we must...

5.Kerrighed modules - Source Code Structure Analysis

5.1 Totally 7 modules

• Kerrighed extension of Linux Kernel
  • Process & load balancing - Aragorn
  • Containers - Gandalf
  • Synchronization - Elrond
  • Ghosts - Nazgul
  • Communication - Gimli, Gloin
  • Tools - lluvatar

5.1.1 Probably the Key modules for this project

• Gandalf module: Global memory management -> (For Container)
  • /kerrighed-2.3.0/libs/include/kerrighed_tools.h
  • /kerrighed-2.3.0/modules/tools/krg_services.h
  • /kerrighed-2.3.0/modules/mm/mmap_handler.h
    

/* Memory Part -> Distributed Shared Memory (DSM) , MPI&OpenMP */

• Aragon module: Global process management -> (For Migration)
  • /kerrighed-2.3.0/modules/proc/distant_syscalls.c
  • /kerrighed-2.3.0/modules/scheduler/mosix_probe.c
  • /kerrighed-2.3.0/modules/tools/krg_arch.h
  • /kerrighed-2.3.0/modules/rpc/rpcid.h
  • /kerrighed-2.3.0/modules/epm/fork_delay.c
  • /kerrighed-2.3.0/modules/epm/migration.c
  • /kerrighed-2.3.0/modules/epm/procfs.c
    

/* Process & Load balancing Part -> MPI , OpenMP */

• Elrond module: Synchronization tools for parallel applications -> (For Synchronization)
  • /kerrighed-2.3.0/modules/epm/ghost_process_api.h
    

/* Synchronization Part -> MPI , OpenMP */

  Elrond provides distributed locks, barriers, semaphores, and wait condition to processes in a implementation that    
  supports process checkpointing as well as migration between nodes.

• Gimli:
  • /kerrighed-2.3.0/libs/libkerrighed/libcomm.c
  • /kerrighed-2.3.0/libs/include/comm.h
  • /kerrighed-2.3.0/libs/include/kerrighed_tools.h
  • /kerrighed-2.3.0/modules/arch/um/krg_um.c
  • /kerrighed-2.3.0/tools/krg_services.h
  • /kerrighed-2.3.0/tools/krg_arch.h
  • /kerrighed-2.3.0/modules/fs/kerfs/dir_io_linker.c
  • /kerrighed-2.3.0/modules/fs/kerfs/inode_io_linker.c
  • /kerrighed-2.3.0/modules/fs/kerfs/file_io_linker.c
    

   All these distributed services are based on Gimli for their communications. Gimli is a service providing high 
   performance communication inside the cluster and a complete abstraction of the communication system to kerrighed that 
   makes it portable on various interconnection networks.

5.2 Private Variables

• mmap (map pages of memory - by adding a MAP_LOCAL flag)
  • /kerrighed-2.3.0/libs/include/kerrighed_tools.h
  • /kerrighed-2.3.0/modules/tools/krg_services.h
  • /kerrighed-2.3.0/modules/mm/mman.h

   It is possible to ensure that memory will be allocated in the virtual address space of each thread of the process 
   without linking it to a container, thus ensuring that no coherence will be maintained between the different copies of  
   that memory segment across the different threads.

5.3 OpenMP on Kerrighed

• Standard Pthread library replaced by krgthread library proving the Posix interface on Kerrighed Clusters.
• Containers to support shared variables in the cluster.
• A thread using kerrighed synchronization primitives can migrate and be checkpointed at any time, and except when blocked in the OS.

Reference

• ​OpenMP
• ​Intel Compiler
• INRIA: ​Kerrighed: A SSI Cluster OS Running OpenMP