Thread: NAMD : compilation, integration with lava / LSF HPC

Originally posted by: mbozzore, Sun Apr 08, 2007 5:17 am

Please post everything related to NAMD ( http://www.ks.uiuc.edu/Research/namd/ ) here.
_________________
Mehdi Bozzo-Rey

Originally posted by: mbozzore, Sun Apr 08, 2007 5:28 am

The first thing to get installed is fftw ( FFTW Home Page )

1- Download the distribution (2.1.5, not version 3). I just noticed the latest version (3.2 alpha2 at this time) supports the Cell BE processor and also has an experimental new MPI interface.

wget http://www.fftw.org/fftw-2.1.5.tar.gz


2- Configure (single precision) and make the distrib

./configure --prefix=/share/apps/libraries/fftw/fftw-2.1.5-s --enable-type-prefix --enable-float


3- Check the build

make check


4- Install

sudo make install


Then you will be able to download / build the other components (assuming the correct module for openmpi is loaded).

1-download the appropriate tcl archive (64 bit version) :

wget http://www.ks.uiuc.edu/Research/namd...x-amd64.tar.gz

The archive mentioned in the NAMD documentation is 32 bits and will not work.


2- Backup and edit the following files, in the namd directory :

In my case, the namd directory was :

/home/mbozzore/compile_temp/NAMD/NAMD_2.6_Source


2.1- Make.charm :

cp -p Make.charm Make.charm.orig

And here is the diff :
---------------------------------------
[mbozzore@frontend NAMD_2.6_Source]$ diff Make.charm Make.charm.orig
3c3
< CHARMBASE = /home/mbozzore/compile_temp/NAMD/charm-5.9
---
> CHARMBASE = /Projects/namd2/charm-5.9
---------------------------------------


2.2- arch/Linux-amd64.fftw

cp arch/Linux-amd64.fftw arch/Linux-amd64.fftw.orig


And here is the diff :
---------------------------------------
[mbozzore@frontend NAMD_2.6_Source]$ diff arch/Linux-amd64.fftw arch/Linux-amd64.fftw.orig
2,4c2,4
< FFTDIR=/share/apps/libraries/fftw/fftw-2.1.5-s/
< FFTINCL=-I$(FFTDIR)/include
< FFTLIB=-L$(FFTDIR)/lib -lsrfftw -lsfftw
---
> FFTDIR=/Projects/namd2/fftw/linux-amd64
> FFTINCL=-I$(FFTDIR)/include -I$(HOME)/fftw/include
> FFTLIB=-L$(FFTDIR)/lib -L$(HOME)/fftw/lib -lsrfftw -lsfftw
---------------------------------------


2.3- arch/Linux-amd64.tcl

cp arch/Linux-amd64.tcl arch/Linux-amd64.tcl.orig


The associated diff :
---------------------------------------
[mbozzore@frontend NAMD_2.6_Source]$ diff arch/Linux-amd64.tcl arch/Linux-amd64.tcl.orig
2,4c2,4
< TCLDIR=/home/mbozzore/compile_temp/NAMD/tcl/linux-amd64/
< TCLINCL=-I$(TCLDIR)/include
< TCLLIB=-L$(TCLDIR)/lib -ltcl8.3 -ldl
---
> TCLDIR=/Projects/namd2/tcl/linux-amd64
> TCLINCL=-I$(TCLDIR)/include -I$(HOME)/tcl/include
> TCLLIB=-L$(TCLDIR)/lib -L$(HOME)/tcl/lib -ltcl8.3 -ldl
---------------------------------------


3- Execute the configure script with the correct arguments :

---------------------------------------
[mbozzore@frontend NAMD_2.6_Source]$ ./config tcl fftw Linux-amd64-MPI
Creating directory: Linux-amd64-MPI
Creating link: .. to .rootdir
Creating Makearch
Linking Makefiles
Linking src, plugins, and psfgen
Creating dpme2 and linking files
Creating dpmta-2.6 and linking files
You are ready to do a make in directory Linux-amd64-MPI now.
---------------------------------------



4- Make the distribution :

You will hit the following problem :

---------------------------------------
[mbozzore@frontend Linux-amd64-MPI]$ make
mkdir inc
mkdir obj
g++ -I/home/mbozzore/compile_temp/NAMD/charm-5.9/mpi-linux-amd64/include -DCMK_OPTIMIZE=1 -Isrc -Iinc -Iplugins/include -I/home/mbozzore/compile_temp/NAMD/tcl/linux-amd64//include -DNAMD_TCL -I/share/apps/libraries/fftw/fftw-2.1.5-s//include -DNAMD_FFTW -DNAMD_VERSION=\"2.6\" -DNAMD_PLATFORM=\"Linux-amd64-MPI\" -O3 -m64 -fexpensive-optimizations -ffast-math -o obj/common.o -c src/common.C
src/common.C:22:21: charm++.h: No such file or directory
src/common.C: In function `void NAMD_quit(const char*)':
src/common.C:67: error: `CkPrintf' was not declared in this scope
src/common.C:69: error: `CmiAbort' was not declared in this scope
src/common.C: In function `void NAMD_die(const char*)':
src/common.C:80: error: `CkPrintf' was not declared in this scope
src/common.C:82: error: `CmiAbort' was not declared in this scope
src/common.C: In function `void NAMD_err(const char*)':
src/common.C:95: error: `CkPrintf' was not declared in this scope
src/common.C:97: error: `CmiAbort' was not declared in this scope
src/common.C: In function `void NAMD_bug(const char*)':
src/common.C:110: error: `CkPrintf' was not declared in this scope
src/common.C:112: error: `CmiAbort' was not declared in this scope
make: *** [obj/common.o] Error 1
---------------------------------------


The solution is to create the correct symbolic link :

ln -s /home/mbozzore/compile_temp/NAMD/charm-5.9/mpi-linux-pthreads /home/mbozzore/compile_temp/NAMD/charm-5.9/mpi-linux-amd64

Then, make again. Everything will be ok and the namd2 binary will be created.

---------------------------------------
[mbozzore@frontend Linux-amd64-MPI]$ ldd namd2
libpthread.so.0 => /lib64/tls/libpthread.so.0 (0x000000319e100000)
libdl.so.2 => /lib64/libdl.so.2 (0x000000319df00000)
libz.so.1 => /usr/lib64/libz.so.1 (0x000000319e500000)
libm.so.6 => /lib64/tls/libm.so.6 (0x000000319dd00000)
libmpi_cxx.so.0 => /share/apps/open-mpi/openmpi-1.2-gnu/lib/libmpi_cxx.so.0 (0x0000002a9557b000)
libmpi.so.0 => /share/apps/open-mpi/openmpi-1.2-gnu/lib/libmpi.so.0 (0x0000002a9569e000)
libopen-rte.so.0 => /share/apps/open-mpi/openmpi-1.2-gnu/lib/libopen-rte.so.0 (0x0000002a95831000)
libopen-pal.so.0 => /share/apps/open-mpi/openmpi-1.2-gnu/lib/libopen-pal.so.0 (0x0000002a9598b000)
libnsl.so.1 => /lib64/libnsl.so.1 (0x00000031a0400000)
libutil.so.1 => /lib64/libutil.so.1 (0x00000031a2500000)
libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00000031a1c00000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00000031a1a00000)
libc.so.6 => /lib64/tls/libc.so.6 (0x000000319da00000)
/lib64/ld-linux-x86-64.so.2 (0x000000319d800000)
---------------------------------------


5- Run the example that comes with the distribution (outside of lava / LSF HPC):

---------------------------------------
[mbozzore@frontend Linux-amd64-MPI]$ mpirun -np 2 --machinefile ../../charm-5.9/mpi-linux-pthreads/tests/charm++/hosts --prefix $MPIHOME ./namd2 src/alanin
---------------------------------------

6- Now you can run the same job through lava (using the openmpi-mpirun script) or LSF HPC (using -a openmpi)
_________________
Mehdi Bozzo-Rey

Originally posted by: hazards, Mon Oct 29, 2007 9:29 pm

I have a dual quad core Xeon system.
The UIUC/libraries directory does not contain Xeon-files and does not appear
to contain generic _86-64bit libraries for tcl or fftw. When I attempt to use the AMD64 libraries as in this post, I get the errors as described but am unable to correct this with symbolic link fix as suggested.

What am I supposed to linking

Originally posted by: hazards, Tue Nov 06, 2007 7:05 pm

I think a crucial detail missing here is the charm++ build command.

./build charm++ mpi-linux pthreads --libdir /opt/openmpi/1.1.4/lib --incdir /opt/openmpi/1.1.4/include --no-shared -O -DCMK_OPTIMIZE=1

This creates the mpi-linux-pthreads directory and a functional charm
but produces errors when compiling the charm tests

../../../bin/charmc -o pgm megatest.o groupring.o nodering.o varsizetest.o varraystest.o groupcast.o nodecast.o synctest.o fib.o arrayring.o tempotest.o packtest.o queens.o migration.o marshall.o priomsg.o priotest.o rotest.o statistics.o templates.o inherit.o reduction.o callback.o immediatering.o bitvector.o -language charm++
/usr/bin/ld: cannot find -lmpich

When I change the build command to make the lib and include files point to

./build charm++ mpi-linux pthreads --libdir /opt/mpich/gnu/lib --incdir /opt/mpich/gnu/include --no-shared -O -DCMK_OPTIMIZE=1


The tests compile but fail with this error:

./charmrun ./pgm +p1

Running on 1 processors: ./pgm
0 - <NO ERROR MESSAGE> : Could not convert index 7041856 into a pointer
The index may be an incorrect argument.
Possible sources of this problem are a missing "include 'mpif.h'",
a misspelled MPI object (e.g., MPI_COM_WORLD instead of MPI_COMM_WORLD)
or a misspelled user variable for an MPI object (e.g.,
com instead of comm).
[0] Aborting program !
[0] Aborting program!
p0_13864: p4_error: : 9039
make[1]: *** [test] Error 1
make[1]: Leaving directory `/home/hazards/COMPILE/NAMD_2.6_Source/charm-5.9/mpi-linux-pthreads/tests/charm++/megatest'
make: *** [test] Error 1

That suggests an error where some feature of the present 64-bit
architecture differs from 32-bit situation. I see NUMEROUS "warnings"
during the build process eg

global-elfgot.C:158: warning: cast to pointer from integer of different size


and

/usr/include/sys/cdefs.h:46:1: warning: this is the location of the previous definition

Building charm with the /opt/lam files fails.

Originally posted by: mbozzore, Tue Nov 06, 2007 10:24 pm

Hello ,

Below is the build command I used :

./build charm++ mpi-linux-amd64

Then, you need to fix the "lmpich problem"; below is the diff for the appropriate conv-mach.sh file (basically, you need to use -lmpi instead of -lmpich if you are using OpenMPI):

-----------------------
diff src/arch/mpi-linux-amd64/conv-mach.sh src/arch/mpi-linux-amd64/conv-mach.sh.orig
13c13
< CMK_SYSLIBS="-lmpi "
---
> CMK_SYSLIBS="-lmpich "
-----------------------


Then, you should be able to recompile the pgm binary (under the megatest directory) and run it using the following :


mpirun -np 4 --machinefile ./machines --prefix $MPIHOME ./pgm

I assume here that the MPIHOME env is correctly setup (I recompiled OpenMPI 1.2.4 and wrote the appropriate module file).

The output of ldd is in my case:

-------------------------
ldd pgm
libmpi.so.0 => /share/apps/openmpi/1.2.4-eth/lib/libmpi.so.0 (0x0000002a95557000)
libdl.so.2 => /lib64/libdl.so.2 (0x0000003e57900000)
libz.so.1 => /usr/lib64/libz.so.1 (0x0000003e58100000)
libmpi_cxx.so.0 => /share/apps/openmpi/1.2.4-eth/lib/libmpi_cxx.so.0 (0x0000002a9570b000)
libopen-rte.so.0 => /share/apps/openmpi/1.2.4-eth/lib/libopen-rte.so.0 (0x0000002a9582e000)
libopen-pal.so.0 => /share/apps/openmpi/1.2.4-eth/lib/libopen-pal.so.0 (0x0000002a95987000)
libnsl.so.1 => /lib64/libnsl.so.1 (0x0000003e5e800000)
libutil.so.1 => /lib64/libutil.so.1 (0x0000003e5bd00000)
libm.so.6 => /lib64/tls/libm.so.6 (0x0000003e57700000)
libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x0000003e5b400000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x0000003e5a900000)
libpthread.so.0 => /lib64/tls/libpthread.so.0 (0x0000003e57f00000)
libc.so.6 => /lib64/tls/libc.so.6 (0x0000003e57400000)
/lib64/ld-linux-x86-64.so.2 (0x0000003e57200000)
-------------------------
_________________
Mehdi Bozzo-Rey