Step-by-step instructions to install KAPSEL

STEP 0: System requirements to use KAPSEL

  • Windows (+ Cygwin) / Linux / Mac
    • To use KAPSEL on Linux, go to STEP 1.
    • To use KAPSEL on Windows, install “Cygwin” first, then go to STEP 1. Be sure to include “gcc”, “make” (both exist under “Devel” tree) “libfftw3-devel” (under “Math” tree) “libhdf5-devel” and “libhdf5cpp-11”. Both 32-bit and 64-bit cygwin should work.
    • You may use KAPSEL on macOS but not officially supported. By default, gcc command calls clang compiler which produces slow executable. You need to install gcc by yourself to obtain full performance.
  • Gnu C++ Compiler / Intel C++ Compiler
  • OCTA
  • FFTW
  • HDF5
  • LIS (optional)
  • KAPSEL (source codes and examples)

STEP 1: Install OCTA

  • Visit OCTA-HP and download an appropriate installer of OCTA.
  • Install OCTA, usually just by running the installer. Please be a member of OCTA-BBS if you have any questions about OCTA/GOURMET.
  • In the following STEPS, we assume that OCTA8.# is installed in /usr/local/OCTA8# (Linux/Mac) or C:\OCTA8.# (Windows). Replace “#” with a number appropriate for your installed version of OCTA.

STEP 2: Build libplatform

  • “libplatform” is an I/O library used by OCTA to access data stored in UDF-formatted files.
  • You may have to install additional software such as Java(JDK) and Python3 to build libplatform depending on your environment.
  • For all architectures, one may have to do the following before “./configure”.
touch aclocal.m4
touch config.h.in
touch Makefile.in
touch configure
  • For Windows:
    • Logon Windows with an administrator authorization
    • Open Cygwin terminal window
ln -s /cygdrive/c/OCTA8.# /usr/local/OCTA8#
cd /usr/local/OCTA8#/GOURMET/src
make clean
./configure --with-python
make (in case of error, make CCOPT=)
make install
  • For Linux:
su
cd /usr/local/OCTA8#/GOURMET/src
make clean
./configure --with-python
make
make install
  • For Mac:
    • with clang (default C compiler of macOS)
cd /usr/local/OCTA8#/GOURMET/src
make distclean; make clean
./configure CC=clang CXX=clang++ --with-python
make
sudo make install
    • with gcc (Optional C compiler to be installed additionally. Replace “9” with an appropriate number of your environment.)
cd /usr/local/OCTA8#/GOURMET/src
make distclean; make clean
./configure CC=gcc-9 CXX=g++-9 --with-python
make
sudo make install
mv ../lib/macosx/libplatform.a ../lib/macosx/libplatform_gcc.a
    • In some cases, GOURMET doesn’t start properly after building libplatform. Please do the following if this happens.
cd ..
Make-All.sh

STEP 3: Install FFTW

  • For Windows
    • Install “libfftw3-devel” using Cygwin installer.
  • For Linux
    • Install libraries (recommended)
yum -y install fftw fftw-devel
    • Or download source codes and install manually (optional)
      • with gcc
./configure --prefix=/opt/fftw/3.3.8 CFLAGS="-O3" FFLAGS="-O3" --enable-openmp --enable-threads --enable-shared --disable-fortran
make
make install
      • with icc
./configure --prefix=/opt/fftw/3.3.8 CC=icc CXX=icpc CFLAGS="-O3" FFLAGS="-O3" --enable-openmp --enable-threads --enable-shared --disable-fortran --enable-sse2 --enable-avx --enable-avx2
make
make install
  • For Mac
    • Install libraries (recommended)
brew install fftw
    • Or download source codes and install manually (optional)
      • with gcc
./configure --prefix=/opt/fftw/3.3.8 CC=gcc-9 CXX=g++-9 CFLAGS="-O3" FFLAGS="-O3" --enable-openmp --enable-threads --enable-shared --disable-fortran
make
sudo make install

STEP 4: Install HDF5

  • For Windows
    • Install “libhdf5-devel” and “libhdf5cpp-11” using Cygwin installer.
  • For Linux
    • Install libraries (recommended)
yum -y install hdf5-devel
    • Or download source codes and install manually (optional)
      • with gcc
./configure --prefix=/opt/hdf5/1.10.5 CFLAGS="-O3" FFLAGS="-O3" --enable-fortran --enable-cxx
make
make install
      • with icc
./configure --prefix=/opt/hdf5/1.10.5 CC=icc CXX=icpc CFLAGS="-O3" FFLAGS="-O3" --enable-fortran --enable-cxx
make
make install
  • For Mac
    • Install libraries (recommended)
brew install hdf5
    • Or download source codes and install manually (optional)
      • with gcc
./configure --prefix=/opt/hdf5/1.10.5 CC=gcc-9 CXX=g++-9 --enable-fortran --enable-cxx
make
sudo make install

STEP 5: Install LIS (optional, can be skipped now)

  • For Windows
    • Download source codes and install manually. This will installfiles in $INSTALLDIR.
configure.bat
nmake
nmake install
  • For Linux
    • Download source codes and install manually
      • with gcc
./configure --prefix=/opt/lis/2.0.18
make
make install
      • with icc
./configure --prefix=/opt/lis/2.0.18 CC=icc CXX=icpc 
make
make install
  • For Mac
    • Download source codes and install manually
      • with gcc
./configure --prefix=/opt/lis/2.0.18 CC=gcc-9 CXX=g++-9
make
sudo make install

STEP 6: Build KAPSEL executables

  • Download the latest version of KAPSEL source codes & examples (kapsel4.#.zip) from below. Replace “#” with an appropriate number.
unzip kapsel4.#.zip
cd kapsel4.#
  • You may modify GOURMET_HOME_PATH in the attached “Makefile” so that the library “libplatform.a” is correctly linked. Also modify -I(include file path) and -L(library file path) appropriately for your environment.
  • Clean-up working files first.
make clean
  • Build a KAPSEL binary executable appropriate for your environment by one of the following make commands. Here, more/less * represent higher/lower computational performance.
    • with Ooura FFT (easy to build but slow, not compatible with OpenMP)
make ENV=CYGWIN       (Cygwin on Windows)*

make ENV=GCC          (gcc on Linux)*

make ENV=ICC          (icc on Linux)*

make ENV=GCC_MAC      (gcc on Mac)*

make ENV=CLANG        (clang on Mac)
    • with Intel MKL (as fast as FFTW, compatible with OpenMP)
make ENV=ICC FFT=IMKL         (icc on Linux)**

make ENV=ICC_OMP FFT=IMKL     (icc on Linux for OpenMP)***
    • with FFTW (faster than Ooura, compatible with OpenMP)
make ENV=CYGWIN FFT=FFTW      (Cygwin on Windows)**

make ENV=CYGWIN_OMP FFT=FFTW  (Cygwin on Windows for OpenMP)***

make ENV=GCC FFT=FFTW         (gcc on Linux)**

make ENV=GCC_OMP FFT=FFTW     (gcc on Linux for OpenMP)***

make ENV=ICC FFT=FFTW         (icc on Linux)**

make ENV=ICC_OMP FFT=FFTW     (icc on Linux for OpenMP)****

make ENV=GCC_MAC FFT=FFTW     (gcc on Mac)**

make ENV=GCC_MAC_OMP FFT=FFTW (gcc on Mac for OpenMP)***

STEP 7: Test KAPSEL

cd UDF
../kapsel -Iinput.udf -Ooutput.udf -Ddefine.udf -Rrestart.udf
#using input.udf as input
#using output.udf as output
#using define.udf as definition
#using restart.udf as restart
  
   ...
  
#output.udf end.
#restart.udf end.
#Simulation has ended!
#Total Running Time (s):      24.77
#                   (m):       0.41
#                   (h):       0.01
#Average Step Time  (s):       0.02
#                   (m):       0.00
#                   (h):       0.00
  • If you see a command-line output as shown above, KAPSEL has been successfully built and run.
  • The sample input.udf conduct a simulation of sedimenting 5-heavier + 5-lighter particles in Newtonian fluid, which is resolved by a 32x32x32 CFD grids. It takes less than a minute to be completed. If this sample successfully run, “output.udf” will be created.

STEP 8: How to visualize simulation data of KAPSEL

  • You can visualize various simulation data using python script from Gourmet. Simple examples are shown below.
  • Start GOURMET
    • Windows: (OCTA install dir)\GOURMET\gourmet
    • Linux: /usr/local/OCTA83/GOURMET/gourmet
    • Mac: /usr/local/OCTA83/GOURMET/src/osxapp/StartGourmet.app
  • Open “output.udf” (“File” -> “Open” -> Open “output.udf”) [1]
    • Instantaneous positions and velocities of all the particles can be seen as variables in “Particles[]”. Use slide bar at the bottom of GOURMET viewer window to see variables at different time steps.
  • Making an animation on GOURMET
    • Go down to “Python” panel, and click “Load” [2]
    • Open “particleshow.py”, and click “Run” [3]
    • A new window will open, then click the playback button there. [4]

  • Plot data using gnuplot from GOURMET
    • Go down to “Python” panel, and click “Load” [2]
    • Open “plot.py”, and click “Run” [3]
    • Go up to “View” box, and check “Table” [4]
    • Go left and select “Graph Sheet[]”. [5]
    • go down to “Plot” panel, and type the following in the command box. [6,7]
plot 'plot.dat' using 2:7 with lines
    • Click “Plot”, then you will see the time evolution of Vx of the 1st particle. [8]

STEP 9: How to analyze simulation data of KAPSEL

  • The history of simulation run (instantaneous positions and velocities of particles) is stored in “output.udf”. One can access to this file by one of the following methods.
    • Python code with “UDFManager.py”: Please read the manual below for more general information on accessing UDF using Python.
      • English: pythoninterface_eng.pdf
      • Japanese: PythonInterface_jpn.pdf
      • “sk.py” is a sample python script to calculate the static structure factor S(k) from the temporal particle positions stored in “output.udf”. It requires the “numpy” package to handle array variables. You can read and edit the script for your own purposes.
      • For Windows: “Start Menu” > “All Programs” > “OCTA” > “StartGourmetTerm
      • For Linux / Mac: Run “/usr/local/OCTA83/GOURMET/gourmetterm”
python sk.py
gnuplot
>> plot 'sk.dat' w line
    • Jupyter Notebook with “UDFManager.py”: You can do the same thing using Jupyter Notebook (distributed with Anaconda etc…) as follows from bash prompt.
. $PF_FILES/bin/gourmet_profile.sh
. $PF_FILES/bin/platform_env.sh
jupyter notebook sk.ipynb
    • Fortran or C codes with “libplatform”: Please read the manual below more general information on accessing UDF using Fortran or C.
  • KAPSEL returns several important data to stderr as well (temporal stress etc). It usually appears in command line, but one can redirect stderr to a file as follows.
    • For csh or tcsh
../kapsel  -Iinput.udf -Ooutput.udf -Ddefine.udf -Rrestart.udf >& out
    • For sh, bash, or Windows command prompt
../kapsel  -Iinput.udf -Ooutput.udf -Ddefine.udf -Rrestart.udf 2> out