Prof. David Darmofal

Software

Solution Adaptive Numerical Simulator (SANS)

We are developing the software, SANS: Solution Adaptive Numerical Simulator:

  • The SANS acronym is also meant to imply the French word sans meaning “without”, as a significant goal of our research is the development of algorithms to numerically solve partial differential equations “without” the headaches of human-determined meshing.
  • So, in terms of pronouncing SANS, think French.
  • Also, given this is being developed at MIT, the pairing of MIT SANS = mit sans = with (German) without (French) is a happy coincidence.
  • While we do in fact frequently use SANS to solve the Reynolds-Averaged Navier-Stokes (RANS) equations with the Spalart-Allmaras (SA) turbulence model, SANS is not a reference to Spalart-Allmaras or RANS.  Though, it is certainly another happy coincidence that a lead developer of SANS is Steve Allmaras (yes, the same Allmaras of the SA turbulence model).  We’ll leave it to you to decide how coincidental that is…

SANS is team-developed using the software engineering principles described in a Galbraith et alGalbraith_2015_Verification_CFD_Development. A key design principle is that SANS is intended to enable academic research into higher-order, adaptive finite element methods.  Thus, while speed, memory efficiency, etc are also important, we place a high value on being able to test new ideas.  The following is a short synopsis of some of the capabilities and features or SANS:

  • Primarily written in C++ with some python
  • Automatic differentiation via operator overloading
  • Templating for efficient yet general code
  • Continuous integration and testing
  • Continuous Galerkin (CG) and discontinuous Galerkin (DG) discretizations from 1D to 4D (time + 3D space)
  • Stabilization for CG discretizations of convection-dominated systems including SUPG, GLS, VMS, and VMSD
  • Parallel implementation including utilization of PETSc
  • Adaptation for CG and DG using the MOESS algorithm
  • Flexible PDE support in particular for convective-diffusive systems.  Some of the PDEs implemented in SANS include:
    • Linear convection-diffusion systems
    • Burgers equation
    • Compressible Euler
    • Compressible Navier-Stokes
    • Compressible Reynolds-averaged Navier-Stokes with Spalart-Allmaras turbulence model
    • Multi-phase porous media flows in mass conservation form
  • Manifold PDE (in particular for integral boundary layer methods and structural shell models)

Cloning SANS

SANS can be cloned for non-development use with the following command:

git clone ssh://USERNAME@acdl.mit.edu/home/svnroot/SANSproto

where ‘USERNAME’ is your username on acdl.mit.edu.

For developers, SANS should be cloned using this script:

https://acdl.mit.edu/csi/userContent/SANS_developer.sh

Change the script’s execute permissions with

chmod +x SANS_developer.sh

and run the script with the command

./SANS_developer.sh ‘username’ [options]

where ‘username’ is your username on acdl.mit.edu

Available options:
–dir : a directory where SANS could be cloned (defaults ot SANS)
–email : your preferred email (defaults to ‘username’@mit.edu)

SANS requirements minimum version dependencies:

  • g++ 4.8.3 (4.8.1 and 4.8.2 have a bug) or clang 3.5 or intel 14
  • CMake 2.8.8
  • boost-iostreams 1.48
  • boost-test 1.48
  • boost-python 1.48
  • boost-filesystem 1.48
  • boost-system 1.48
  • ESP 1.19
  • nlopt 2.4.2
  • suitesparse 3.4 (oldest tested so far)

For developers, additional dependencies are:

  • valgrind 3.10
  • cppcheck 1.74
  • vera++ 1.2
  • lcov 1.12

Optional dependencies:

  • numpy
  • refine
  • avro
  • Advancing Front/Local Reconnection (AFLR3) 16.30.11
  • TetGen 1.5.0 (viral license)
  • triangle (viral license)
  • Intel Math Kernel Library
  • MPI
  • parmetis 3.2
  • metis
  • boost-mpi 1.48
  • boost-serialization 1.48
  • PETSc 3.7.6

Dependency install instructions

Engineering Sketch Pad

Download Engineering Sketch Pad and OpenCASCADE from: http://acdl.mit.edu/ESP/ (Beta version if needed: ESPbeta.tgz). Follow “2. Building the Software” instructions in ESPreadme.txt to compile ESP. Afer compiling ESP, add the following to your .bashrc file on Linux and .bash_profile file on Mac OS X:

export ESP_DIR=/path/to/EngSketchPad
export CAS_DIR=/path/to/OpenCASCADE-?.?.?

If the ESP_DIR and CAS_DIR variables are not set, then the ESP_ROOT and CASROOT environment variables (typically defined by ESPenv) will be used to find ESP.

Advancing Front/Local Reconnection

Download and untar AFLR3, and add the following to your .bashrc file on Linux and .bash_profile file on Mac OS X:

export AFLR_DIR=/path/to/aflr3

Intel Math Kernel Library

The MKL library is part of the intel compiler suite, or it can be downloaded independently from intel. This also facilitates BLAS library ourintes. The use of the MKL library can be enabled with the cmake variable USE_MKL.

NOTE: The intel environment variables are incomplete on macOS, pleaes make sure to use the lines below in your .bash_profile

source /opt/intel/oneapi/setvars.sh
export LIBRARY_PATH=/opt/intel/oneapi/compiler/latest/mac/compiler/lib:$LIBRARY_PATH

All other dependencies

The following instructions will install all the SANS dependency minimum version requirements:

Ubuntu 20.04 and 22.04 

Execute the following commands:

sudo apt-get install cmake cmake-curses-gui libsuitesparse-dev libboost-all-dev libnlopt-dev libnlopt-cxx-dev libblas-dev python3-numpy zlib1g-dev libeigen3-dev

For parallel computing with MPI:

sudo apt-get install libopenmpi-dev libmetis-dev liblapack-dev 

For developers, you should also run:

sudo add-apt-repository ppa:ubuntu-toolchain-r/test
sudo apt-get update
sudo apt-get install vera++ valgrind cppcheck ccache clang kcachegrind massif-visualizer

Optionally, developers may also install:

sudo apt-get install liblapacke-dev

Ubuntu 16.04 and 18.04

Execute the following commands:

sudo apt-get install cmake cmake-curses-gui libsuitesparse-dev libboost-all-dev libnlopt-dev libblas-dev python-numpy zlib1g-dev

For parallel computing with MPI:

sudo apt-get install libopenmpi-dev libmetis-dev liblapack-dev 

For developers, you should also run:

sudo add-apt-repository ppa:ubuntu-toolchain-r/test
sudo apt-get update
sudo apt-get install vera++ valgrind cppcheck ccache clang libc++-dev kcachegrind massif-visualizer

Optionally, developers may also install:

sudo apt-get install liblapacke-dev

Ubuntu 14.04

Execute the following commands:

sudo add-apt-repository ppa:bzindovic/suitesparse-bugfix-1319687
sudo apt-get update
sudo apt-get install cmake cmake-curses-gui libsuitesparse-dev libboost-all-dev libnlopt-dev libblas-dev python-numpy

For parallel computing with MPI:

sudo apt-get install libopenmpi-dev libmetis-dev liblapack-dev 

For developers, you should also run:

sudo add-apt-repository ppa:ubuntu-toolchain-r/test
sudo add-apt-repository ppa:csaba-kertesz/aiboplus
sudo add-apt-repository ppa:kubuntu-ppa/backports
sudo apt-get update
sudo apt-get install vera++ valgrind cppcheck ccache clang libc++-dev kcachegrind massif-visualizer

Optionally, developers may also install:

sudo apt-get install liblapacke-dev

CentOS 8

Execute the following commands:

sudo dnf groupinstall "Development Tools"
sudo dnf config-manager --set-enabled powertools
sudo dnf --enablerepo=powertools install cmake blas-devel suitesparse-devel python3-devel boost-devel boost-python3-devel zlib-devel
sudo alternatives --set python /usr/bin/python3

Note: nlopt is currently not available as a package and CMake will instead download and compile it.

For parallel computing with MPI:

sudo dnf --enablerepo=PowerTools install metis-devel openmpi-devel lapack-devel

For developers, you should also run:

sudo dnf --enablerepo=PowerTools install cppcheck

Optionally, developers may also install:

Apple macOS

Install the Clang compiler with the command:

xcode-select --install

All other dependencies can be install with either homebrew (http://brew.sh/) or macports (http://www.macports.org/).

homebrew

Install homebrew using the instructions at http://brew.sh/.

All required dependencies can be installed with the commands:

brew doctor
brew install cmake boost boost-python3 numpy suite-sparse nlopt eigen

For parallel computing with MPI:

brew install gcc boost-mpi metis

Dependencies for developers:

brew install lcov cppcheck vera++ ccache lapack

If lcov generates error messages when generating coverage information, try running:

brew install libgd
sudo cpan -i -f GD

Shell completion allows the use of tabs to complete makefile and git commands. You can tell which shell you are using with

echo $SHELL

and then install the correpsonding completion with on of the commands:

brew install bash-completion

or

brew install zsh-completion

Then follow the instructions here https://docs.brew.sh/Shell-Completion.

Verifing SANS

In the SANS root directory, execute the commands:

mkdir -p build/deploy
cd build/deploy
cmake ../../
make check

If you want parallel build you should pass the -j flag to make, e.g. to compile with 4 processes

make -j 4 check