Welcome to the home of MFC! MFC simulates compressible multi-component and multi-phase flows, amongst other things. It scales ideally to exascale; tens of thousands of GPUs on NVIDIA- and AMD-GPU machines on Oak Ridge Summit and Frontier. MFC is written in Fortran and makes use of metaprogramming to keep the code short (about 20K lines).

Get in touch with Spencer if you have questions! We have an active Slack channel and development team. MFC has high-level documentation, visualizations, and more on its website.

An example

We keep many examples. Here's one! MFC can execute high-fidelity simulations of shock-droplet interaction (see examples/3d_shockdroplet)

Another example is the high-Mach flow over an airfoil, shown below.

Getting started

You can navigate to this webpage to get started using MFC! It's rather straightforward. We'll give a brief intro. here for MacOS. Using brew, install MFC's modest set of dependencies:

brew install wget python cmake gcc@13 mpich

You're now ready to build and test MFC! Put it to a convenient directory via

git clone https://github.com/MFlowCode/MFC.git
cd MFC

and be sure MFC knows what compilers to use by appending and sourcing your ~/.profile file via this command

echo -e "export CC=gcc-13 \nexport CXX=g++-13 \nexport FC=gfortran-13" >> ~/.profile
source ~/.profile

then you can build MFC and run the test suite!

./mfc.sh build -j $(nproc)
./mfc.sh test -j $(nproc)

And... you're done!

You can learn more about MFC's capabilities via its documentation or play with the examples located in the examples/ directory (some are shown here)!

The shock-droplet interaction case above was run via

./mfc.sh run ./examples/3d_shockdroplet/case.py -n 8

where 8 is the number of cores the example will run on. You can visualize the output data, located in examples/3d_shockdroplet/silo_hdf5, via Paraview, Visit, or your other favorite software.

Is this really exascale

OLCF Frontier is the first exascale supercomputer. The weak scaling of MFC on this machine is below, showing near-ideal utilization.

What else can this thing do

MFC has many features. They are organized below. Just click the drop-downs!

Physics

• 1-3D
• Compressible
  • Low Mach number treatment available
• Multi- and single-component
  • 4, 5, and 6 equation models for multi-component/phase features
  • Kapila and Allaire models
• Multi- and single-phase
  • Phase change via p, pT, and pTg schemes
• Grids
  • 1-3D Cartesian, cylindrical, axisymmetric.
  • Arbitrary grid stretching for multiple domain regions.
  • Complex/arbitrary geometries via immersed boundary methods
  • STL geometry files supported
• Surface tension for multiphase cases
• Sub-grid Euler-Euler multiphase models for bubble dynamics and similar
• Viscous effects (high-order accurate representations)
• Ideal and stiffened gas equations of state
• Body forces
• Acoustic wave generation (one- and two-way sound sources)

Numerics

• Shock and interface capturing schemes
  • First-order upwinding, WENO3 and 5.
  • Monotonicity-preserving reconstructions
  • Reliable handling of high density ratios.
• Exact and approximate (e.g., HLL, HLLC) Riemann solvers
• Boundary conditions: Periodic, reflective, extrapolation/Neumann, slip/no-slip, non-reflecting characteristic buffers, inflows, outflows, and more.
• Runge-Kutta orders 1-3 (SSP TVD)
• Interface sharpening (THINC-like)

Large-scale and accelerated simulation

• GPU compatible on NVIDIA (P/V/A/H100, GH200, etc.) and AMD (MI200+) hardware
• Ideal weak scaling to 100% of the largest GPU supercomputers
  • >10K NVIDIA GPUs on OLCF Summit (NV V100-based)
  • >66K AMD GPUs on the first exascale computer, OLCF Frontier (AMD MI250X-based)
• Near compute roofline behavior
• RDMA (remote data memory access; GPU-GPU direct communication) via GPU-aware MPI on NVIDIA (CUDA-aware MPI) and AMD GPU systems

Software robustness and other features

• Fypp metaprogramming for code readability, performance, and portability
• Continuous Integration (CI)
  • >100 Regression tests with each PR.
    • Performed with GNU, Intel, and NVIDIA compilers on NVIDIA and AMD GPUs.
    • Line-level test coverage reports via Codecov and gcov
  • Benchmarking to avoid performance regressions and identify speed-ups
• Continuous Deployment (CD) of website and API documentation

Citation

If you use MFC, consider citing it as:

S. H. Bryngelson, K. Schmidmayer, V. Coralic, K. Maeda, J. Meng, T. Colonius (2021) Computer Physics Communications 266, 107396

@article{Bryngelson_2021,
  title   = {{MFC: A}n open-source high-order multi-component, multi-phase, and multi-scale compressible flow solver},
  author  = {S. H. Bryngelson and K. Schmidmayer and V. Coralic and J. C. Meng and K. Maeda and T. Colonius},
  journal = {Computer Physics Communications},
  year    = {2021},
  volume  = {266},
  pages   = {107396},
  doi     = {10.1016/j.cpc.2020.107396}
}

@article{Radhakrishnan_2024,
  title   = {Method for portable, scalable, and performant {GPU}-accelerated simulation of multiphase compressible flow},
  author  = {A. Radhakrishnan and H. {Le Berre} and B. Wilfong and J.-S. Spratt and M. {Rodriguez Jr.} and T. Colonius and S. H. Bryngelson},
  journal = {Computer Physics Communications},
  year    = {2024},
  volume  = {302},
  pages   = {109238},
  doi     = {10.1016/j.cpc.2024.109238}
}

License

Copyright 2021-2024 Spencer Bryngelson and Tim Colonius. MFC is under the MIT license (see LICENSE for full text).

Acknowledgements

Multiple federal sponsors have supported MFC development, including the US Department of Defense (DOD), National Institutes of Health (NIH), Department of Energy (DOE), and National Science Foundation (NSF).

MFC computations have used many supercomputing systems. A partial list is below

• OLCF Frontier and Summit, and testbed systems Wombat, Crusher, and Spock (allocation CFD154, PI Bryngelson)
• PSC Bridges(1/2), NCSA Delta, SDSC Comet and Expanse, Purdue Anvil, TACC Stampede(1-3), and TAMU ACES via ACCESS-CI (allocations TG-CTS120005 (PI Colonius) and TG-PHY210084 (PI Bryngelson))
• DOD systems Onyx, Carpenter, and Nautilus via the DOD HPCMP program
• Sandia National Labs systems Doom and Attaway and testbed systems Weaver and Vortex