Skip to content
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,11 @@ docs/documentation/*-example.png
docs/documentation/examples.md
docs/documentation/case_constraints.md
docs/documentation/physics_constraints.md
docs/documentation/architecture.md
docs/pre_process/readme.md
docs/simulation/readme.md
docs/post_process/readme.md
docs/api/readme.md

examples/*batch/*/
examples/**/D/*
Expand Down
26 changes: 23 additions & 3 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -803,9 +803,10 @@ if (MFC_DOCUMENTATION)
\"${CMAKE_CURRENT_SOURCE_DIR}/docs/custom.css\"")

# > Generate Documentation & Landing Page
GEN_DOCS(pre_process "MFC: Pre-Process")
GEN_DOCS(simulation "MFC: Simulation")
GEN_DOCS(post_process "MFC: Post-Process")
GEN_DOCS(pre_process "MFC")
GEN_DOCS(simulation "MFC")
GEN_DOCS(post_process "MFC")
GEN_DOCS(api "MFC")
GEN_DOCS(documentation "MFC")

# Generate API landing pages for pre_process, simulation, post_process.
Expand All @@ -826,6 +827,7 @@ if (MFC_DOCUMENTATION)
add_dependencies(pre_process_doxygen gen_api_landing)
add_dependencies(simulation_doxygen gen_api_landing)
add_dependencies(post_process_doxygen gen_api_landing)
add_dependencies(api_doxygen gen_api_landing)

# Fix @file/@brief headers to match actual module/program declarations.
# Handles mixed-case Fortran names and catches stale module renames.
Expand All @@ -846,6 +848,24 @@ if (MFC_DOCUMENTATION)
add_dependencies(simulation_doxygen fix_file_briefs)
add_dependencies(post_process_doxygen fix_file_briefs)

# Generate architecture.md from template + module_categories.json + source briefs.
add_custom_command(
OUTPUT "${CMAKE_CURRENT_BINARY_DIR}/gen-architecture.stamp"
DEPENDS "${CMAKE_CURRENT_SOURCE_DIR}/docs/gen_architecture.py"
"${CMAKE_CURRENT_SOURCE_DIR}/docs/module_categories.json"
"${CMAKE_CURRENT_SOURCE_DIR}/docs/documentation/architecture.md.in"
${pre_process_FPPs} ${pre_process_F90s}
${simulation_FPPs} ${simulation_F90s}
${post_process_FPPs} ${post_process_F90s}
COMMAND "${Python3_EXECUTABLE}" "${CMAKE_CURRENT_SOURCE_DIR}/docs/gen_architecture.py"
"${CMAKE_CURRENT_SOURCE_DIR}"
COMMAND "${CMAKE_COMMAND}" -E touch "${CMAKE_CURRENT_BINARY_DIR}/gen-architecture.stamp"
COMMENT "Generating architecture page"
VERBATIM
)
add_custom_target(gen_architecture DEPENDS "${CMAKE_CURRENT_BINARY_DIR}/gen-architecture.stamp")
add_dependencies(documentation_doxygen gen_architecture)
Comment thread
coderabbitai[bot] marked this conversation as resolved.

# Inject per-page last-updated dates into documentation markdown files.
# Runs after auto-generated .md files exist, before Doxygen processes them.
# Uses a stamp file so it only runs once per build.
Expand Down
6 changes: 6 additions & 0 deletions docs/custom.css
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,12 @@
* Overrides for doxygen-awesome theme
*/

/* Hide empty nav-path footer (kept for navtree.js height calculation) */
#nav-path {
height: 0;
overflow: hidden;
}
Comment thread
sbryngelson marked this conversation as resolved.

/* Cross-navigation panel at top of sidebar */
#mfc-nav {
display: flex;
Expand Down
118 changes: 118 additions & 0 deletions docs/documentation/architecture.md.in
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
# Code Architecture {#architecture}

This page explains how MFC's source code is organized, how data flows through the solver, and where to find things. Read this before diving into the source.

## Three-Phase Pipeline

MFC runs as three separate executables that communicate via binary files on disk:

```
pre_process ──> simulation ──> post_process
(grid + (time (derived
initial advance) quantities +
conditions) visualization)
```

| Phase | Entry Point | What It Does |
|-------|-------------|--------------|
| **Pre-Process** | `src/pre_process/p_main.f90` | Generates the computational grid and initial conditions from patch definitions. Writes binary grid and state files. |
| **Simulation** | `src/simulation/p_main.fpp` | Reads the initial state and advances the governing equations in time. Periodically writes solution snapshots. |
| **Post-Process** | `src/post_process/p_main.fpp` | Reads snapshots, computes derived quantities (vorticity, Schlieren, etc.), and writes Silo/HDF5 files for VisIt or ParaView. |

Each phase is an independent MPI program. The simulation phase is where nearly all compute time is spent.

## Directory Layout

```
src/
common/ Shared modules used by all three phases
pre_process/ Pre-process source (grid generation, patch construction)
simulation/ Simulation source (solver core, physics models)
post_process/ Post-process source (derived quantities, formatted I/O)
```

Shared modules in `src/common/` include MPI communication, derived types, variable conversion, and utility functions. They are compiled into each phase.

## Key Data Structures

Two arrays carry the solution through the entire simulation:

| Variable | Contents | When Used |
|----------|----------|-----------|
| `q_cons_vf` | **Conservative** variables: \f$\alpha\rho\f$, \f$\rho u\f$, \f$\rho v\f$, \f$\rho w\f$, \f$E\f$, \f$\alpha\f$ | Storage, time integration, I/O |
| `q_prim_vf` | **Primitive** variables: \f$\rho\f$, \f$u\f$, \f$v\f$, \f$w\f$, \f$p\f$, \f$\alpha\f$ | Reconstruction, Riemann solving, physics |

Both are `vector_field` types (defined in `m_derived_types`), which are arrays of `scalar_field`. Each `scalar_field` wraps a 3D real array `sf(0:m, 0:n, 0:p)` representing one variable on the grid.

The index layout within `q_cons_vf` depends on the flow model:

```
For model_eqns == 2 (5-equation, multi-fluid):

Index: 1 .. num_fluids | num_fluids+1 .. +num_vels | E_idx | adv_idx
Meaning: alpha*rho_k | momentum components | energy | volume fractions
```

Additional variables are appended for bubbles, elastic stress, magnetic fields, or chemistry species when those models are enabled. The total count is `sys_size`.

## The Simulation Loop

The simulation advances the solution through this call chain each time step:

```
p_main (time-step loop)
└─ s_perform_time_step
├─ s_compute_dt [adaptive CFL time step]
└─ s_tvd_rk [Runge-Kutta stages]
├─ s_compute_rhs [assemble dq/dt]
│ ├─ s_convert_conservative_to_primitive_variables
│ ├─ s_populate_variables_buffers [MPI halo exchange]
│ └─ for each direction (x, y, z):
│ ├─ s_reconstruct_cell_boundary_values [WENO]
│ ├─ s_riemann_solver [HLL/HLLC/HLLD]
│ ├─ s_compute_advection_source_term [flux divergence]
│ └─ (physics source terms: viscous, bubbles, etc.)
├─ RK update: q_cons = weighted combination of stages
├─ s_apply_bodyforces [if enabled]
├─ s_pressure_relaxation [if 6-equation model]
└─ s_ibm_correct_state [if immersed boundaries]
```

### What happens at each stage

1. **Conservative → Primitive**: Convert stored `q_cons_vf` to `q_prim_vf` (density, velocity, pressure) using the equation of state. This is done by `m_variables_conversion`.

2. **MPI Halo Exchange**: Ghost cells at subdomain boundaries are filled by communicating with neighbor ranks. Handled by `m_mpi_proxy`.

3. **WENO Reconstruction** (`m_weno`): For each coordinate direction, reconstruct left and right states at cell faces from cell-average primitives using high-order weighted essentially non-oscillatory stencils.

4. **Riemann Solver** (`m_riemann_solvers`): At each cell face, solve the Riemann problem between left and right states to compute intercell fluxes. Available solvers: HLL, HLLC, HLLD.

5. **Flux Differencing** (`m_rhs`): Accumulate the RHS as \f$\partial q / \partial t = -\frac{1}{\Delta x}(F_{j+1/2} - F_{j-1/2})\f$ plus source terms (viscous stress, surface tension, bubble dynamics, body forces, etc.).

6. **Runge-Kutta Update** (`m_time_steppers`): Combine the RHS with the current state using TVD Runge-Kutta coefficients (1st, 2nd, or 3rd order SSP).

## Module Map

MFC has ~80 Fortran modules organized by function. Here is where to look for what:

<!-- MODULE_MAP -->

## MPI Parallelization

The computational domain is decomposed into subdomains via `MPI_Cart_create`. Each rank owns a contiguous block of cells in (x, y, z). Ghost cells of width `buff_size` surround each subdomain and are filled by halo exchange before each RHS evaluation.

On GPUs, the same domain decomposition applies. GPU kernels operate on the local subdomain, with explicit host-device transfers for MPI communication (unless GPU-aware MPI / RDMA is available).

## Adding New Physics

To add a new source term or physics model:

1. **Create a module** in `src/simulation/` (e.g., `m_my_model.fpp`)
2. **Add initialization/finalization** subroutines called from `m_start_up`
3. **Add RHS contributions** called from the dimensional loop in `m_rhs:s_compute_rhs`
4. **Add parameters** to `m_global_parameters` and input validation to `m_checker`
5. **Add a module-level brief** (enforced by the linter in `lint_docs.py`)
6. **Add the module to `docs/module_categories.json`** so it appears in this page

Follow the pattern of existing modules like `m_body_forces` (simple) or `m_viscous` (more involved) as a template.
4 changes: 4 additions & 0 deletions docs/documentation/contributing.md
Original file line number Diff line number Diff line change
Expand Up @@ -536,6 +536,10 @@ contains
end module m_my_feature
```

**Step 3: Register the module in the architecture docs**

Add your module name to the appropriate category in `docs/module_categories.json`. This ensures it appears on the @ref architecture "Code Architecture" page. The precheck linter will fail if a module is missing from this file.
Comment thread
sbryngelson marked this conversation as resolved.

Key conventions:
- `private` by default, explicitly `public` for the module API
- Initialize/finalize subroutines for allocation lifecycle
Expand Down
5 changes: 2 additions & 3 deletions docs/documentation/readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ Welcome to the Multi-component Flow Code (MFC) documentation.

## Advanced Topics

- @ref architecture "Code Architecture" - How the source code is organized, data flow, and module map
Comment thread
sbryngelson marked this conversation as resolved.
- @ref expectedPerformance "Performance" - Optimization and benchmarks
- @ref gpuParallelization "GPU Parallelization" - GPU macro API (developer reference)
- @ref docker "Containers" - Docker usage
Expand All @@ -32,9 +33,7 @@ Welcome to the Multi-component Flow Code (MFC) documentation.
## Development

- @ref contributing "Contributing" - Developer guide and coding standards
- [Pre-Process API](../pre_process/index.html) - Source code reference for mesh generation and initial conditions
- [Simulation API](../simulation/index.html) - Source code reference for the flow solver
- [Post-Process API](../post_process/index.html) - Source code reference for data extraction and visualization
- [API Documentation](../api/index.html) - Source code reference for all three components

## About

Expand Down
3 changes: 3 additions & 0 deletions docs/footer.html
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
<!-- HTML footer for doxygen 1.9.1-->
<!-- start footer part -->
<!--BEGIN GENERATE_TREEVIEW-->
<div id="nav-path" class="navpath">
<ul></ul>
</div>
<!--END GENERATE_TREEVIEW-->
<!--BEGIN !GENERATE_TREEVIEW-->
<!--END !GENERATE_TREEVIEW-->
Expand Down
33 changes: 31 additions & 2 deletions docs/gen_api_landing.py
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
#!/usr/bin/env python3
"""Generate API landing pages for pre_process, simulation, and post_process.
"""Generate API landing pages for MFC documentation.

Usage: python3 gen_api_landing.py [source_dir]
source_dir defaults to current directory.

Scans src/{target}/*.fpp,*.f90 and src/common/*.fpp,*.f90 to produce module
tables in docs/{target}/readme.md. Intro text is defined below per target.
tables in docs/{target}/readme.md. Also generates a unified API landing page
at docs/api/readme.md that links to all three sub-project APIs.
"""

from __future__ import annotations
Expand Down Expand Up @@ -149,3 +150,31 @@ def get_modules(directory: Path) -> list[tuple[str, str]]:
out.parent.mkdir(parents=True, exist_ok=True)
out.write_text("\n".join(lines), encoding="utf-8")
print(f"Generated {out}")

# --- Unified API landing page ---
api_lines = [
"@mainpage API Documentation",
"",
"MFC's source code is organized into three components that form a complete "
"simulation pipeline. Each component has full module-level API documentation.",
"",
]

for target, info in TARGETS.items():
label = info["title"].replace("MFC ", "")
api_lines.append(f"### [{label}](../{target}/index.html)")
api_lines.append("")
api_lines.append(info["intro"])
api_lines.append("")

api_lines.append(
"All three components share a set of **common modules** for MPI communication, "
"variable conversion, derived types, and utility functions. "
"These are documented within each component's API reference."
)
api_lines.append("")

api_out = src_dir / "docs" / "api" / "readme.md"
api_out.parent.mkdir(parents=True, exist_ok=True)
api_out.write_text("\n".join(api_lines), encoding="utf-8")
print(f"Generated {api_out}")
108 changes: 108 additions & 0 deletions docs/gen_architecture.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
#!/usr/bin/env python3
"""Generate architecture.md from template + auto-generated module map.

Usage: python3 gen_architecture.py [source_dir]
source_dir defaults to current directory.

Reads docs/documentation/architecture.md.in as a template, generates the
module map section from docs/module_categories.json and source file briefs,
and writes the final docs/documentation/architecture.md.
"""

from __future__ import annotations

import json
import re
import sys
from pathlib import Path

src_dir = Path(sys.argv[1]) if len(sys.argv) > 1 else Path(".")

_BRIEF_RE = re.compile(r"^!>\s*@brief\s+(.+)", re.IGNORECASE)


def _extract_brief(path: Path) -> str:
"""Extract the module-level @brief from a Fortran source file."""
lines = path.read_text(encoding="utf-8").splitlines()
parts: list[str] = []
in_file_block = False
collecting = False
for line in lines[:40]:
stripped = line.strip()
if stripped.startswith("!>") and not stripped.startswith("!> @brief"):
in_file_block = True
continue
if in_file_block and stripped.startswith("!!"):
continue
if in_file_block:
in_file_block = False
m = _BRIEF_RE.match(stripped)
if m and not collecting:
brief_text = m.group(1).strip()
if re.match(r"Contains\s+(module|program)\s+\w+", brief_text):
continue
parts.append(brief_text)
collecting = True
continue
if collecting:
if stripped.startswith("!!"):
parts.append(stripped.lstrip("! ").strip())
else:
break
if not parts:
return ""
text = " ".join(parts)
dot = text.find(". ")
if dot != -1:
text = text[:dot]
return text.rstrip(". ").strip()


def _find_module_file(name: str) -> Path | None:
"""Find the source file for a module name across all src/ directories."""
for subdir in ("common", "pre_process", "simulation", "post_process"):
for ext in (".fpp", ".f90", ".F90"):
path = src_dir / "src" / subdir / (name + ext)
if path.exists():
return path
return None


def generate_module_map() -> str:
"""Generate the module map markdown from categories + source briefs."""
categories_file = src_dir / "docs" / "module_categories.json"
categories = json.loads(categories_file.read_text(encoding="utf-8"))

lines: list[str] = []
for entry in categories:
cat = entry["category"]
modules = entry["modules"]

lines.append(f"### {cat}")
lines.append("| Module | Role |")
lines.append("|--------|------|")

Comment thread
sbryngelson marked this conversation as resolved.
for mod in modules:
path = _find_module_file(mod)
brief = _extract_brief(path) if path else ""
lines.append(f"| `{mod}` | {brief} |")

lines.append("")

return "\n".join(lines)


def main() -> None:
template = src_dir / "docs" / "documentation" / "architecture.md.in"
output = src_dir / "docs" / "documentation" / "architecture.md"

text = template.read_text(encoding="utf-8")
module_map = generate_module_map()
text = text.replace("<!-- MODULE_MAP -->", module_map)

output.write_text(text, encoding="utf-8")
print(f"Generated {output}")


if __name__ == "__main__":
main()
Loading
Loading