 |
FrontISTR
5.9.0
Large-scale structural analysis program with finit element method
|
|
FrontISTR
5.9.0
Large-scale structural analysis program with finit element method
|
Smoothed Aggregation AMG preconditioner : distributed coarsening. More...
Functions/Subroutines | |
| subroutine, public | hecmw_saamg_coarsen_struct (A, cmt, min_size, max_size, theta, m, aggr, naggr_local, agg_loc, ncnode, cmt_c, my_coarse_off, chalo_gid, agg_order) |
| Uncoupled aggregation + coarse communication table for one distributed level. Inputs: A = rectangular distributed operator (nint*nb rows x nnode*nb cols), cmt = this level's comm table, m = near-kernel size (coarse block size). Outputs: aggr(1:nint) local aggregate id per internal fine node (0 = excluded) naggr_local # aggregates (coarse nodes) owned by this rank agg_loc(1:nnode) localized coarse-node id per fine node: internal fine -> 1..naggr_local, halo fine -> naggr_local+1..ncnode (the coarse halo nodes owned by neighbors), 0 = excluded ncnode total coarse nodes referenced locally (internal + halo) cmt_c coarse communication table (nint=naggr_local, nnode=ncnode) More... | |
| subroutine, public | hecmw_saamg_tentative_ext (cmt, nb, m, bfine_int, aggr, naggr_local, agg_loc, ncnode, phat_ext, bcoarse) |
| Distributed tentative prolongator with halo-extended rows. Builds P-hat over this rank's internal aggregates (per-aggregate QR), then exchanges each fine node's nb x m P-hat block so that halo fine rows carry the owner's block. The result phat_ext (nnode*nb x ncnode*m, columns in the localized coarse numbering) is what the smoother's A*P-hat needs, since A has halo columns. bcoarse (naggr_local*m x m) is the coarse near-kernel for the rows this rank owns. More... | |
| real(kind=kreal) function, public | hecmw_saamg_lambda_max (A, cmt, D, niter) |
| Distributed D-inner-product Lanczos estimate of lambda_max(D^{-1} A) using the halo-aware matvec and globally-reduced (allreduce) inner products, so all ranks agree on one lambda_max (required for a globally consistent prolongator-smoothing omega). More... | |
| subroutine, public | hecmw_saamg_prolongator (A, cmt, m, safety, lanczos_iter, bfine_int, aggr, naggr_local, agg_loc, ncnode, P, bcoarse, omega) |
| Distributed smoothed prolongator P = (I - omega D^{-1} A) P-hat_ext. Builds the halo-extended tentative P-hat, the block-diagonal D of A, the shared lambda_max -> omega, then reuses the sequential smooth_prolongator (which, given the rectangular A with An = nint*nb rows, emits only internal fine rows of P while the SpGEMM A*P-hat_ext correctly sums over halo columns). More... | |
| integer(kind=kint) function, public | hecmw_saamg_nc_global (cmt, naggr_local) |
| Total number of coarse nodes across all ranks (= sum of naggr_local). More... | |
| subroutine, public | hecmw_saamg_galerkin_global (A, cmt, P, naggr_local, ncnode, my_off, chalo_gid, m, Ac_global, fuse_this_level) |
| Redundant Galerkin: gather every rank's partial triplets and sum -> the full global coarse operator (identical on every rank). Used ONLY at the coarsest level now (the distributed levels use hecmw_saamg_galerkin). More... | |
| subroutine, public | hecmw_saamg_galerkin (A, cmt, P, naggr_local, ncnode, my_off, chalo_gid, m, A_next, cmt_op, gplan, mem_level, fuse_this_level) |
| Distributed Galerkin: assemble THIS rank's owned rows of the coarse operator A_next (rectangular naggr_local*m x ncnode_op*m) by routing each rank's partial triplets to the OWNING rank (MPI_Alltoallv; owners may be non-neighbors), summing duplicates; and build the coarse OPERATOR comm table cmt_op from A_next's column sparsity. Each global Ac(I,J) is summed exactly once on its owner. More... | |
| subroutine, public | hecmw_saamg_verify_galerkin (A, cmt, P, naggr_local, ncnode, my_off, chalo_gid, m, reldiff, sym_ok) |
| S1 self-check: build the coarse operator BOTH ways – distributed (hecmw_saamg_galerkin, with row-routing + operator comm table) and redundant (hecmw_saamg_galerkin_global, Allgatherv). Gather the distributed one and confirm it equals the redundant one (probe by matvec on a seeded global coarse vector). Also check the operator comm table is symmetric. More... | |
| subroutine, public | hecmw_saamg_setup (A, cmt, bfine_int, m, prm, dh, fine_matvec, move_in) |
| Build a fully-distributed multilevel SA-AMG hierarchy. bfine_int is the near-kernel on internal nodes, built with a GLOBALLY consistent reference (zero / global centroid) so owner and halo agree on shared nodes. More... | |
| subroutine, public | hecmw_saamg_refresh (dh, A_new) |
| Numeric-only refresh (Newton): reuse each level's aggregation / comm table / tentative P-hat AND the cached Galerkin routing plan (S4), recompute only D / lambda / smoothed P / coarse operator VALUES. No comm-table rebuild: the coarse operator pattern and every level's cmt are untouched, so only the values move (SpGEMM + values-only Alltoallv). More... | |
| subroutine, public | hecmw_saamg_apply (dh, r, z) |
| Apply one distributed multilevel V-cycle: z = M^{-1} r (internal, length nint*nb). More... | |
| subroutine, public | hecmw_saamg_free (dh) |
| subroutine, public | hecmw_saamg_verify_prows (cmt, m, P, naggr_local, my_off, chalo_gid, mism) |
| Halo-exchange the smoothed prolongator's rows (F4c Galerkin needs P rows for halo fine nodes, since A has halo columns). P columns are converted to GLOBAL coarse-dof ids before sending (each rank localizes differently); the owner's row arrives padded to maxe entries per fine row. This routine VERIFIES the row exchange by a round-trip checksum: the owner also sends, as a per-node scalar, sum(P_row^2); the importer compares it to the checksum of the row it received. mism (global max) must be ~0. The packed buffers are the basis of the F4c distributed Galerkin (built next). More... | |
| subroutine, public | hecmw_saamg_verify_commtable (cmt_c, ok) |
| Check coarse comm-table symmetry: for each neighbor, the number of coarse nodes it exports to me must equal the number I import from it. Returns the per-rank result in ok (collective consistency is checked by the caller). More... | |
Smoothed Aggregation AMG preconditioner : distributed coarsening.
MPI-distributed construction of one coarse level. Aggregation is "uncoupled": each rank aggregates only its internal nodes (halo neighbors are dropped by the node-graph builder), so an aggregate never crosses a rank boundary and is owned entirely by one rank. Coarse nodes are globally numbered via allgather offsets; the coarse communication table is derived directly from the fine import/export lists plus the exchanged global aggregate ids – no extra communication.
F4b-1 scope: aggregation + coarse global numbering + coarse comm table. The distributed prolongator (F4b-2) and Galerkin operator (F4c) build on the maps produced here (aggr, agg_loc, the coarse comm table).
| subroutine, public hecmw_precond_saamg_core::hecmw_saamg_apply | ( | type(hecmwst_saamg_hier), intent(inout) | dh, |
| real(kind=kreal), dimension(:), intent(in) | r, | ||
| real(kind=kreal), dimension(:), intent(out) | z | ||
| ) |
Apply one distributed multilevel V-cycle: z = M^{-1} r (internal, length nint*nb).
Definition at line 1478 of file hecmw_precond_SAAMG_core.f90.
| subroutine, public hecmw_precond_saamg_core::hecmw_saamg_coarsen_struct | ( | type(hecmwst_saamg_bcsr), intent(in) | A, |
| type(hecmwst_saamg_comm), intent(in) | cmt, | ||
| integer(kind=kint), intent(in) | min_size, | ||
| integer(kind=kint), intent(in) | max_size, | ||
| real(kind=kreal), intent(in) | theta, | ||
| integer(kind=kint), intent(in) | m, | ||
| integer(kind=kint), dimension(:), intent(out), allocatable | aggr, | ||
| integer(kind=kint), intent(out) | naggr_local, | ||
| integer(kind=kint), dimension(:), intent(out), allocatable | agg_loc, | ||
| integer(kind=kint), intent(out) | ncnode, | ||
| type(hecmwst_saamg_comm), intent(out) | cmt_c, | ||
| integer(kind=kint), intent(out), optional | my_coarse_off, | ||
| integer(kind=kint), dimension(:), intent(out), optional, allocatable | chalo_gid, | ||
| integer(kind=kint), intent(in), optional | agg_order | ||
| ) |
Uncoupled aggregation + coarse communication table for one distributed level. Inputs: A = rectangular distributed operator (nint*nb rows x nnode*nb cols), cmt = this level's comm table, m = near-kernel size (coarse block size). Outputs: aggr(1:nint) local aggregate id per internal fine node (0 = excluded) naggr_local # aggregates (coarse nodes) owned by this rank agg_loc(1:nnode) localized coarse-node id per fine node: internal fine -> 1..naggr_local, halo fine -> naggr_local+1..ncnode (the coarse halo nodes owned by neighbors), 0 = excluded ncnode total coarse nodes referenced locally (internal + halo) cmt_c coarse communication table (nint=naggr_local, nnode=ncnode)
| [out] | my_coarse_off | global coarse-node offset of this rank, and the global coarse id of each halo coarse node (local id naggr_local+t -> chalo_gid(t)). Optional: the F4b checks do not need them; the F4c Galerkin does (local<->global cols). |
| [in] | agg_order | seed-scan ordering (prmagg_order; absent = 1 = BFS default) |
Definition at line 140 of file hecmw_precond_SAAMG_core.f90.
| subroutine, public hecmw_precond_saamg_core::hecmw_saamg_free | ( | type(hecmwst_saamg_hier), intent(inout) | dh | ) |
Definition at line 1585 of file hecmw_precond_SAAMG_core.f90.
| subroutine, public hecmw_precond_saamg_core::hecmw_saamg_galerkin | ( | type(hecmwst_saamg_bcsr), intent(in) | A, |
| type(hecmwst_saamg_comm), intent(in) | cmt, | ||
| type(hecmwst_saamg_bcsr), intent(in) | P, | ||
| integer(kind=kint), intent(in) | naggr_local, | ||
| integer(kind=kint), intent(in) | ncnode, | ||
| integer(kind=kint), intent(in) | my_off, | ||
| integer(kind=kint), dimension(:), intent(in) | chalo_gid, | ||
| integer(kind=kint), intent(in) | m, | ||
| type(hecmwst_saamg_bcsr), intent(out) | A_next, | ||
| type(hecmwst_saamg_comm), intent(out) | cmt_op, | ||
| type(hecmwst_saamg_gplan), intent(out), optional | gplan, | ||
| integer(kind=kint), intent(in), optional | mem_level, | ||
| logical, intent(in), optional | fuse_this_level | ||
| ) |
Distributed Galerkin: assemble THIS rank's owned rows of the coarse operator A_next (rectangular naggr_local*m x ncnode_op*m) by routing each rank's partial triplets to the OWNING rank (MPI_Alltoallv; owners may be non-neighbors), summing duplicates; and build the coarse OPERATOR comm table cmt_op from A_next's column sparsity. Each global Ac(I,J) is summed exactly once on its owner.
| [out] | gplan | S4: record refresh routing |
| [in] | mem_level | >=2 prints [mem] peak probes (default off) |
| [in] | fuse_this_level | Galerkin product form (see prmgalerkin_lowmem) |
Definition at line 654 of file hecmw_precond_SAAMG_core.f90.
| subroutine, public hecmw_precond_saamg_core::hecmw_saamg_galerkin_global | ( | type(hecmwst_saamg_bcsr), intent(in) | A, |
| type(hecmwst_saamg_comm), intent(in) | cmt, | ||
| type(hecmwst_saamg_bcsr), intent(in) | P, | ||
| integer(kind=kint), intent(in) | naggr_local, | ||
| integer(kind=kint), intent(in) | ncnode, | ||
| integer(kind=kint), intent(in) | my_off, | ||
| integer(kind=kint), dimension(:), intent(in) | chalo_gid, | ||
| integer(kind=kint), intent(in) | m, | ||
| type(hecmwst_saamg_bcsr), intent(out) | Ac_global, | ||
| logical, intent(in), optional | fuse_this_level | ||
| ) |
Redundant Galerkin: gather every rank's partial triplets and sum -> the full global coarse operator (identical on every rank). Used ONLY at the coarsest level now (the distributed levels use hecmw_saamg_galerkin).
Definition at line 628 of file hecmw_precond_SAAMG_core.f90.
| real(kind=kreal) function, public hecmw_precond_saamg_core::hecmw_saamg_lambda_max | ( | type(hecmwst_saamg_bcsr), intent(in) | A, |
| type(hecmwst_saamg_comm), intent(in) | cmt, | ||
| type(hecmwst_saamg_blockdiag), intent(in) | D, | ||
| integer(kind=kint), intent(in) | niter | ||
| ) |
Distributed D-inner-product Lanczos estimate of lambda_max(D^{-1} A) using the halo-aware matvec and globally-reduced (allreduce) inner products, so all ranks agree on one lambda_max (required for a globally consistent prolongator-smoothing omega).
Definition at line 368 of file hecmw_precond_SAAMG_core.f90.
| integer(kind=kint) function, public hecmw_precond_saamg_core::hecmw_saamg_nc_global | ( | type(hecmwst_saamg_comm), intent(in) | cmt, |
| integer(kind=kint), intent(in) | naggr_local | ||
| ) |
Total number of coarse nodes across all ranks (= sum of naggr_local).
Definition at line 448 of file hecmw_precond_SAAMG_core.f90.
| subroutine, public hecmw_precond_saamg_core::hecmw_saamg_prolongator | ( | type(hecmwst_saamg_bcsr), intent(in) | A, |
| type(hecmwst_saamg_comm), intent(in) | cmt, | ||
| integer(kind=kint), intent(in) | m, | ||
| real(kind=kreal), intent(in) | safety, | ||
| integer(kind=kint), intent(in) | lanczos_iter, | ||
| real(kind=kreal), dimension(:,:), intent(in) | bfine_int, | ||
| integer(kind=kint), dimension(:), intent(in) | aggr, | ||
| integer(kind=kint), intent(in) | naggr_local, | ||
| integer(kind=kint), dimension(:), intent(in) | agg_loc, | ||
| integer(kind=kint), intent(in) | ncnode, | ||
| type(hecmwst_saamg_bcsr), intent(out) | P, | ||
| real(kind=kreal), dimension(:,:), intent(out), allocatable | bcoarse, | ||
| real(kind=kreal), intent(out) | omega | ||
| ) |
Distributed smoothed prolongator P = (I - omega D^{-1} A) P-hat_ext. Builds the halo-extended tentative P-hat, the block-diagonal D of A, the shared lambda_max -> omega, then reuses the sequential smooth_prolongator (which, given the rectangular A with An = nint*nb rows, emits only internal fine rows of P while the SpGEMM A*P-hat_ext correctly sums over halo columns).
Definition at line 423 of file hecmw_precond_SAAMG_core.f90.
| subroutine, public hecmw_precond_saamg_core::hecmw_saamg_refresh | ( | type(hecmwst_saamg_hier), intent(inout) | dh, |
| type(hecmwst_saamg_bcsr), intent(in) | A_new | ||
| ) |
Numeric-only refresh (Newton): reuse each level's aggregation / comm table / tentative P-hat AND the cached Galerkin routing plan (S4), recompute only D / lambda / smoothed P / coarse operator VALUES. No comm-table rebuild: the coarse operator pattern and every level's cmt are untouched, so only the values move (SpGEMM + values-only Alltoallv).
Definition at line 1425 of file hecmw_precond_SAAMG_core.f90.
| subroutine, public hecmw_precond_saamg_core::hecmw_saamg_setup | ( | type(hecmwst_saamg_bcsr), intent(inout) | A, |
| type(hecmwst_saamg_comm), intent(in) | cmt, | ||
| real(kind=kreal), dimension(:,:), intent(in) | bfine_int, | ||
| integer(kind=kint), intent(in) | m, | ||
| type(hecmwst_saamg_params), intent(in) | prm, | ||
| type(hecmwst_saamg_hier), intent(out) | dh, | ||
| procedure(hecmw_saamg_matvec_if), optional | fine_matvec, | ||
| logical, intent(in), optional | move_in | ||
| ) |
Build a fully-distributed multilevel SA-AMG hierarchy. bfine_int is the near-kernel on internal nodes, built with a GLOBALLY consistent reference (zero / global centroid) so owner and halo agree on shared nodes.
| [in,out] | a | moved into the hierarchy if move_in |
| fine_matvec | fast finest matvec (HEC-MW) | |
| [in] | move_in | take ownership of A (move, no copy) |
Definition at line 1131 of file hecmw_precond_SAAMG_core.f90.
| subroutine, public hecmw_precond_saamg_core::hecmw_saamg_tentative_ext | ( | type(hecmwst_saamg_comm), intent(in) | cmt, |
| integer(kind=kint), intent(in) | nb, | ||
| integer(kind=kint), intent(in) | m, | ||
| real(kind=kreal), dimension(:,:), intent(in) | bfine_int, | ||
| integer(kind=kint), dimension(:), intent(in) | aggr, | ||
| integer(kind=kint), intent(in) | naggr_local, | ||
| integer(kind=kint), dimension(:), intent(in) | agg_loc, | ||
| integer(kind=kint), intent(in) | ncnode, | ||
| type(hecmwst_saamg_bcsr), intent(out) | phat_ext, | ||
| real(kind=kreal), dimension(:,:), intent(out), allocatable | bcoarse | ||
| ) |
Distributed tentative prolongator with halo-extended rows. Builds P-hat over this rank's internal aggregates (per-aggregate QR), then exchanges each fine node's nb x m P-hat block so that halo fine rows carry the owner's block. The result phat_ext (nnode*nb x ncnode*m, columns in the localized coarse numbering) is what the smoother's A*P-hat needs, since A has halo columns. bcoarse (naggr_local*m x m) is the coarse near-kernel for the rows this rank owns.
Definition at line 310 of file hecmw_precond_SAAMG_core.f90.
| subroutine, public hecmw_precond_saamg_core::hecmw_saamg_verify_commtable | ( | type(hecmwst_saamg_comm), intent(in) | cmt_c, |
| logical, intent(out) | ok | ||
| ) |
Check coarse comm-table symmetry: for each neighbor, the number of coarse nodes it exports to me must equal the number I import from it. Returns the per-rank result in ok (collective consistency is checked by the caller).
Definition at line 1689 of file hecmw_precond_SAAMG_core.f90.
| subroutine, public hecmw_precond_saamg_core::hecmw_saamg_verify_galerkin | ( | type(hecmwst_saamg_bcsr), intent(in) | A, |
| type(hecmwst_saamg_comm), intent(in) | cmt, | ||
| type(hecmwst_saamg_bcsr), intent(in) | P, | ||
| integer(kind=kint), intent(in) | naggr_local, | ||
| integer(kind=kint), intent(in) | ncnode, | ||
| integer(kind=kint), intent(in) | my_off, | ||
| integer(kind=kint), dimension(:), intent(in) | chalo_gid, | ||
| integer(kind=kint), intent(in) | m, | ||
| real(kind=kreal), intent(out) | reldiff, | ||
| logical, intent(out) | sym_ok | ||
| ) |
S1 self-check: build the coarse operator BOTH ways – distributed (hecmw_saamg_galerkin, with row-routing + operator comm table) and redundant (hecmw_saamg_galerkin_global, Allgatherv). Gather the distributed one and confirm it equals the redundant one (probe by matvec on a seeded global coarse vector). Also check the operator comm table is symmetric.
Definition at line 825 of file hecmw_precond_SAAMG_core.f90.
| subroutine, public hecmw_precond_saamg_core::hecmw_saamg_verify_prows | ( | type(hecmwst_saamg_comm), intent(in) | cmt, |
| integer(kind=kint), intent(in) | m, | ||
| type(hecmwst_saamg_bcsr), intent(in) | P, | ||
| integer(kind=kint), intent(in) | naggr_local, | ||
| integer(kind=kint), intent(in) | my_off, | ||
| integer(kind=kint), dimension(:), intent(in) | chalo_gid, | ||
| real(kind=kreal), intent(out) | mism | ||
| ) |
Halo-exchange the smoothed prolongator's rows (F4c Galerkin needs P rows for halo fine nodes, since A has halo columns). P columns are converted to GLOBAL coarse-dof ids before sending (each rank localizes differently); the owner's row arrives padded to maxe entries per fine row. This routine VERIFIES the row exchange by a round-trip checksum: the owner also sends, as a per-node scalar, sum(P_row^2); the importer compares it to the checksum of the row it received. mism (global max) must be ~0. The packed buffers are the basis of the F4c distributed Galerkin (built next).
Definition at line 1629 of file hecmw_precond_SAAMG_core.f90.
1.9.1