Scalability studies and computations using the largest grids to date for free-surface flows are performed using message-passing interface (MPI)-based CFDShip-Iowa toolbox curvilinear (V4) and Cartesian (V6) grid solvers on Navy high-performance computing systems. Both solvers show good strong scalability up to 2048 processors, with V6 showing somewhat better performance than V4. V6 also outperforms V4 in terms of the memory requirements and central processing unit (CPU) time per time-step per grid point. The explicit solvers show better scalability than the implicit solvers, but the latter allows larger time-step sizes, resulting in a lower total CPU time. The multi-grid HYPRE solver shows better scalability than the portable, extensible toolkit for scientific computation solver. The main scalability bottleneck is identified to be the pressure Poisson solver. The memory bandwidth test suggests that further scalability improvements could be obtained by using hybrid MPI/open multi-processing (OpenMP) parallelization. V4-detached eddy simulation (DES) on a 300 M grid for the surface combatant model DTMB 5415 in the straight-ahead condition provides a plausible description of the vortical structures and mean flow patterns observed in the experiments. However, the vortex strengths are over predicted and the turbulence is not resolved. V4-DESs on up to 250 M grids for DTMB 5415 at 20° static drift angle significantly improve the forces and moment predictions compared to the coarse grid unsteady Reynolds averaged Navier–Stokes, due to the improved resolved turbulence predictions. The simulations provide detailed resolution of the free-surface and breaking pattern and vortical and turbulent structures, which will guide planned experiments. V6 simulations on up to 276 M grids for DTMB 5415 in the straight-ahead condition predict diffused vortical structures due to poor wall-layer predictions. This could be due to the limitations of the wall-function implementation for the immersed boundary method.
BalaySBuschelmanKGroppWKaushikDKnepleyMCurfmanL. (2002) PETSc User Manual, ANL-95/11, Revision 2.1.5. Argonne National Laboratory, USA.
2.
BhushanSXingTCarricaPMSternF (2009) Model- and full-Scale URANS simulations of Athena resistance, powering and seakeeping, and 5415 maneuvering. Journal of Ship Research53(4): 179–198.
3.
CarricaPMHuangJNoackRKaushikDSmithBSternF (2010) Large-scale DES computations of the forward speed diffraction and pitch and heave problems for a surface combatant. Computers and Fluids39(7): 1095–1111.
4.
CarricaPMPaikKJHosseiniHSSternF (2008) URANS analysis of a broaching event in irregular quartering seas. Journal of Marine Science and Technology13(4): 395–407.
5.
CarricaPMWilsonRVNoackRSternF (2007a) Ship motions using single-level set with dynamic overset grids. Computers and Fluids36(9): 1415–1433.
6.
CarricaPMWilsonRVSternF (2007b) An unsteady single-phase level set method for viscous free surface flows. International Journal of Numerical Methods in Fluids53(2): 229–256.
7.
ChorleyMJWalkerDWGuestMF (2009) Hybrid message-passing and shared-memory programming in a molecular dynamics application on multicore clusters. The International Journal of High Performance Computing Applications23(3): 196–211.
8.
CohenJMolemakerJ (2009) A fast double precision CFD code using CUDA. In: Proceedings of the 21st Parallel CFD Conference, 18–22 May, Moffett Field, CA, 252–256.
9.
CuicchiC (2009) Navy DSRC retrospective: 1990-2009. HPC Insights, Spring Issue.
10.
DongarraJGannonDFoxGKennedK (2007) The impact of multicore on computational science software. CTWatch Quarterly3: 3–10.
11.
FalgoutRDJonesJEYangUM (2006) The design and implementation of HYPRE, library of parallel high performance preconditioners. In: BruasetAMTveitoA (eds) Numerical Solution of Partial Differential Equations on Parallel Computers. Springer-Verlag, Netherlands51: 267–294.
12.
FurebyC (2008) Large eddy simulation of ship hydrodynamics. In: Proceedings of the 27th Symposium on Naval Hydrodynamics, 5–10 October, Seoul, Korea.
13.
GicquelLYMStaffelbachGCuenotBPoinsotT (2008) Large eddy simulations of turbulent reacting flows in real burners: the status and challenges. Journal of Physics: Conference Series125: 012029.
14.
HinoT (2005) Proceedings of the CFD Workshop Tokyo2005, National Maritime Research Institute, Tokyo.
15.
HuangJCarricaPMSternF (2007) Coupled ghost fluid/two-phase level set method for curvilinear body-fitted grids. International Journal Numerical Methods Fluids55: 867–897.
16.
HuangJCarricaPMSternF (2008) Semi-coupled air/water immersed boundary approach for curvilinear dynamic overset grids with application to ship hydrodynamics. International Journal Numerical Methods Fluids58: 591–624.
17.
IsmailFCarricaPMXingTSternF (2010) Evaluation of linear and non-linear convection schemes on multidimensional non-orthogonal grids with applications to KVLCC2 tanker. International Journal for Numerical Methods in Fluids, 64(8): 850–886.
18.
IssaRI (1985) Solution of the implicit discretised fluid flow equations by operator splitting. Journal of Computational Physics62: 40–65.
19.
KalitzinGMedicGIaccarinoGDurbinP (2005) Near-wall behavior of RANS turbulence models and implications for wall functions. Journal of Computational Physics204: 265–291.
20.
KaushikDBalaySKeyesDSmithB (2009) Understanding the performance of hybrid MPI/OpenMP programming model for implicit CFD codes. In: Proceedings of the 21st Parallel CFD Conference, 18–22 May, Moffett Field, CA, 174–177.
21.
KeatingAPriscoGDPiomelliU (2006) Interface conditions for hybrid RANS/LES calculations. International Journal of Heat and Fluid Flow27: 777–788.
22.
KremenetskyMKodiyalamS (2009) Parallel performance of CFD algorithm and the ubiquitous need for HPC with high fidelity, multidisciplinary analysis and optimization (MDO). In: Proceedings of the 21st Parallel CFD Conference, 18–22 May, Moffett Field, CA, 338–342.
23.
LarssonLSternFBertramV (2000) Proceedings of Gothenburg 2000- A workshop on numerical ship hydrodynamics, Chalmers University of Technology, Gothenburg, Sweden.
24.
LignellDYooCChenJSankaranRFaheyM (2007) S3D: Petascale combustion science, performance, and optimization. In: Proceedings of the Cray Scaling Workshop, Oak Ridge National Laboratory, TN.
25.
LimJStrawnRC (2007) Prediction of HART II rotor BVI loading and wake system using CFD/CSD loose coupling, AIAA paper 2007-1281. In: Proceedings of the 45th AIAA Aerospace Sci. Meeting and Exhibit, Reno, NV.
26.
LongoJShaoJIrvineMSternF (2007) Phase-averaged PIV for the nominal wake of a surface ship in regular head waves. Journal of Fluids Engineering129(5): 524–540.
27.
MenterFR (1994) Two equation eddy viscosity turbulence models for engineering applications. AIAA Journal32(8): 1598–1605.
28.
Mohd-YusofJLivescuDKellyT (2009) Adapting the CFDNS Compressible Navier-Stokes Solver to the Roadrunner Hybrid Supercomputer. In: Proceedings of the 21st Parallel CFD Conference, 18–22 May, Moffett Field, CA, 64–67.
29.
NoackR (2005) SUGGAR: a general capability for moving body overset grid assembly. In: Proceedings of the 17th AIAA Computational Fluid Dynamics Conference, Toronto.
30.
OedW (2009) Scaling applications on Cray supercomputers. In: Proceedings of the T-Systems Solutions for Research GmbH HPCN-Workshop, 14–15 May, Germany.
31.
OlivieriAPistaniFAvanziniASternFPennaR (2001) Towing tank experiments of resistance, sinkage and trim, boundary layer, wake, and free surface flow around a naval combatant INSEAN 2340 model. IIHR Technical Report: 421.
32.
OlivieriAPistaniFWilsonRCampanaEFSternF (2007) Scars and vortices induced by ship bow and shoulder wave breaking. Journal of Fluids Engineering129(11): 1445–1459.
33.
ParCFD. (2009). Proceedings of 21st International Conference on Parallel Computational Fluid Dynamics, 18–22 May, Moffett Field, CA.
34.
PulliamTHJespersenDC (2009) Large scale aerodynamic calculations on pleiades. In: Proceedings of the 21st Parallel CFD Conference, 18–22 May, Moffett Field, CA, 73–77.
35.
RabenseifnerRWelleinG (2003) Communication and optimization aspects of parallel programming models on hybrid architectures. The International Journal of High Performance Computing Applications17(1): 49–62.
36.
SahniOZhouMShephardMSJansenKE (2009) Scalable implicit finite element solver for massively parallel processing with demonstration to 160K cores. In: Proceedings of the International Conference of High Performance Computing, SC09, 14–20 November, Portland, OR.
37.
SainiSJespersenDCTalcottDDjomehriJSandstromT (2008). Performance comparison of SGI Altix 4700 and SGI Altix 300 Bx2. NAS Technical Report NAS-01-2001.
38.
SakamotoN (2009) URANS and DES simulations of static and dynamic maneuvering for surface combatant. PhD thesis, The University of Iowa.
39.
SanmiguelR-EOrtegaC-JFernandezF-R (2003) On the efficiency of a numerical method with periodic time strides for solving incompressible flows. Journal of Computational Physics186: 212–229.
40.
ShalfJ (2009) Hardware trends. Panel discussion presentation. In: Proceedings of the 21st Parallel CFD Conference, 18–22 May, Moffett Field, CA.
41.
SpalartPR (2009) Detached-eddy simulation. Annual Review of Fluid Mechanics41: 181–202.
42.
SternFAgdrupC (2008) Proceedings of SIMMAN 2008 Workshop on Verification and Validation of Ship Maneuvering Simulation Methods, Lingby, Denmark.
43.
SternFBhushanSCarricaPYangJ (2009) Large scale parallel computing and scalability study for surface combatant static maneuver and straight ahead conditions using CFDShip-Iowa. In: Proceedings of the 21st Parallel CFD Conference, 18–22 May, Moffett Field, CA, 52–56.
44.
SternFWilsonRShaoJ (2006) Quantitative approach to V&V of CFD simulations and certification of CFD codes. International Journal of Computational Fluid Dynamics50: 1335–1355.
45.
ThibaultJCSenocakI (2009) CUDA Implementation of a Navier-Stokes solver on multi-GPU desktop platforms for incompressible flows. In: Proceedings of the 47th AIAA Aerospace Sciences Meeting, 5–8 January, Orlando, FL.
WallinSJohanssonA (2000) An explicit algebraic Reynolds stress model for incompressible and compressible turbulent flows. Journal of Fluid Mechanics403: 89–132.
48.
WangZYangJSternF (2009a) An improved particle correction procedure for the particle level set method. Journal of Computational Physics228(16): 5819–5837.
49.
WangZYangJKooBSternF (2009b) A coupled level set and volume-of-fluid method for sharp interface simulation of plunging breaking waves. International Journal of Multiphase Flow35(3): 227–246.
50.
XingTCarricaPMSternF (2008) Computational towing tank procedures for single run curves of resistance and propulsion. Journal of Fluids Engineering130(101102): 1–14.
51.
XingTKandasamyMSternF (2007a) Unsteady free-surface wave-induced separation: analysis of turbulent structures using DES and single-phase level-set. Journal of Turbulence8(44): 1–35.
52.
XingTShaoJSternF (2007b) BKW-RS-DES of unsteady vortical flow for KVLCC2 at large drift angles. In: Proceedings of the 9th International Conference on Numerical Ship Hydrodynamics, 5–8 August, Ann Arbor, MI, 187–205.
53.
XingTSternF (2010) Factors of safety for Richardson extrapolation for industrial applications. Journal of Fluids Engineering132(061403): 1–13.
54.
YangJBhushanSSuhJSWangZKooBSakamotoN. (2008) Large-eddy simulation of ship flows with wall-layer models on Cartesian grids. In: Proceedings of the 27th Symposium on Naval Hydrodynamics, 5–10 October, Seoul, Korea.
55.
YangJMichaelTBhushanSHanaokaAWangZSternF (2010) Motion prediction using wall-resolved and wall-modeled approaches on a Cartesian grid. In: Proceedings of the 28th Symposium on Naval Hydrodynamics, 12–17 September, Pasadena, CA.
56.
YangJSternF (2009) Sharp interface immersed-boundary/level-set methods for wave-body interactions. Journal of Computational Physics228(17): 6590–6616.
57.
YoonHS (2009) Force/moment and phase-averaged stereo PIV flow measurements for surface combatant in PMM maneuvers. PhD thesis, The University of Iowa.
