大网格-奇怪报错
-
还是在算圆柱绕流。为了进入critical雷诺数，网格加密了很多，导致的问题也很多。192核那个例子的网格数在600W，运行没有什么问题。
可是在其他条件相同，但是用更多的网格的288核算例中出现奇怪的错误。读取初始和边界条件，湍流模型都没有什么问题，就是到 smoothsovler 那里出现了问题。
288核这个case，网格数在3千万。
个人感觉可能是内存的问题，但是我们这的计算单个节点内存是32GB，照理说12个节点的总内存量应该是能应付的？难道是计算到smoothsovler这一步，在某一个节点的内存量大大超过了32GB？
192 正常运算
Create time Create mesh for time = 300 PISO: Operating solver in PISO mode Reading field p Reading field U Reading/calculating face flux field phi Selecting incompressible transport model Newtonian Selecting turbulence model type LES Selecting LES turbulence model dynamicKEqn Selecting LES delta type cubeRootVol bounding k, min: 0 max: 0.0015093 average: 1.99878e-05 dynamicKEqnCoeffs { filter simple; Ce 1.048; } No MRF models present No finite volume options present Time = 300.02 Courant Number mean: 0.026086 max: 0.983623 smoothSolver: Solving for Ux, Initial residual = 0.000816211, Final residual = 2.84247e-08,\ No Iterations 3 smoothSolver: Solving for Uy, Initial residual = 0.00267408, Final residual = 2.09581e-08, \ No Iterations 4 smoothSolver: Solving for Uz, Initial residual = 0.00457375, Final residual = 3.57969e-08, \ No Iterations 4 GAMG: Solving for p, Initial residual = 0.220597, Final residual = 0.00856105, No Iteration\ s 3 time step continuity errors : sum local = 6.24882e-08, global = 5.53688e-10, cumulative = 5.\ 53688e-10 GAMG: Solving for p, Initial residual = 0.00891699, Final residual = 0.000438876, No Iterat\ ions 12 time step continuity errors : sum local = 3.18791e-09, global = -1.1464e-10, cumulative = 4.\ 39048e-10 GAMG: Solving for p, Initial residual = 0.000445623, Final residual = 9.53731e-08, No Itera\ tions 25 time step continuity errors : sum local = 6.89127e-13, global = -7.61126e-14, cumulative = 4\ .38972e-10 smoothSolver: Solving for k, Initial residual = 0.000553063, Final residual = 3.40938e-08, \ No Iterations 3 bounding k, min: -3.68878e-09 max: 0.00149781 average: 1.99956e-05 ExecutionTime = 6.64 s ClockTime = 7 s288 核 异常错误
Create time Create mesh for time = 0 PISO: Operating solver in PISO mode Reading field p Reading field U Reading/calculating face flux field phi Selecting incompressible transport model Newtonian Selecting turbulence model type LES Selecting LES turbulence model dynamicKEqn Selecting LES delta type cubeRootVol bounding k, min: 0 max: 2e-05 average: 0 dynamicKEqnCoeffs { filter simple; Ce 1.048; } No MRF models present No finite volume options present Starting time loop Time = 0.005 Courant Number mean: 0.000119441 max: 0.017568 smoothSolver: Solving for Ux, Initial residual = 1, Final residual = 2.06726e-09, No Iterat\ ions 2 smoothSolver: Solving for Uy, Initial residual = 1, Final residual = 1.30712e-09, No Iterat\ ions 2 smoothSolver: Solving for Uz, Initial residual = 1, Final residual = 1.31187e-09, No Iterat\ ions 2 GAMG: Solving for p, Initial residual = 1, Final residual = 0.035832, No Iterations 36 time step continuity errors : sum local = 8.55925e-06, global = -9.25811e-08, cumulative = -\ 9.25811e-08 GAMG: Solving for p, Initial residual = 0.000128631, Final residual = 6.05672e-06, No Itera\ tions 39 time step continuity errors : sum local = 4.27446e-07, global = -5.85326e-09, cumulative = -\ 9.84343e-08 GAMG: Solving for p, Initial residual = 8.87864e-06, Final residual = 9.28164e-08, No Itera\ tions 74 time step continuity errors : sum local = 6.57138e-09, global = 9.95505e-13, cumulative = -9\ .84333e-08 -------------------------------------------------------------------------- mpiexec noticed that process rank 105 with PID 0 on node cp1056 exited on signal 8 (Floating\ point exception). --------------------------------------------------------------------------另外的报错信息：
An MPI process has executed an operation involving a call to the "fork()" system call to create a child process. Open MPI is currently operating in a condition that could result in memory corruption or other system errors; your MPI job may hang, crash, or produce silent data corruption. The use of fork() (or system() or other calls that create child processes) is strongly discouraged. The process that invoked fork was: Local host: cp1815 (PID 31613) MPI_COMM_WORLD rank: 89 If you are *absolutely sure* that your application will successfully and correctly survive a call to fork(), you may disable this warning by setting the mpi_warn_on_fork MCA parameter to 0. -------------------------------------------------------------------------- -------------------------------------------------------------------------- mpiexec noticed that process rank 105 with PID 0 on node cp1652 exited on signal 8 (Floating\ point exception).我查找了相关信息，在提交作业的脚本文件做了相应的改变：
mpiexec --mca mpi_warn_on_fork 0 -n 288 pisoFoam -parallel > simpleFoam_${PBS_JOBID}.out还是不能计算
Time = 0.005 Courant Number mean: 0.000119441 max: 0.017568 smoothSolver: Solving for Ux, Initial residual = 1, Final residual = 2.06726e-09, No Iterat\ ions 2 smoothSolver: Solving for Uy, Initial residual = 1, Final residual = 1.30712e-09, No Iterat\ ions 2 smoothSolver: Solving for Uz, Initial residual = 1, Final residual = 1.31187e-09, No Iterat\ ions 2 GAMG: Solving for p, Initial residual = 1, Final residual = 0.035832, No Iterations 36 time step continuity errors : sum local = 8.55925e-06, global = -9.25811e-08, cumulative = -\ 9.25811e-08 GAMG: Solving for p, Initial residual = 0.000128631, Final residual = 6.05672e-06, No Itera\ tions 39 time step continuity errors : sum local = 4.27446e-07, global = -5.85326e-09, cumulative = -\ 9.84343e-08 GAMG: Solving for p, Initial residual = 8.87864e-06, Final residual = 9.28164e-08, No Itera\ tions 74 time step continuity errors : sum local = 6.57138e-09, global = 9.95505e-13, cumulative = -9\ .84333e-08 -------------------------------------------------------------------------- mpiexec noticed that process rank 105 with PID 0 on node cp1056 exited on signal 8 (Floating\ point exception). --------------------------------------------------------------------------还请有经验的朋友分析一下这个问题。
-
Sorry, Chinese input missing.
mpiexec noticed that process rank 105 with PID 0 on node cp1056 exited on signal 8 (Floating\ point exception).This is a quite general error, which does not mean it comes from setting up of mpi. Meanwhile, sometimes your simulation blows up in parallel running, see: link text.
Which version of OpenFOAM do you use? Could u please try
PBiCGinstead ofsmoothSolver? How do you decompose your grid? -
@cfd-china
补充信息：OpenFOAM v4.1
/*--------------------------------*- C++ -*----------------------------------*\ | ========= | | | \\ / F ield | OpenFOAM: The Open Source CFD Toolbox | | \\ / O peration | Version: 4.x | | \\ / A nd | Web: www.OpenFOAM.org | | \\/ M anipulation | | \*---------------------------------------------------------------------------*/ FoamFile { version 2.0; format ascii; class dictionary; location "system"; object decomposeParDict; } // * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * // numberOfSubdomains 288; method hierarchical; hierarchicalCoeffs { n (16 9 2); delta 0.001; order xyz; hierarchicalCoeffs { n (16 9 2); delta 0.001; order xyz; } manualCoeffs { dataFile "cellDecomposition"; } // ************************************************************************* //*--------------------------------*- C++ -*----------------------------------*\ | ========= | | | \\ / F ield | OpenFOAM: The Open Source CFD Toolbox | | \\ / O peration | Version: 4.x | | \\ / A nd | Web: www.OpenFOAM.org | | \\/ M anipulation | | \*---------------------------------------------------------------------------*/ FoamFile { version 2.0; format ascii; class dictionary; location "system"; object fvSolution; } // * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * // solvers { p { solver GAMG; tolerance 1e-07; relTol 0.05; smoother GaussSeidel; } pFinal { $p; smoother DICGaussSeidel; tolerance 1e-07; relTol 0; } "(U|k|B|nuTilda|s)" { solver smoothSolver; smoother GaussSeidel; tolerance 1e-07; relTol 0; } } PISO { nCorrectors 3; nNonOrthogonalCorrectors 0; } relaxationFactors { "U.*" 1; "nuTilda.*" 1; } -
再补充：
192 case 的雷诺数在10^4
288 case 的在10^5这两个case的分解方法，controDict, fvSchemes, fvSolution 基本一样。
不同的是288 case 的时间步长更小而已。 -
k 文件 tubulent kinetic energy：
*--------------------------------*- C++ -*----------------------------------*\ | ========= | | | \\ / F ield | OpenFOAM: The Open Source CFD Toolbox | | \\ / O peration | Version: 4.x | | \\ / A nd | Web: www.OpenFOAM.org | | \\/ M anipulation | | \*---------------------------------------------------------------------------*/ FoamFile { version 2.0; format ascii; class volScalarField; object k; } // * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * // dimensions [0 2 -2 0 0 0 0]; internalField uniform 0; boundaryField { INLET { type fixedValue; value uniform 2e-05; } OUTLET { type inletOutlet; inletValue uniform 0; value uniform 0; } BACK_CYC { type cyclic; } FRONT_CYC { type cyclic; } CYLINDER { type fixedValue; value uniform 0; } } -
@cfd-china
非常感谢你的建议！非常管用，起码现在能跑起来了。
Time = 0.005 Courant Number mean: 0.000119441 max: 0.017568 DILUPBiCG: Solving for Ux, Initial residual = 1, Final residual = 2.34151e-11, No Iterations 2 DILUPBiCG: Solving for Uy, Initial residual = 1, Final residual = 1.61665e-10, No Iterations 2 DILUPBiCG: Solving for Uz, Initial residual = 1, Final residual = 1.63882e-10, No Iterations 2 GAMG: Solving for p, Initial residual = 1, Final residual = 0.035832, No Iterations 36 time step continuity errors : sum local = 8.55925e-06, global = -9.25811e-08, cumulative = -9.25811e-08 GAMG: Solving for p, Initial residual = 0.000128631, Final residual = 6.05672e-06, No Iterations 39 time step continuity errors : sum local = 4.27446e-07, global = -5.85326e-09, cumulative = -9.84343e-08 GAMG: Solving for p, Initial residual = 8.87864e-06, Final residual = 9.28164e-08, No Iterations 74 time step continuity errors : sum local = 6.57138e-09, global = 9.95505e-13, cumulative = -9.84333e-08 DILUPBiCG: Solving for k, Initial residual = 0.297752, Final residual = 7.56559e-07, No Iterations 4 bounding k, min: -2.24189e-08 max: 2e-05 average: 2.54114e-09 ExecutionTime = 42.97 s ClockTime = 43 s我改动的fvSolution 部分是：
"(U|k|B|nuTilda|s)" { solver PBiCG; preconditioner DILU tolerance 1e-07; relTol 0; }目前运行了半个小时了。
我知道后面还会有的新的问题。