Unable to successfully call PETSc function 'KSPSolve'

Hi everyone.

  1. The code run but the process finished with exit code 1 as below. Line 167 is the last line of the code which is my_model.run(). I would like to present a MWE to reproduce this error but failed. What can I do to solve this problem?
    Error information:
Traceback (most recent call last):
  File "/mnt/d/Pycharm/Projects/pythonProject/interface conditions paper/iter_2d_case.py", line 167, in <module>
    my_model.run()
  File "/home/xinshen/.local/lib/python3.10/site-packages/festim/generic_simulation.py", line 351, in run
    self.run_transient()
  File "/home/xinshen/.local/lib/python3.10/site-packages/festim/generic_simulation.py", line 372, in run_transient
    self.iterate()
  File "/home/xinshen/.local/lib/python3.10/site-packages/festim/generic_simulation.py", line 400, in iterate
    self.h_transport_problem.update(self.t, self.dt)
  File "/home/xinshen/.local/lib/python3.10/site-packages/festim/h_transport_problem.py", line 250, in update
    nb_it, converged = self.solve_once()
  File "/home/xinshen/.local/lib/python3.10/site-packages/festim/h_transport_problem.py", line 288, in solve_once
    nb_it, converged = solver.solve()
RuntimeError: 

*** -------------------------------------------------------------------------
*** DOLFIN encountered an error. If you are not able to resolve this issue
*** using the information listed below, you can ask for help at
***
***     https://fenicsproject.discourse.group/
***
*** Remember to include the error message listed below and, if possible,
*** include a *minimal* running example to reproduce the error.
***
*** -------------------------------------------------------------------------
*** Error:   Unable to successfully call PETSc function 'KSPSolve'.
*** Reason:  PETSc error code is: 76 (Error in external library).
*** Where:   This error was encountered inside ./dolfin/la/PETScKrylovSolver.cpp.
*** Process: 0
*** 
*** DOLFIN version: 2019.2.0.13.dev0
*** Git changeset:  ubuntu
*** -------------------------------------------------------------------------

Thank you!

Hi Xin, can you provide a code that we can run, even if it’s not a MWE. It’s impossible to help you without a runnable script

@xinshen I have managed to reproduce your issue with the mesh you sent me.

The full error message is:

Succesfully load mesh with 845815 cells
/home/remidm/FESTIM/festim/stepsize.py:54: DeprecationWarning: stepsize_stop_max and t_stop attributes will be deprecated in a future release, please use max_stepsize instead
  warnings.warn(
/home/remidm/FESTIM/festim/exports/derived_quantities/derived_quantities.py:64: DeprecationWarning: The derived_quantities attribute will be deprecated in a future release, please use festim.DerivedQuantities as a list instead
  warnings.warn(
Calling FFC just-in-time (JIT) compiler, this may take some time.
/home/remidm/FESTIM/festim/materials/materials.py:151: UserWarning: rho key will be ignored
  warnings.warn(key + " key will be ignored", UserWarning)
/home/remidm/FESTIM/festim/materials/materials.py:151: UserWarning: heat_capacity key will be ignored
  warnings.warn(key + " key will be ignored", UserWarning)
/home/remidm/FESTIM/festim/materials/materials.py:169: UserWarning: thermal_cond key will be ignored
  warnings.warn("thermal_cond key will be ignored", UserWarning)
Calling FFC just-in-time (JIT) compiler, this may take some time.
No Jacobian form specified for nonlinear variational problem.
Differentiating residual form F to obtain Jacobian J = F'.
Calling FFC just-in-time (JIT) compiler, this may take some time.
Calling FFC just-in-time (JIT) compiler, this may take some time.
Calling FFC just-in-time (JIT) compiler, this may take some time.
Calling FFC just-in-time (JIT) compiler, this may take some time.
Solving nonlinear variational problem.
  Newton iteration 0: r (abs) = 0.000e+00 (tol = 1.000e-10) r (rel) = -nan (tol = 1.000e-09)
  Newton solver finished in 0 iterations and 0 linear solver iterations.
No Jacobian form specified for nonlinear variational problem.
Differentiating residual form F to obtain Jacobian J = F'.
Calling FFC just-in-time (JIT) compiler, this may take some time.
Calling FFC just-in-time (JIT) compiler, this may take some time.
Solving nonlinear variational problem.
  Newton iteration 0: r (abs) = 1.930e-08 (tol = 1.000e-10) r (rel) = 1.000e+00 (tol = 1.000e-09)
  Newton iteration 1: r (abs) = 0.000e+00 (tol = 1.000e-10) r (rel) = 0.000e+00 (tol = 1.000e-09)
  Newton solver finished in 1 iterations and 1 linear solver iterations.
Calling FFC just-in-time (JIT) compiler, this may take some time.
Defining initial values
Defining variational problem
Defining source terms
Defining boundary conditions
Time stepping...
Calling FFC just-in-time (JIT) compiler, this may take some time.
  WARNING: The number of integration points for each cell will be: 125
           Consider using the option 'quadrature_degree' to reduce the number of points
Calling FFC just-in-time (JIT) compiler, this may take some time.
Calling FFC just-in-time (JIT) compiler, this may take some time.
Calling FFC just-in-time (JIT) compiler, this may take some time.
  WARNING: The number of integration points for each cell will be: 125
           Consider using the option 'quadrature_degree' to reduce the number of points
Solving nonlinear variational problem.
  Newton iteration 0: r (abs) = 4.491e+25 (tol = 1.000e+11) r (rel) = 1.000e+00 (tol = 1.000e-10)

UMFPACK V5.7.9 (Oct 20, 2019): ERROR: out of memory

Traceback (most recent call last):
  File "/home/remidm/FESTIM/xin.py", line 185, in <module>
    my_model.run()
  File "/home/remidm/FESTIM/festim/generic_simulation.py", line 365, in run
    self.run_transient()
  File "/home/remidm/FESTIM/festim/generic_simulation.py", line 388, in run_transient
    self.iterate()
  File "/home/remidm/FESTIM/festim/generic_simulation.py", line 416, in iterate
    self.h_transport_problem.update(self.t, self.dt)
  File "/home/remidm/FESTIM/festim/h_transport_problem.py", line 250, in update
    nb_it, converged = self.solve_once()
                       ^^^^^^^^^^^^^^^^^
  File "/home/remidm/FESTIM/festim/h_transport_problem.py", line 288, in solve_once
    nb_it, converged = solver.solve()
                       ^^^^^^^^^^^^^^
RuntimeError: 

*** -------------------------------------------------------------------------
*** DOLFIN encountered an error. If you are not able to resolve this issue
*** using the information listed below, you can ask for help at
***
***     fenics-support@googlegroups.com
***
*** Remember to include the error message listed below and, if possible,
*** include a *minimal* running example to reproduce the error.
***
*** -------------------------------------------------------------------------
*** Error:   Unable to successfully call PETSc function 'KSPSolve'.
*** Reason:  PETSc error code is: 76 (Error in external library).
*** Where:   This error was encountered inside /home/conda/feedstock_root/build_artifacts/fenics-pkgs_1696906506503/work/dolfin/dolfin/la/PETScKrylovSolver.cpp.
*** Process: 0
*** 
*** DOLFIN version: 2019.1.0
*** Git changeset:  2e001bd1aae8e14d758264f77382245e6eed04b0
*** -------------------------------------------------------------------------

As indicated by the error message UMFPACK V5.7.9 (Oct 20, 2019): ERROR: out of memory it is a memory issue.

Several things you can try:

  • using MUMPS as a linear solver
my_model.settings = F.Settings(
    absolute_tolerance=1e11,
    relative_tolerance=1e-10,
    maximum_iterations=20,
    linear_solver="mumps",
    traps_element_type="DG",
)
  • grouping traps 1 2 3 into one trap. This will greatly reduce the number of degrees of freedom ( see this for more details)
  • running the script in parallel with
mpirun -np 4 python your_script.py

Here 4 is the number of cores.

I’ve used MUMPS and grouped the traps and was able to have a few Newton iterations (although the model is struggling to converge, maybe due to the fairly low temperature, but that’s another problem)

1 Like

Thank you so much!

I’ll try!

Thank you!