Differentiable Programming for Inverse Problems

The gradient of an inverse problem used to cost a season of a graduate student’s life. To recover a subsurface velocity model, a tissue’s stiffness, or a contaminant’s release history, you derived the adjoint equations by hand, implemented them, and then spent months hunting the single sign error that made the gradient check fail three chapters into the project. I did this for two decades, across seismic exploration, medical imaging, and environmental flow, and the hardest part of every project was the same: the adjoint.

Reverse-mode automatic differentiation changed that. The gradient that once cost a season now comes, exact, for the price of a backward pass — a small constant times one forward evaluation, independent of how many parameters you are solving for. This is not a machine-learning fashion. It is the quiet completion of the adjoint-state method the inverse-problems community has used since the 1970s: autodiff is the adjoint, computed mechanically, and a working scientist can now obtain it by differentiating the forward code rather than re-deriving the mathematics by hand. The mathematics did not change. The labor did.

This book exists because the two literatures that should have met have not. The texts that teach automatic differentiation explain the machine beautifully and then stop at the gradient, as though a gradient were a reconstruction. The texts that teach inverse problems were written before cheap differentiation and still hand-derive every adjoint, treating the gradient as the expensive part. Neither shows the whole pipeline on real problems: the exact gradient, yes, but then the regularization that the gradient cannot supply, the uncertainty quantification that turns an estimate into a result, the resolution analysis that says honestly what the data can and cannot recover — and runnable code that reproduces every number and figure. The gap between “I can compute the gradient” and “I have a trustworthy reconstruction with an error bar” is where the real work lives, and it is the subject of this book.