The previous session reconstructed a simple, thin sample where standard iterative ptychography converges cleanly. Real problems are messier: low-dose data of beam-sensitive materials, thick multislice objects, and datasets where the optimization is badly conditioned. This session shows how deep learning strengthens the reconstruction while keeping the physics in charge.
Physics-based solvers with learned priors¶
The most robust way to use machine learning here is not to replace the forward model but to re-parameterize the unknowns: the object and probe are generated by neural networks (deep generative priors, DGPs) whose weights are optimized so that the same mixed-state multislice physics model still has to explain the measured data arXiv:2511.07795. The convolutional architecture acts as an implicit structural prior (spatial coherence for free, no hand-tuned regularization), and an autoencoder pre-training step keeps the joint optimization stable:
The DGP framework: two networks generate the complex object and probe inside a differentiable ptychography forward model (a). Noise-initialized networks are unstable (b); pre-training them as autoencoders on a conventional reconstruction (c) makes the full reconstruction stable (d). Adapted from McCray et al., arXiv:2511.07795.
Related learned approaches include deep image priors for general inverse problems and trained solver networks such as PtychoNN Cherukara et al. (2020); the DGP framework used in this session is implemented in the open-source quantEM package.
What you gain¶
Dose efficiency and noise robustness: with a learned prior, usable reconstructions emerge from far fewer counts, which matters most exactly where phase contrast matters most: beam-sensitive samples.
Noise robustness on a beam-sensitive metal-organic framework (100 e⁻/Ų): DGP object and probe (left) vs. conventional pixelated reconstruction (right). The learned prior suppresses the noise floor and extends the information limit from 1.98 Å to 1.57 Å. Adapted from McCray et al., arXiv:2511.07795.
Speed and stability: the prior smooths the optimization landscape, so reconstructions converge in fewer iterations and are less sensitive to initialization.
Multislice and mixed-state reconstructions inherit the same benefits, with improved depth regularization extending deep-learning ptychography to thick, complex systems:
Pixelated vs. deep-generative-prior reconstructions of the same data: at matched (or shorter) reconstruction times, the learned prior suppresses the noise floor while preserving the atomic-scale features the physics model demands.
Watching a reconstruction converge: the learned prior recovers the low spatial frequencies (particle shapes) in a fraction of the iterations that a pixel-based reconstruction needs.
Caveats¶
A prior is a bias. Validate learned reconstructions the same way as any other: check convergence against the data (not just visual appeal), compare against a conventional reconstruction, and be suspicious of features at the resolution limit.
Machine learning across STEM analysis, including its failure modes, is reviewed in Kalinin et al. (2022); the iterative phase-retrieval framework these methods build on is covered in arXiv:2309.05250.
- Cherukara, M. J., Zhou, T., Nashed, Y., Enfedaque, P., Hexemer, A., Harder, R. J., & Holt, M. V. (2020). AI-enabled high-resolution scanning coherent diffraction imaging. Applied Physics Letters, 117(4). 10.1063/5.0013065
- Kalinin, S. V., Ophus, C., Voyles, P. M., Erni, R., Kepaptsoglou, D., Grillo, V., Lupini, A. R., Oxley, M. P., Schwenker, E., Chan, M. K. Y., Etheridge, J., Li, X., Han, G. G. D., Ziatdinov, M., Shibata, N., & Pennycook, S. J. (2022). Machine learning in scanning transmission electron microscopy. Nature Reviews Methods Primers, 2(1). 10.1038/s43586-022-00095-w