Recording atomic-resolution transmission electron microscopy (TEM) images isbecoming increasingly routine. A new bottleneck is then analyzing thisinformation, which often involves time-consuming manual structuralidentification. We have developed a deep learning-based algorithm forrecognition of the local structure in TEM images, which is stable to microscopeparameters and noise. The neural network is trained entirely from simulationbut is capable of making reliable predictions on experimental images. We applythe method to single sheets of defected graphene, and to metallic nanoparticleson an oxide support.