Objective: To determine whether calibration improves observer reliability when assessing DMFT-score, root-filled teeth and periapical lesions on panoramic radiographs. Material and methods: A sample of 100 panoramic radiographs was selected from a cohort of myocardial infarction patients (n = 797) and matched controls (n = 796). The following variables were assessed: DMFT-score, remaining teeth, root-filled teeth and periapical lesions. Two specialists, an endodontist and a radiologist, served as reference examiners and undertook two separate assessments. Disagreement cases were jointly assessed and the final results were used as the reference standard. Three observers undertook three separate assessments, the first without prior training, the second after calibration against the reference standard and the third with the sample concealed in the complete material. Statistical analysis was made with Wilcoxon Signed rank test and Sign test. Agreement was calculated as Intraclass Correlation Coefficient (ICC) (95% CI) and Weighted Kappa (κ) (95% CI). Results: Periapical lesions disclosed high inter-observer variability for the reference examiners and diverged significantly between the observers and the reference standard. For the reference examiners, inter-observer agreement was κ = 0.53. The observers, in their first assessments had κ values ranging from 0.22 to 0.60 in relation to the reference standard. Following calibration, the κ values increased, ranging from 0.59 to 0.80. For the third assessment, the κ values ranged from 0.54 to 0.75. DMFT-score, remaining teeth and root-filled teeth disclosed high reliability throughout all assessments (ICC = 0.88-0.98 and κ = 0.98–0.99). Conclusions: DMFT-score, remaining teeth and root-filled teeth can be reliably assessed on panoramic radiographs. Calibration against a reference standard improves observer reliability in the detection of periapical lesions.