Our visual environment contains statistical structure at various scales. This offers rich opportunity for the brain to exploit this structure for efficient ways to code and process information, and ultimately to improve perception. Theories such as Predictive Coding suggest that previous experience enables us to make predictions about upcoming events and to test them against current sensory inputs. However, the neural mechanisms underlying perceptual inference remain poorly understood. I will present experiments investigating how cortical networks and single neurons implement perceptual inference, employing functional magnetic resonance imaging, electrocorticography, and single unit recordings in monkeys and humans. I will show that predictions are generated in hierarchical networks of brain areas, including prefrontal cortex, and that single neurons compute prediction errors that go beyond the stimulus features that these neurons encode. Importantly, the hierarchical structure of visual processing pathways that gives rise to invariant representations allows constructing internal models that can make abstract predictions not bound to the specific low-level features of the learned stimuli. This equips sensory areas with a machinery for generalization previously ascribed only to higher-order association cortices. Together, these studies provide a mechanistic understanding of perceptual inference and show how interactions between brain areas can give rise to stable percepts.