The neural basis of reading and writing has been a source of inquiry as well as controversy in the neuroscience literature. Reading has been associated with both left posterior ventral temporal zones (termed the “visual word form area”) as well as more dorsal zones, primarily in left parietal cortex. Writing has also been associated with left parietal cortex, as well as left sensorimotor cortex and prefrontal regions. Typically, the neural basis of reading and writing are examined in separate studies and/or rely on single case studies exhibiting specific deficits. Functional neuroimaging studies of reading and writing typically identify a large number of activated regions but do not necessarily identify the core, critical hubs. Last, due to constraints on the functional imaging environment, many previous studies have been limited to measuring the brain activity associated with single-word reading and writing, rather than sentence-level processing. In the current study, the brain correlates of reading and writing at both the single- and sentence-level were studied in a large sample of 111 individuals with a history of chronic stroke using voxel-based lesion symptom mapping (VLSM). VLSM provides a whole- brain, voxel-by-voxel statistical analysis of the role of distinct regions in a particular behavior by comparing performance of individuals with and without a lesion at every voxel. Rather than comparing individual cases or small groups with particular behavioral dissociations in reading and writing, VLSM allowed us to analyze data from a large, well-characterized sample of stroke patients exhibiting a wide range of reading and writing impairments. The VLSM analyses revealed that reading was associated with a critical left inferior temporo-occipital focus, while writing was primarily associated with the left supramarginal gyrus. Separate VLSM analyses of single-word versus sentence-level reading showed that sentence-level reading was uniquely associated with anterior to mid-portions of the middle and superior temporal gyri. Both single-word and sentence-level writing overlapped to a great extent in the left supramarginal gyrus, but sentence-level writing was associated with additional underlying white matter pathways such as the internal capsule. These findings suggest that critical aspects of reading and writing processes diverge, with reading relying critically on the ventral visual recognition stream and writing relying on a dorsal visuo-spatial-motor stream.