The remarkable ability of Mycobacterium tuberculosis to survive attacks from the host immune response and drug treatment is due to the resilience of a few bacilli rather than a result of survival of the entire population. Maintenance of mycobacterial subpopulations with distinct phenotypic characteristics is key for survival in the face of dynamic and variable stressors encountered during infection. Mycobacterial populations develop a wide range of phenotypes through an innate asymmetric growth pattern and adaptation to fluctuating microenvironments during infection that point to heterogeneity being a vital survival strategy. In this Review, we describe different types of mycobacterial heterogeneity and discuss how heterogeneity is generated and regulated in response to environmental cues. We discuss how this heterogeneity may have a key role in recording memory of their environment at both the single-cell level and the population level to give mycobacterial populations plasticity to withstand complex stressors.