The vast variation in floral traits across angiosperms is often interpreted as the result of adaptation to pollinators. However, studies in wild populations often find no evidence of pollinator-mediated selection on flowers. Evolutionary theory predicts this could be the outcome of periods of stasis under stable conditions, followed by shorter periods of pollinator change that provide selection for innovative phenotypes. We asked if periods of stasis are caused by stabilizing selection, absence of other forms of selection or by low trait ability to respond even if selection is present. We studied a plant predominantly pollinated by one bee species across its range. We measured heritability and evolvability of traits, using genome-wide relatedness in a large wild population, and combined this with estimates of selection on the same individuals. We found evidence for both stabilizing selection and low trait heritability as potential explanations for stasis in flowers. The area of the standard petal is under stabilizing selection, but the variability is not heritable. A separate trait, floral weight, presents high heritability, but is not currently under selection. We show how a simple pollination environment coincides with the absence of current prerequisites for adaptive evolutionary change, while heritable variation remains to respond to future selection pressures.