The response of pions and protons in the energy range of 20 to 180 GeV produced at CERN's SPS H8 test beam line in the ATLAS iron-scintillator Tile hadron calorimeter has been measured. The test-beam configuration allowed to measure the longitudinal shower development for pions and protons up to 20 nuclear interaction lengths. It is found that pions penetrate deeper in the calorimeter than protons. However, protons induce showers that are wider laterally to the direction of the impinging particle. Including the measured total energy response, the pion to proton energy ratio and the resolution, all observations are consistent with a higher electromagnetic energy fraction in pion induced showers. The data are compared with GEANT4 simulations using several hadronic physics lists. The measured longitudinal shower profiles are described by an analytical shower parameterization within an accuracy of 5-10%. The amount of energy leaking out behind the calorimeter is determined and parameterised as a function of the beam energy and the calorimeter depth. This allows for a leakage correction of test-beam results in the standard projective geometry.