We report results of elastic- and inelastic-neutron-scattering studies of three single crystals of the layered perovskite La2NiO4+delta, with delta = 0.00, 0.067, and 0.077, as well as neutron powder diffraction on a material with an oxygen excess delta almost-equal-to 0.065. The magnetic correlations are highly sensitive to the oxygen content, and three-dimensional antiferromagnetic order sets in at T(N) greater-than-or-equal-to 300 K, and T(N) = 68, and 48 K for the single crystals, respectively, and at T(N) = 74 K for the powder sample. The crystal with delta = 0.067 was studied in detail to characterize the tetragonal to orthorhombic (T(s) = 232-240 K) and antiferromagnetic (T(N) = 68 K) transitions. The order parameter eta(T) for the orthorhombic distortion shows a power-law dependence on temperature with the exponent beta = 0.21 and hysteresis, suggesting that this transition is less than second order. Two-dimensional critical scattering is observed around the antiferromagnetic phase transition. The magnetic excitation spectra in the three-dimensional ordered state are characterized by small anisotropy gaps between approximately 0 and 3 meV, and large effective in-plane spin-wave velocities, which decrease with increasing delta. For delta = 0. 077, c almost-equal-to 130 meV angstrom; and for delta = 0.00, c greater-than-or-equal-to 300 meV angstrom.