A key step in engineering resistive switching is the ability to control the device switching behavior. Here, we investigate the possibility to tune the resistive switching of tantalum oxide (TaOx)-based memristors from a non-switchable state to a switchable state by applying post-fabrication annealing of the devices. The switching of the devices was found to be related to: (1) the oxidation state changes in the TaOx thin film after annealing and (2) the local variations in oxygen stoichiometry in the vicinity of the interface between the TiN electrode and the TaOx active resistive layer. We further discuss the possible mechanism behind the resistive switching after annealing. This experimental approach provides a simple but powerful pathway to trigger the resistive switching in devices that do not show any resistive switching initially.