The few existing surface complexation models (SCM) for the brine-oil interface have important limitations: the chemistry of each crude oil is not considered, they cannot capture the water/non-polar hydrocarbons surface charge, the interactions between Na+ and the acid sites are not included, and the equilibrium constants for the adsorption reactions are not validated against experimental data. We address the aforementioned constraints by proposing an improved diffuse-layer SCM for the oil-brine interface. The new model accounts for the chemistry of crude oils by considering surface sites linearly dependent on the TAN (total acid number) and TBN (total base number). We define weak sites to account for the negative surface charge observed for non-polar hydrocarbons in water. We optimize the parameters of our model by fitting the model to reported zeta potential measurements of oil in aqueous solutions. When we validate the optimized model against different experimental data sets, it generally shows a good performance in predicting the surface charge of oil in different brines with different pHs. We show that the acid and base numbers are only useful as a qualitative estimation of the distribution of polar groups at the oil surface, and more sophisticated analysis is necessary to quantify the chemistry of the oil-brine interface.