Aiming at multi-modal knowledge representation we construct a dataset with pairs of digital photos of objects. We manually score image pairs for semantic object similarity. A pre-trained ImageNet- based deep neural network predicts the objects and we use the output to estimate the similarity between two images. With a linkage between the neural network and Wikidata, we augment the model and incorporate knowledge graph information into the similarity measure. We compare the machine-based predicted similarity with the human-based semantic similarity.
|Journal||CEUR Workshop Proceedings|
|Publication status||Published - 1 Jan 2018|
|Event||1st Workshop on Deep Learning for Knowledge Graphs and Semantic Technologies - Heraklion, Crete, Greece|
Duration: 4 Jun 2018 → 4 Jun 2018
Conference number: 1
|Conference||1st Workshop on Deep Learning for Knowledge Graphs and Semantic Technologies|
|Period||04/06/2018 → 04/06/2018|