ORMN: A Deep Neural Network Model for Referring Expression Comprehension

KIPS Transactions on Software and Data Engineering, Vol. 7, No.2, pp.69-76, February 2018
10.3745/KTSDE.2018.7.2.069, Full Text

Abstract

Referring expressions are natural language constructions used to identify particular objects within a scene. In this paper, we propose a new deep neural network model for referring expression comprehension. The proposed model finds out the region of the referred object in the given image by making use of the rich information about the referred object itself, the context object, and the relationship with the context object mentioned in the referring expression. In the proposed model, the object matching score and the relationship matching score are combined to compute the fitness score of each candidate region according to the structure of the referring expression sentence. Therefore, the proposed model consists of four different sub-networks: Language Representation Network(LRN), Object Matching Network (OMN), Relationship Matching Network(RMN), and Weighted Composition Network(WCN). We demonstrate that our model achieves state-of-the-art results for comprehension on three referring expression datasets.


Statistics

Show / Hide Statistics

Statistics (Cumulative Counts from October 15, 2016)

Multiple requests among the same browser session are counted as one view. If you mouse over a chart, the values of data points will be shown.


Cite this paper

[KIPS Transactions Style]
D. Shin and I. Kim, "ORMN: A Deep Neural Network Model for Referring Expression Comprehension," KIPS Transactions on Software and Data Engineering, Vol.7, No.2, pp.69-76, 2018, DOI: 10.3745/KTSDE.2018.7.2.069.

[IEEE Style]
Donghyeop Shin and Incheol Kim, "ORMN: A Deep Neural Network Model for Referring Expression Comprehension," KIPS Transactions on Software and Data Engineering, vol. 7, no. 2, pp. 69-76, 2018. DOI: 10.3745/KTSDE.2018.7.2.069.

[ACM Style]
Shin, D. and Kim, I. 2018. ORMN: A Deep Neural Network Model for Referring Expression Comprehension. KIPS Transactions on Software and Data Engineering, 7, 2, (2018), 69-76. DOI: 10.3745/KTSDE.2018.7.2.069.