NTCIR STC2 is the second round of NTCIR Short Text Conversation. This page overviews the Japanese subtask. If you are interested in the Chinese subtask, please refer to this page.
In the Japanese subtask, given an input comment, the task is to retrieve or generate a suitable response to that comment. If we can create reasonable responses, it will lead to the improvement in the language generation capability of dialogue systems. Unlike STC1, both retrieval and generative methods are allowed for response generation.
Note that, in the Japanese subtask, comments posted at Yahoo! News are provided as a pool of comments. This is different from STC1, in which Twitter was used. In Yahoo! News, users post comments about an article. Users can also post comments to other users’ comments as responses. We use these comment-response pairs as our dataset in STC2.
The following data will be provided:
The input data for 2 and 3 are those sampled from Yahoo! News comments. The data will be provided to participants upon signing an agreement with NII.
As in STC1, retrieved/generated responses will be evaluated by human judges. The evaluation is done by the organizers. In the evaluation process, each retrieved/generated response is labeled with 0 (inappropriate), 1 (appropriate in some context), and 2 (appropriate) by multiple judges. The rate of appropriate answers and information retrieval (IR) related measures (graded relevance IR measures) will be used as evaluation metrics.
Registration is closed.
Regarding the Japanese task, please contact the organizers of the Japanese task
With regards to the information about STC, please contact the following address.