Summary
In this talk, Rahimi
introduces her recent work about automatically extracting topic components from
source materials. Such source-based topic component extraction can be a
replacement of manual efforts performed by experts and provides convenience for
automatic assessment process.
Rahimi starts
from the application end. To address the issue of automatic essay scoring, many
prior approaches have been provided, such as bag of words, semantic similarity,
content vector analysis and cosine similarity. However, many of them do not
take source materials into consideration. Rahimi points out that, different
from those prior work, their research highly relies on source materials, lying
in the domain of response-to-text writing assessment. Given source materials,
how to automatic evaluate students’ essays? Rahimi and her colleagues approach
this problem by localizing pieces of evidence in students’ essays that match
source materials. Instead of manually extract those evidence by experts, they
aim to offer an automatic way to find such evidence.
To be specific,
they use natural language processing techniques to automatically extract a
comprehensive list of topics from source materials. The list of topics consists
of topic words as well as specific expressions (N-grams) that students should
include in their essays, also defined as “topic components”. Table 1 gives us a
direct illustration about topic components.
 |
|
Table 1. Automatically
extracted topic words and N-gram expressions for each topic. They are extracted
by the proposed data-driven LDA-enabled model.
|
To evaluate the
performance of automatic extraction of topic components. Rahimi compare their
method with manual results and other competing baselines. Results are shown in table
2. It shows that their proposed method is very promising and outperforms all
other models. However, compared with manual upper bound, they still have much
improvement space.
 |
Table 2. Performance of models using automatically extracted
topical components, baseline models, and manual upper-bound.
|
About the talk:
Talk URL: http://halley.exp.sis.pitt.edu/comet/presentColloquium.do?col_id=10540
Speaker: Zahra
Rahimi
Homepage: http://people.cs.pitt.edu/~zar10/
Date: Nov 18,
2016
