posted by Zeke Shore on Mar 2nd, 2010
The New York Times online consistently delivers interesting data visualizations to help enrich the stories surrounding popular news topics. The New York Times Innovation Portfolio provides a beautiful overview of all of these interactive explorations, organized by topic with project overviews, documention, and links to the actual interactive pieces.
Since VoxPop is working with New York Times data, this collection of existing data visualizations is a treasure trove of strong precedents, several of which relate very closely to our project. Here are a handful that live within the realm of reader sentiment.
Health Care Debate
Health Care Debate is a conversation platform that allows users to discuss various issues within the health care debate. The most interesting aspect of this tool is how the relevance of specific sub-topics within the debate can be instantly comprehended at first glance, with the surface of the tool depicting multiple “rooms” that are scaled relative to the number comments relating to that subtopic.
Obama’s Address In Cairo
This interactive video of Obama’s speech to the Muslim world allows users to provide comments along the timeline of the speech, allowing a global discussion to unfold in the context of the time-based content that is seeding the discussion.
Election Word Train
Election Word Train asked New York times readers to share one word that describes their current state of mind on the day of the 2008 presidential election. Much like a tag cloud, words are scaled relative to the number of people sharing the sentiment, and can be filtered to show words shared by Obama or McCain supporters. By leveraging scale and letting these words ’speak for themselves’ does effectively provide a general glimpse of reader sentiment, even if the forum is is somewhat contrived, specifically with the goal reducing group sentiment into a few dozen words, possibly hindering truly organic sentiment visualization.
Inaugural Words
Inaugural Words ranks the frequency of words used by presidents in Inaugural Addresses, showing what words each president used the most. While is not really reflecting reader sentiment. it does show an interesting break down of word frequency across time and political position.
Twitter Bowl
The Twitter Bowl interactive visualization maps twitter chatter over the course of the 2009 Super Bowl, according to key topic mentions. This hits an interesting cross section of communicating time, space, and group sentiment, even if it is somewhat cryptic in what is actually being communicated. There is something very satisfying about seeing topics grow and shrink geographically over time, although it does not reveal what specifically about “steelers” or “ads” or “springsteen” people are sharing.
These projects all have several aspects that worth analyzing and building upon. As we begin to re-think how people engage with the news, its exciting to see major players like the New York Times continuing to push the envelope, and continue to keep their data open so that others can do the same.
posted by Zeke Shore on Feb 17th, 2010
While exploring existing sentiment analysis processes, we stumbled across what looks like a fully integrate open source solution to several issues identified in our recent round of research.
OpinionFinder appears to be hosted and primarily developed at the University of Pittsburgh with contributions from Cornell University and University of Utah. While the OpinionFinder system was only mentioned off hand in Bo Pang’s article Opinion Mining and Sentiment Analysis, it appears to include some of the best solutions available for a lot of the common challenges that accompany effective sentiment analysis.
OpinionFinder, which was initially released in 2006, employs a multi-stage NLP process. As stated in the project’s extended abstract,
“OpinionFinder aims to identify subjective sentences and to mark various aspects of subjectivity in these sentences, including the source (holder) of the subjectivity and words that are included in phrases expressing positive or negative sentiments.”
Working in “batch” mode as more of a back-end pipe, OpinionFinder works as follows:
Document Processing
Taking any incoming text source, HTML or XML meta info is removed, and sentences are split and POS tagged using OpenNLP. Next, stemming is accomplished using Steven Abney’s SCOL v1K stemmer program. SUNDANCE (Sentence UNDerstanding And Concept Extraction), a partial parser from the NLP laboratory at the University of Utah, is used by Autoslog-TS to identify extraction patterns needed by the sentence classifiers and the SourceFinder (which identifies the source of subjective content, distinguishing author statements from related or quoted statements). A final parse in batch mode establishes constituency parse trees which are converted to dependency parse trees for Named Entity and subject detection.
Subjectivity and Sentiment Analysis
At this point a Naive Bayes classifier identifies subjective sentences. The specs seem to indicate that the classifier is trained against subjective and objective sentences generated by two additional “rule-based” (unsupervised?) classifiers drawing from “a large corpus.” This point in the process will require some exploration and validation.
Next a direct subjective expression and speech event classifier, built by Eric Breck, tags the direct subjective expressions and speech events found within the document using WordNet.
The final step applies actual sentiment analysis to sentences that have been identified as subjective. This is accomplished with two classifiers that were developed using the BoosTexter machine learning program and trained on the MPQA Corpus.
Evaluation
While we still need to rigorously explore the source code, this system appears to be a gold mine of solutions to both previously unresolved and newly discovered issues in our sentiment analysis process. Named Entity detection along with dependency parse trees will help us filter content to only include sentiment regarding the actual topic being explored (rather than visualizing all subjective content in a comment) as well as helping to reveal popular related topics that exist within any given topic of discussion.
Subjectivity detection and Speech Event Classification are challenges that are acknowledged in a lot of research on the topic of sentiment analysis, but comprehensive solutions have been much more difficult to come by. This system seems to combine a few processes towards those goals (including leveraging WordNet in a new way), and again could really help us filter down our corpus to relevant statements of sentiment for a given topic.
Finally the actual positive/negative sentiment analysis that is applied to subjective sentences is different than any other process I have read about (most including WordNet and trained classifiers, or our original ad hoc method of matching against the General Inquirer Dictionary). We might want to experiment a bit with this phase to see how more or less effective different methods are.
One process that is surprisingly absent from the OpinionFinder system is any sort of negation detection. We may want to explore possibly integrating the algorithm Bruno Ohana experimented with in his dissertation on sentiment analysis, or investigate other solutions.
It also maybe be interesting to see how things change if we begin to stack some of the process used by OpinionFinder with systems that we already have in place, such as our GI Osgood Emotive Assignments.
You can download OpinionFinder for free from the project’s website under an open academic license, or download a PDF of the extended abstract/description of the project here:
OpinionFinder-Extended Abstract
posted by Zeke Shore on Feb 16th, 2010
One issue of accurate sentiment analysis identified in a recent round of research is the problem of negation detection. This is the process by which a negating word (such as ‘not’) inverts the evaluative value of an affective word ( for example, “not good” is similar to saying “bad”). This can be resolved in natural language processing by identifying negating words, and then inverting the value of any positive or negative word within n-words of the negating word, where n is the window of potential negation.
In Bruno Ohan’s 2009 dissertation “Opinion Mining with with the SentWordNet Lexical Resource” (Dublin Institute of Technology), a python algorithm is presented to perform this task. While Ohan’s tests with this negation detection algorithm only yielded accuracy improvements of about 0.5%, this might be a good start point for further exploration.
#
# populates array of negated terms based on document terms
# negation[i] indicates if term in doc[i] is negated
#
def getNegationArray(doc, windowsize):
PSEUDO = ( 'no increase', 'no wonder', 'no change' , 'not cause' ,
'not only' , 'not necessarily' )
PRENEGATION = ( 'not' , 'no' , 'n\'t' ,'cannot', 'declined' ,
'denied' , 'denies' , 'free of' , 'fails to' , 'no evidence' ,
'no new' , 'no sign' , 'no suspicious' . 'no suggestion' ,
'rather than', 'with no' , 'unremarkable', 'without' ,
'rules out' , 'ruled out', 'rule out')
POSNEGATION = ( 'unlikely', 'free', 'ruled out' )
ENDOFWINDOW = ( '.', ':', ',', 'but' , 'however' , 'nevertheless' ,
'yet' , 'though' , 'although' , 'still' , 'aside from' , 'except' ,
'apart from')
# Initialise array
vNEG = [ 0 for t in range(len(doc)) ]
# Initialise window counters
winstart = 0
winend = min( windowsize, len(doc) - 1 )
docsize = len(doc)
i = 0
found_pseudo = 0
found_neg_fwd = 0
found_neg_bck = 0
inwindow = 0
for i in range(docsize):
#
# build 1-ter and 2-term strings
#
unigram = doc[i].split('/')[0]
if i < (docsize - 1):
bigram = unigram + ' ' + doc[i+1].split('/')[0]
else:
bigram = unigram
#
# Search for pseudo negations
#
for negterm in PSEUDO:
if bigram == negterm:
found_pseudo=1
##print 'found pseudo!', bigram, i
if (found_pseudo == 0):
#
# Look for pre negations
#
for negterm in PRENEGATION:
if unigram == negterm or bigram == negterm:
found_neg_fwd = 1
for negterm in POSNEGATION:
if unigram == negterm or bigram == negterm:
found_neg_bck = 1
#
# If found fwd/backw negation, then negate window
#
if (found_neg_fwd == 1):
##print 'found forwards!', unigram, bigram, i
#
# negate terms forward up to window
#
if inwindow < windowsize:
vNEG[i] = 1
inwindow+=1
else:
# out of window space
found_neg_fwd = 0
inwindow = 0
#
# backward negation
#
if (found_neg_bck == 1):
##print 'found backwards!', unigram, bigram, i
#
# negate back until window start
#
for counter in range(max(winstart, i-windowsize), i):
vNEG[counter] = 1
#
# done with backwards negation
#
found_neg_bck = 0
#
# now move window
#
for negterm in ENDOFWINDOW:
if unigram == negterm or bigram == negterm:
#
# found end of negation, must reset windows
#
##print 'found negterm!', unigram, bigram, i
inwindow = 0
found_neg_fwd = 0
winstart = i
winend = min( windowsize + i, len(doc) - 1 )
return vNEG
posted by Zeke Shore on Feb 16th, 2010
I have come across some fantastic Semantic Analysis research over the past few days, and was able to tap into several research papers and dissertations exploring computational Sentiment Analysis or Opinion Mining (OM). Two that provided significant insight were “Opinion Mining and Sentiment Analysis” (Pang et al, 2008) and “Opinion Mining with the SentWordNet Lexical Resource” (Ohana, 2009).
Recent progress in Opinion Mining techniques within natural language processing tasks identify a handful of challenges and potential solutions for accurate sentiment analysis of text based content.
Subjectivity
If our goal is to extract the sentiment, opinions or emotions of users, then we should really only be looking at subjective statements within a user’s comment. This will prevent positively or negatively charged words that are present in objective statements to effect the comment’s overall sentiment score. Subjectivity could be assed through a trained classifier algorithm like Naive Bayes or Max Entropy.
On Topic
A concern for topic relevance is an issue that we were already aware of, and were searching (with much difficulty) for solutions with dependency grammars. This new round of research seems to dismiss that approach as unrealistically difficult (I’m thinking that could be a project on its own). Unfortunately no good solution strategies were explored for this issue.
Polarity
This is our root goal of applying a negative or positive sentiment score at various text-unit levels, such as word, sentence, or comment. While VoxPop has thus far been using the General Inquirer Dictionary evaluative definitions… It appears a few recent projects have been utilizing the WordNet (which we explored earlier in our research) and news SentiWordNet lexicons for evaluative sentiment assignments.
Negation Detection
An issue that was just now revealed to us is the problem of Negation Detection. Consider the following two sentences:
Obama’s policies are good.
Obama’s policies are not good.
A normal polarity tagger would give these two sentences the same sentiment score, both of them containing containing 1 positive word (good). Of course our second sentence expresses the opposite of positive sentiment, with the adverb ‘not’ inverting the value of “good.” A negation detection process aims to identify these negating word, and then invert the value of any positive or negative words that appear wither n-words before or after the negating term.
Here are PDFs of two of the more informative articles:
Opinion Mining and Sentiment Analysis
Bo Pang, Lillian Lee
Opinion mining and sentiment analysis
Opinion Mining with the SentWordNet Lexicon
Bruno Ohan
Opinion mining with the SentWordNet lexical resource
