Midterm Slide Deck
tags: presentation process
posted by Zeke Shore on Mar 1st, 2010
Slide deck for our midterm presentation in Dave Carroll’s Thesis Studio.
comment
0
tags: presentation process
New Sentiment Analysis Research
tags: NLP opinion mining sentiment analysis WordNet
posted by Zeke Shore on Feb 16th, 2010
I have come across some fantastic Semantic Analysis research over the past few days, and was able to tap into several research papers and dissertations exploring computational Sentiment Analysis or Opinion Mining (OM). Two that provided significant insight were “Opinion Mining and Sentiment Analysis” (Pang et al, 2008) and “Opinion Mining with the SentWordNet Lexical Resource” (Ohana, 2009).
Recent progress in Opinion Mining techniques within natural language processing tasks identify a handful of challenges and potential solutions for accurate sentiment analysis of text based content.
Subjectivity
If our goal is to extract the sentiment, opinions or emotions of users, then we should really only be looking at subjective statements within a user’s comment. This will prevent positively or negatively charged words that are present in objective statements to effect the comment’s overall sentiment score. Subjectivity could be assed through a trained classifier algorithm like Naive Bayes or Max Entropy.
On Topic
A concern for topic relevance is an issue that we were already aware of, and were searching (with much difficulty) for solutions with dependency grammars. This new round of research seems to dismiss that approach as unrealistically difficult (I’m thinking that could be a project on its own). Unfortunately no good solution strategies were explored for this issue.
Polarity
This is our root goal of applying a negative or positive sentiment score at various text-unit levels, such as word, sentence, or comment. While VoxPop has thus far been using the General Inquirer Dictionary evaluative definitions… It appears a few recent projects have been utilizing the WordNet (which we explored earlier in our research) and news SentiWordNet lexicons for evaluative sentiment assignments.
Negation Detection
An issue that was just now revealed to us is the problem of Negation Detection. Consider the following two sentences:
Obama’s policies are good.
Obama’s policies are not good.
A normal polarity tagger would give these two sentences the same sentiment score, both of them containing containing 1 positive word (good). Of course our second sentence expresses the opposite of positive sentiment, with the adverb ‘not’ inverting the value of “good.” A negation detection process aims to identify these negating word, and then invert the value of any positive or negative words that appear wither n-words before or after the negating term.
Here are PDFs of two of the more informative articles:
Opinion Mining and Sentiment Analysis
Bo Pang, Lillian Lee
Opinion mining and sentiment analysis
Opinion Mining with the SentWordNet Lexicon
Bruno Ohan
Semester 2 Introduction
tags: dave carroll presentation thesis
posted by Andrew Mahon on Feb 1st, 2010
Slide deck from our semester introduction in Dave Carroll’s Thesis Studio.
Midpoint Presentation
tags: nltk NYT API process visualization
posted by Zeke Shore on Jan 27th, 2010
Back in November we had the honor of presenting our progress to the R&D department at the New York Times, followed by a similar presentation to our upcoming thesis advisor, Dave Carroll for the final semester of the project. Feel free to check out the slide deck, however links to the working prototypes may be down (running servers is expensive!)
Validating the General Inquirer Dictionary
tags: general inquirer lasswell MIT namenwirth NLP
posted by Zeke Shore on Nov 15th, 2009
We have been trying to hunt down more information about the General Inquirer Dictionary, since it is currently serving as our primary reference table for emotively evaluating the words within the discussions of New York Times articles. We were able to get in contact with Roger Hurwitz, a research scientist at MIT’s Artificial Intelligence Lab, and one of the GI dictionary’s moderators, who was able to shed some light:
The General Inquirer scores sentiment in texts on the basis of surface text words whose root forms and contextually disambiguated senses mark negative or positive attitudes, per the General Inquirer dictionary. I realize that sounds circular, but there are many such words in the dictionary, so that coverage has proved adequate and results have acceptable inter-coder reliability with scoring of the same texts by human coders. The GI also scores texts in just over 200 other fields or any subset thereof per users’ desires. these fields include expressions of the eight social values that political scientist Harold Lasswell found basic to human social activity. Namenwirth and Weber using the GI and Lasswell values dictionaries to code American political party platforms and speeches from the British throne, respectively, found long and short value cycles in American and English society (following a relative attention paradigm, as measured by frequency of mention.) The book Dynamics of Culture (Boston: Allen & Unwin, 1987) may be out of print. However, an article by Namenwirth lays out the theory and is available online.
So I found and reviewed the J. Zvi Namenwirth study that was published in the Journal of Interdisciplinary History (MIT Press) in 1973. Namenwirth is mostly mapping public values through the content of presidential campaign transcriptions from 1844 to 1964.
The following two graphs show the frequency of the word ‘wealth’ over time, normalizing for transcript lengths, and begin to reveal some interesting cyclical patterns over the 120 year stretch.
These early natural language processing studies are interesting to look at, partially because of how much was accomplished with such little computational resources available. While word count may be a relatively trivial metric by today’s NLP capabilities, it does reveal interesting patterns over longer time lines.
This seems to validate our efforts to develop a lens through which the pre-aggregated corpus of the web can be analyzed through more rigorous NLP systems, revisiting what the General Inquirer Dictionary might be able to reveal.
The study is not openly published, so I cannot post the PDF on the site, but here is the citation and Jstor link:
J.Z. Namenwirth, “The Wheels of Time and the Interdependence of Value Change,” J. Interdisciplinary History, 3 (1973): 649-683
Stable URL: http://www.jstor.org/stable/202687
Emotive Analysis Process V1.1
tags: emotion grammar lasswell NLP NYT API Osgood python
posted by Zeke Shore on Nov 11th, 2009
Here is a quick update on how our emotive analysis engine is playing out. The end to end process (for this initial prototype) will work as follows:
First, the user provides a search query, and we pull (and cache) all of the NY Times articles that are related to that query that have comments using the Article Search API and the Community API (this will be made more efficient in the near future… more to come on that later).
After article or comment results to a query are returned from either the cache or a new API call, what we will need to deal with initially on the Natural Language Processing (NLP) side of the equation will be comments, in the form of text strings.
Using NLTK in python, there is an information extraction architecture that is structured as follows:
For our purposes, one of the more difficult challenges that we have is knowing what words we care about. If we are trying to visualize the emotional or affective characteristics of the discourse surround a keyword, we cannot just look at the full thread of comments for an article that was returned for a given keyword, and log every word that holds emotive weight. The NY Times article Bipartisan Spirit, at Least for a Moment is a perfect example as to why not. The article is about a meeting between President Obama and George Bush Sr. So as one may guess, that article would have been returned when querying either ‘Bush’ or “Obama,’ and the 38-comment discussion that follows the article contains references to both.
So before any sort of emotive analysis can occur, we must parse the text down to the words that we care about. This first involves identifying instances of our keyword within each comment, and extracting the sentences that contain the keyword.
For further coverage, and also to account for the fact the web-based comments are often less verbose and less refined than other forms of discourse, if our keyword is a proper noun, we might also look at sentences with pronouns that immediately precede or follow sentences with our keyword.
Ultimately, we will need to develop a comprehensive weighted dependency grammar, so that we can efficiently parse the sentences that we care about into relatively accurate dependency structures. This will allows to know (with far more precision) what words are referring to or modifying our keyword, and should therefor be emotively classified.
So now the fun part. Once we know what words we care about in relation to our keyword, we will go back to Charles Osgood’s Semantic Differential Theory which maps words along three main axises: the Evaluative (good/bad), the Potency (strong/weak) and the Activity (active/passive) which I have discussed in a previous post. We can do this using the General Inquirer Dictionary, including the Lasswell Value Dictionary and the Harvard IV-4 dictionary, which maps about 12,000 words across Osgood’s semantic differential axises (among other classifications).
To make the process more efficient, since we have tagged the part of speech of every word, we can throw out words that we know should have neutral affective values, such any determiners (’the,’ ‘a,’ etc) or any proper nouns, and map every other word against our three axises. For each axis, we will give a word a value of 1, 0, or -1, so on the evaluative (EVA) axis, for example, any word living at the ‘positive’ or ‘good’ end of the axis would hold a value of 1, whereas a word living at the ‘negative’ or ‘bad’ end of the axis would hold a value of -1, and of course words that are neutral on the evaluative scale would hold a value of 0. This system would carry out across the activity (ACT) and potency (POT) axises as well in the form of
affectiveValue(word) = [EVA, ACT, POT]
affectiveValue(respect) = [1, -1, 0]
Where the word “respect” holds an evaluative value of ‘positive’ or ‘good,’ an active value of ‘passive’ and a potency value of ‘neutral’ (neither ’strong’ nor ‘weak’).
So ultimately this will leave us with six lists of words for each article in relationship to a given keyword, which we can then use as metrics for our data visualization.
Research Progress Presentation
tags: lasswell NLP NYT API Osgood WordNet
posted by Zeke Shore on Oct 24th, 2009
We recently presented our research progress, mostly focusing around proof of concept results of using the NYT API as an effective corpus, and exploring the work of Charles Osgood, Shortest Path Distance mapping with WordNet in the NLTK, and mapping words against the Lasswell Value Dictionary. We show some initial emotive analysis on a NYT article comment (which was part of a 38 comment discourse) to show what our WordNet + Lasswell engine might reveal.
Download the full presentation below:
Coupling Niche Browsers and Affect Analysis for an Opinion Mining Application
tags: data emotion lasswell NLP
posted by Zeke Shore on Oct 13th, 2009
Abstract
Newspapers generally attempt to present the news objectively. But textual affect analysis shows that many words carry positive or negative emotional charge. In this article, we show that coupling niche browsing technology and affect analysis technology allows us to create a new application that measures the slant in opinion given to public figures in the popular press.
by Gregory Grefenstette, Yan Qu, James G. Shanahan, David A. Evans
Download the full PDF:
Coupling niche browsers and affect analysis for an opinion mining application
Notes and Further Exploration
Interesting research on computationally evaluating emotional charge in news reports from various sources surrounding various topics. The research utilized the Lasswell Value Dictionary, which was a project started in the 1960s as a lexicon of over 2,000 words emotively classified and rated. The formatted text version of the General Inquiry library could serve as an invaluable resource for the VoxPop project, providing a source of quantitatively mapped linguistic information that web based discourse could be mapped against.
Some interesting findings from the research include the fact that positivity charged words are often grouped with other positively charged words, and likewise negatively charged words are often grouped with other negatively charged words, allowing word proximity between classified and unclassified words to serve as a machine-learning tactic.
This article strongly demonstrates the complexity of attempting to accurately quantify the emotional tone of a body of text across even just two dimensions: positive and negative. While the process involved in this project was not simple, it was successful, and could definitely be tailored the emotive qualities of a heated op-ed debate surrounding a topic of interest.
Due to the complex classification network employed by the Lasswell General Inquiry Dictionary and the Harvard IV-4 dictionary, I would imagine more layers of emotive understanding could be quantitatively extracted from linear textual exchanges (i.e, emotions exchanged between users vs. emotions towards the seeding article).
Jonathan Harris Visualizes Feelings
tags: API data jonathan harris visualization
posted by Zeke Shore on Oct 12th, 2009
Digital artist Jonathan Harris created an interesting data visualizing piece in 2006 called We Feel Fine. The project crawls the web for blog posts that contain the words “I” and “feel” in the same sentence then extracts that sentence from it’s original context. This content is visualized through a swarm of “feelings” as bouncing colored balls, interacting on a dark canvas. These balls swarm around the user’s cursor, prompting them to be clicked open and explored, allowing the feelings to be read in there entirety.
We Feel Fine functions strongly as a poetic exploration of feelings as they surface on the web, and the style of the visualization supports the idea of the web as a living breathing repository of human thought. The project does begin to explore this data through a quantitative lens, allowing users to sort feelings by year, gender, age, weather, location, and even the associated adjectives that accompanied these “feelings.” Alternate views of the data also begin to reveal more meaningful trends when examined.
Also of interest, We Feel Fine has an open API allowing other developers to harness the data that they crawl and format so that it can be used by other projects. More information on the API is available at http://wefeelfine.org/api.
Words With Attitude
tags: emotion linguistics NLP Osgood WordNet
posted by Zeke Shore on Oct 9th, 2009
Abstract
The traditional notion of word meaning used in natural language processing is literal or lexical meaning as used in dictionaries and lexicons. This relatively objective notion of lexical meaning is different from more subjective notions of emotive or affective meaning. Our aim is to come to grips with subjective aspects of meaning expressed in written texts, such as the attitude or value expressed in them. This paper explores how the structure of the WordNet lexical database might be used to assess affective or emotive meaning. In particular, we construct measures based on Osgood’s semantic differential technique.
By Jaap Kamps and Maarten Marx
Download the full PDF bellow
Notes and Further Exploration
Kamps and Marx present some interesting research that is very applicable to emotively analyzing online discourse. In this paper, subjective understanding is computationally extracted from text using Charles Osgood’s Theory of Semantic Differentiation as a guide for mapping word relationships in Princeton University’s WordNet Lexical Database. Osgood’s work in the late 1950’s established
“semantic differential technique is using several pairs of bipolar adjectives to scale the responses of subjects to words, short phrases, or texts. That is, subjects are asked to rate their meaning on scales like active–passive; good–bad; optimistic–pessimistic; positive–negative; strong–weak; serious–humorous; and ugly–beautifully”
Osgood research further revealed that most variance in affective meaning assigned itself to three major factors:
“These three factors of the affective or emotive meaning are the evaluative factor (e.g., good–bad); the potency factor (e.g., strong-weak); and the activity factor (e.g., active–passive). Among these three factors, the evaluative factor has the strongest relative weight.”
Using these three emotive axises, Kamps and Mark mapped words as they related to “good” and “bad” within the context of the WordNet lexical database through finding minimal path lengths:
The minimal path-length is a straightforward generalization of the synonymy relation. The synonymy relation connects words with similar meaning, so the minimal distance between words says something on the similarity of their meaning.
Words can now be scaled via this process as either negative, neutral, or positive. This same process can be followed to assign activity and potency ratings to words as well, revealing clear emotive qualities of the content.
