SIGML is planning a series of external skype lectures/talks on some recent
 advances in machine learning and NLP. The first lecture is schedule on Jan
 21,7:00 PM on Word Embeddings. All are invited for the same

 *Talk Details* :

 *Title : *Understanding Word Embeddings
 *Speaker : *Omer Levy ( PhD student at Bar-Ilan University’s Natural
 Language Processing lab )
 *Time : *21 January 201
 ​6​
 , 7:00 PM - 8:00 PM
 *Venue* : KD 101 (Tentative)



 *Abstract*
 Neural word embeddings, such as word2vec (Mikolov et al., 2013), have
 become increasingly popular in both academic and industrial NLP. These
 methods attempt to capture the semantic meanings of words by processing
 huge unlabeled corpora with methods inspired by neural networks and the
 recent onset of Deep Learning. The result is a vectorial representation of
 every word in a low-dimensional continuous space. These word vectors
 exhibit interesting arithmetic properties (e.g. king - man + woman = queen)
 (Mikolov et al., 2013), and seemingly outperform traditional vector-space
 models of meaning inspired by Harris's Distributional Hypothesis (Baroni et
 al., 2014). Our work attempts to demystify word embeddings, and understand
 what makes them so much better than traditional methods at capturing
 semantic properties.

 Our main result shows that state-of-the-art word embeddings are actually
 "more of the same". In particular, we show that skip-grams with negative
 sampling, the latest algorithm in word2vec, is implicitly factorizing a
 word-context PMI matrix, which has been thoroughly used and studied in the
 NLP community for the past 20 years. We also identify that the root of
 word2vec's perceived superiority can be attributed to a collection of
 hyperparameter settings. While these hyperparameters were thought to be
 unique to neural-network-inspired embedding methods, we show that they can,
 in fact, be ported to traditional distributional methods, significantly
 improving their performance. Among our qualitative results is a method for
 interpreting these seemingly-opaque word-vectors, and the answer to why
 king - man + woman = queen.

 (Based on joint work with Yoav Goldberg and Ido Dagan.)


 *Bio :*
 Omer is a Computer Science PhD student at Bar-Ilan University’s Natural
 Language Processing lab, working with Prof. Ido Dagan and Dr. Yoav Goldberg.

 Omer is interested in realizing high-level semantic applications such as
 question answering and summarization to help people cope with information
 overload. At the heart of these applications are challenges in textual
 entailment and semantic similarity, which form the core of Omer's current
 research. He is also interested in the current advances in representation
 learning (aka “deep learning”), particularly in the scope of word
 embeddings, and how they can support semantic applications.