Computer Science and Engineering
University Of California, San Diego

CSE 258a

Connectionist Approaches to Natural Language Processing

Winter 2006

Professor Gary Cottrell, CSE

Course Description

This  course  will  cover  connectionist  approaches  to  Natural Language Processing (NLP) through reading selected papers in  the field.  Rather than papers trying to achieve good performance for computational linguistics  applications,  the  emphasis  will  be placed  on cognitive modeling papers.  Because of the vast number of papers in this field, after some historical context  has  been set,  emphasis  will  be placed on distributed approaches to NLP,  rather than structured connectionist approaches.  Students should have  some background in neural nets or NLP or (preferably) both. We will be reading at least two papers a week (some  are  short). Course  credit will be obtained by answering questions in class, summarizing papers for the professor (made available to the class *after*  the discussion - no simply reading the summary!), and by  implementing a final project of  your  choosing  with  a  writeup suitable  for  framing  or submission to a conference.  This last requirement is the most important - so it is best if  you  really already  know your way around a neural net. These final projects, however, can be done in small teams of two or three.  Of  course, the larger the team, the more complex the project.

Course Information

  •   Meets Tues/Thursday 12:30-1:50 Center Hall 224C
  • Instructor: Gary Cottrell

    • Mailing list

    I will set up an email list. Please send me your preferred email address. I may occasionally email pdfs.

    Meeting Schedule & Readings

    DATE

    SPEAKER

    PAPERS

    01/10/06

    Gary Cottrell

    		 My thesis
    01/12/06
    		 all of us
    		 Kawamoto, Alan Distributed representations of words and their resolution in a connectionist network.


    		Same paper, but higher contrast scanning.
    01/17/06
    		 all of us
    		 Kawamoto, A.H.; Farrar, W.T.; and Kello, C.T. When two meanings are better than one: Modeling the ambiguity advantage using a recurrent distributed network. Journal of Experimental Psychology: Human Perception and Performance, 1994, 20, 1233-1247.
    01/24/06
    		 all of us
    		 Burgess, C. & Lund, K. (2000) The dynamics of meaning in memory. In Dietrich & Markman (Eds.), Cognitive Dynamics: Conceptual Change in Humans and Machines.
    01/24/06
    		 SUPPLEMENTARY READING
    		 Burgess, C. (2001). Representing and resolving semantic ambiguity: A contribution from high-dimensional memory modeling. In Gorfein, D.S. (Ed.), On the Consequences of Meaning Selection: Perspectives on Resolving Lexical Ambiguity. APA Press.
    01/31/06
    		 all of us
    		 Jeff Elman (1990) Finding Structure in Time. Cognitive Science14:179-211
    02/02/06
    		 all of us
    		 READ THIS ONE: Kim Plunkett and Virginia Marchman (1993) From Rote Learning to System Building (author's preprint 1993 Cognition paper)

    NOT THIS ONE: Kim Plunkett and Virginia Marchman (1990) From Rote Learning to System Building (Tech report version of 1993 Cognition paper


    Thursday Feb 16
    all of us!
    Herbert Jaeger. Echo state networks. (Tech report)
    02/23/06
    		 all of us
    		 St. John, Mark and McClelland, James (1990) Learning and applying contextual constraints in sentence comprehension. Artificial Intelligence 46:217-256.
    March 2, 2006
    		 all of us!
    		 Shimon Edelman, Zach Solan, David Horn, Eytan Ruppin (2004) Bridging computational, formal and psycholinguistic approaches to language
    In Proceedings of the 2004 Cognitive Science Society Conference, Chicago, Ill.
    March 2, 2006
    		 all of us!
    		 Unsupervised learning of natural languages Zach Solan, David Horn, Eytan Ruppin and Shimon Edelman (2005) Proceedings of the National Academy of Sciences102(33):11629-11634.
    march 2, 2006
    		Supplementary material - we won't go over this in class.
    March 7, 2006
    		 all of us!
    		 Henderson, James (2003) Inducing History Representations for Broad Coverage Statistical Parsing. In Proc. North American Chap. Assoc. Computational Linguistics and Human Language Technology Conf. (HLT-NAACL 2003).
    March 7, 2006
    		 all of us
    		 Henderson, James (2004). Discriminative Training of a Neural Network Statistical Parser. In Proc. 42nd Meeting of Association for Computational Linguistics (ACL 2004), Barcelona, Spain, 2004.



















































    Acknowledgement: Special thanks to Serge Belongie, who kindly provided me with the web site template for his highly successful CSE291 seminar, based upon the web site template kindly provided him by Charles Elkan, from his highly successful CSE 254 seminar!

    Most recently updated on January 10th, 2006