topics: |  neuro | computer | simulation | neural-ann

ch3: Networks

... there are two general classes of neurons that have been identified in the
cortex: excitatory neurons that release the excitatory neurotransmitter
glutamate, and inhibitory neurons that release the inhibitory
neurotransmitter GABA... 

There are two primary subtypes of excitatory neurons, the pyramidal and spiny
stellate neurons, and a larger number of different subtypes of inhibitory
neurons, with the chandelier and basket being some of the most prevalent
(figure 3.1). The excitatory neurons constitute roughly 85 percent of the
total number of neurons in the cortex (White, 1989a), and are apparently
responsible for carrying much of the information flow, because they form
long-range projections to different areas of the cortex and to subcortical
areas. Thus, most of the following discussion of connectivity is focused on
these excitatory neurons. Although the inhibitory neurons receive both
long-range and localized inputs, they project within small localized areas of
cortex...

Cortical neurons are organized into six distinct layers (figure 3.2). The six
cortical layers have been identified on anatomical grounds and are important
for understanding the detailed biology of the cortex. However, for our
purposes, we can simplify the picture by considering three functional layers:
the input, hidden, and output layers (figure 3.3). We will use the term layer
to refer to these functional layers, and the term cortical layer for the
biologically based layers.

ch10: Language

LEXICON: repository of word-level representations: traditional approaches
   have assumed a centralized, canonical lexicon in the brain where each
   word is uniquely represented. In contrast, our basic principles of
   representation (chapter 7) suggest that word-level representations
   should be distributed across a number of different pathways specialized
   for processing different aspects of words.

This idea of a distributed lexicon has been championed by those who model
language from the neural network perspective (e.g., Seidenberg & McClelland,
1989; Plaut, 1997). We begin this chapter with a model instantiating this
idea, where orthographic (written word forms), phonological (spoken word
forms), and semantic (word meaning) representations interact during basic
language tasks such as reading for meaning, reading aloud, speaking, and so
forth. The orthographic and phonological pathways constitute specialized
perceptual and motor pathways, respectively, while the semantic
representations likely reside in higher-level association areas. In this
model, activation in any one of these areas can produce appropriate
corresponding activation in the other areas. Furthermore, interesting
dependencies develop among the pathways, as revealed by damage to one or more
of the pathways. Specifically, by damaging different parts of this model, we
simulate various forms of acquired dyslexia—disorders in reading that can
result from brain damage.
[Phonological repr is motor - what of auditory?]

The visual word perception pathway appears to be located within the ventral
object recognition pathway, and can be viewed as a specialized version of
object recognition.  Thus, we apply the basic principles of visual object
recognition from chapter 8 to this model. We focus on the model’s ability to
generalize its knowledge of the orthography–phonology mapping to the
pronunciation of nonwords (e.g., “nust,” “mave”), according to the
regularities of the English language. These generalization tests reveal the
model’s ability to capture the complex nature of these regularities.

Another extension of the distributed lexicon model explores the production of
properly inflected verbs...  We focus on the past-tense inflectional system,
which has played a large role in the application of neural networks to
language phenomena. Developmentally, children go through a period where they
sometimes overregularize the regular past-tense inflection rule (i.e., add
the suffix -ed), for example producing goed instead of
went. Overregularization has been interpreted as evidence for a rule-based
system that overzealously applies its newfound rule. However, neural networks
can simulate the detailed pattern of overregularization data, so a separate
rule-based system is unnecessary. We will see that the correlational
sensitivity of Hebbian learning, combined with error-driven learning, may be
important for capturing the behavioral phenomena.

A third extension of the distributed lexicon model explores the ultimate
purpose of language, which is to convey meaning (semantics). We assume that
semantic representations in the brain involve the entirety of the
associations between language representations and those in the rest of the
cortex, and are thus complex and multifaceted. Language input may shape
semantic representations by establishing co-occurrence relationships among
different words, such that words that co-occur together are likely to be
semantically related. Landauer and Dumais (1997) have shown that a
Hebbian-like PCA-based mechanism can develop useful semantic representations
from word co-occurrence in large bodies of text, and that these
representations appear to capture common-sense relationships among words. We
explore a model of this idea using the CPCA Hebbian learning developed in
chapter 4.

The neural basis of semantic representations is a very complicated and
contentious issue, but one that neural network models have made important
contributions to, as we will discuss in more detail in section 10.6. Part of
the complication is that ... there are visual, auditory, and functional
semantics that are most likely associated with the cortical areas that
process the relevant kind of information (e.g., visual cortex for visual
semantics). The result is that virtually every part of the cortex can make a
semantic contribution, and it is therefore very difficult to provide a
detailed account of “the” neural basis of semantics (e.g., Farah &
McClelland, 1991; Damasio, Grabowski, & Damasio, 1996).  Certainly,
Wernicke’s area is only a very small part of the semantics story.

10.2.2. Phonology

The human speech production system is based on vibrating and modulating air
expelled from the lungs up through the vocal cords (also known as the
glottis) and out the mouth and nose. This pathway is called the vocal
tract.

If the vocal cords are open, they do not vibrate when air passes through
them. For speech sounds made with open cords, the phoneme is said to be
UNVOICED, whereas it is VOICED if the cords are closed and vibrating.
Vowels are always voiced...
  /s/ is unvoiced; tongue pushed up against the gums (alveolar ridge)
  /z/ very similar - except it is voiced (vocal cords are closed).

the consonants are typically produced by restricting airflow with
   - location (lb=labial=lips, ld=labio-dental=lips-teeth, dt=dental=teeth,
     	      al=alveolar=gums, pl=palatal=palate, vl=velar=soft palate,
	      gl=glottal=epiglottis),
   -  manner (ps=plosive, fr=fricative, sv=semi-vowel, lq=liquid, ns=nasal),

MANNER (restrictions): 
 * PLOSIVE: a restriction as in the phoneme /p/ (“push”) —the air
   is restricted and then has an “explosive” burst through the restriction. 
 * FRICATIVE: constant “friction” sound, like the phoneme /s/.
 * SEMIVOWEL (also known as GLIDE) is a consonant that is produced a
   lot like a vowel, without much restriction, such as the phoneme /y/ as in
   “yes.” 
 * LIQUID: smooth and “liquid” sound, like the phoneme /l/ in
   “lit.” 
 * NASAL restriction: involves a complete blockage of the air out the
   mouth, so that the nose becomes the primary outlet (e.g., in the phoneme
   /n/ as in “nun”).

The representation is vowel centered, with slots on each side for the onset
and coda consonants that surround the word.


[AS A MATTER OF INTEREST: 

Sanskrit / Indo-European consonant structure

The (5x5 varga) originatied in the shikShA tradition, taught via the
ancient phonetic prAtishAkhhya texts, abt 1000BCE, 

stop consonants 
         voiceless         voiced      nasal
   inaspirate aspirated     aspirated  
	k	kh	g	gh	N 	[velar]
	c	chh	j[dz]	jh[dzh]	n~	[palatal] (Alveolar laminal affricates)
	T	Th	D	Dh	N	[retroflex] (Alveolar apical stops)
	t	th	d	dh	n	[dental]
	p	ph	b	bh	m	[labial] (bilabial)


fricatives, sibilants, semi-vowerls
	y	r	l	v/w		(Glides and liquids)
     appr    vibrant liquid  approximant
	Sh	sh 	s	h		(Alveolar and velar fricatives)
	retrof  palatal	dental

approximants: y, v

]