⇐ ← → ⇒ SE367 Term Project ~ Agrim Gupta Paper Review Hand in Motion reveals Mind in Motion ~ Jonathan B. Freeman Abstract ~ How it spells out Till recently, hand movements, or any other motor actions were considered to be the end result of a complex chain of thought. However recent studies reveal that motor movements are indeed an important effect of the continuous comprehension of language. This evaluation happens continuously and subsconsciously. In this review we try to realise how we can understand cognitive processes through analysing hand motion movements of people using a mouse and some appropriate stimuli. Mouse tracking provides a very succinctly reliable and high-fidelity method for tracing the thought processes going on. Effort has been made to outline the nature of cognitive processes that are suited to this method of cognitive experimentation. Introduction ~ How and why we got here Cognitive psychology has always been understood as the cumulative culmination of several branches of physical and mental understanding. Mental train-of-thoughts, eye-movements, hand-movements all have been studid in great detail and their significance in the understanding of the subject has been proven & understood. Earlier, mental and physical processes were considerd entirely separate, but it now remains clearly proven that they are much closely tied up than thought. Unlike other methods of probing into the cognitive processes - actions and hand movements provide the most direct way. One no longer needs to rely just on the behaviorial outcomes of a stimuli - with mouse tracking, one can see the entire chain of thought process happening when a stimuli is present. Its a continuous, real-time account of the mental dynamics happening. As Magnuson (2005) rightly says: 
Lexical decision yields single postperceptual response times that provide condition means and coarse clues to underlying processing. Eye-tracking trials provide a few discrete data points during processing; averaging over many trials yields time-course estimates of processing. Each mouse-tracking trial provides continuous time-course data (points correspond to normalized time steps), providing the most direct measure of time course and making it amenable to dynamical systems theory (DST) analysis. An image pertaining to this thought is given below More Methods ~ How extensively can we test hand movements A major obstacle in understanding motor movements is understanding the nature of language - as a set of discrete tools or as a continuously evaluated factor. Spivey et al. (2005) did extensive work to understand this nature using mouse tracking by using the 'click-the-candle' experiment. His experiments showed that language is continuously evaluated and as the listener aggregates the acoustic-phonetic input, his motor movements can change mid-way. triggering parallel competing representations that resolve over time. This permitted a discrimination among theoretical models previously not possible. Dale et. al (2008) conducted experiments on hand movements over longer periods of time and found out that besides the reaction to other stimuli occuring around them, the participants showed a certain level of learning of the task. Dale et. al performed this by making people associate together random pairs of shapes by making them choose complimentary paired shapes using wireless Nintendo Wii-mote. Such gesture patterns and hand movements have long been understood to be evolving with time and practice. This was demonstrated by Goldin-Meadow & Alibali (2002). This effect was much evident amongst children. Expertise of a certain task results in far more different patterns of hand/mouse movements than for novices. A good example to study that is through IOGraph a software used for mouse-tracking pattern development. From the above IOGraph session we can clearly see the various patterns of mouse movement happening during 3 hours of photoshopping. Three hours of photoshopping for the same task by a different person with a different level of photoshop expertise will lead to a totally different IOGraph. This gives insight into the style of application usage based on expertise and is regularly used by software development companies to improve there applications. Scope of Study ~ Where can we go? Amongst all the studies, there is a lack of clear understanding of the nature of mental thought processes involved during cognitive- stimuli based motor decisions. Mouse tracking inrecent times based on the competitive attention objects model as demonstrated by Farmer et al. (2007b). Another issue to understand is the reaction time and its distributions based on the stimulus provided. Finally, we can relate various findings over long periods of time to understand the mouse usage for different appplications and diffferent levels of expertise. References ~ Where did I look? Magnuson, J. S. (2005). Moving hand reveals dynamics of thought. Proc. Natl. Acad. Sci. U.S.A. 102, 9995–9996. Spivey, M. J.,Grosjean, M., and Knoblich, G. (2005). Continuous attraction toward phonological competitors. Proc. Natl. Acad. Sci. 102, 10393–10398.Dale, R., Roche, J., Snyder, K., and McCall, R. (2008). Exploring action dynamics as an index of paired-associate learning. PLoS ONE 3, e1728. doi: 10.1371/journal.pone.0001728Goldin-Meadow, S., and Alibali, M. W. (2002).“Looking at the hands through time: a microgenetic perspective on learning and instruction,” in Microdevelopment: Transition Processes in Development and Learning, eds N. Granott and J. Parziale (Cambridge: Cambridge University Press), 80–105. Farmer, T. A., Cargill, S., Hindy, N., Dale, R., and Spivey, M. J. (2007b). Tracking the continuity of language comprehension: computer-mouse trajectories suggest parallel syntactic processing. Cogn. Sci. 31, 889–909. SE367 Introduction to Cognitive Science By Agrim
