Homepage with submission
Submission website: http://mc.manuscriptcentral.com/tamd-ieee
Website for Table of Contents, Abstracts & Authors' emails: http://research.microsoft.com/~zhang/IEEE-TAMD/
Date of Publication: December 2012
Link to IEEE: http://ieeexplore.ieee.org/xpl/tocresult.jsp?isnumber=6376146&punumber=4563672
(Previous issue: Vol. 4, No. 3, June 2012)
Abstract: Gaze following, the ability to redirect one's visual attention to look at what another person is seeing, is foundational for imitation, word learning, and theory-of-mind. Previous theories have suggested that the development of gaze following in human infants is the product of a basic gaze following mechanism, plus the gradual incorporation of several distinct new mechanisms that improve the skill, such as spatial inference, and the ability to use eye direction information as well as head direction. In this paper, we offer an alternative explanation based on a single learning mechanism. From a starting state with no knowledge of the implications of another organism's gaze direction, our model learns to follow gaze by being placed in a simulated environment where an adult caregiver looks around at objects. Our infant model matches the development of gaze following in human infants as measured in key experiments that we replicate and analyze in detail.
Full Text from IEEE: PDF (1919KB); Contact the author by email for a copy.
Abstract: Reservoir computing (RC) is a computational framework for neural network based information processing. Little work, however, has been conducted on adapting the structure of the neural reservoir. In this paper, we propose a developmental approach to structural self-organization in reservoir computing. More specifically, a recurrent spiking neural network is adopted for building up the reservoir, whose synaptic and structural plasticity are regulated by a gene regulatory network (GRN). Meanwhile, the expression dynamics of the GRN is directly influenced by the activity of the neurons in the reservoir. We term this proposed model as GRN-regulated self-organizing RC (GRN-SO-RC). Contrary to a randomly initialized and fixed structure used in most existing RC models, the structure of the reservoir in the GRN-SO-RC model is self-organized to adapt to the specific task using the GRN-based mechanism. To evaluate the proposed model, experiments have been conducted on several benchmark problems widely used in RC models, such as memory capacity and nonlinear auto-regressive moving average. In addition, we apply the GRN-SO-RC model to solving complex real-world problems, including speech recognition and human action recognition. Our experimental results on both the benchmark and real-world problems demonstrate that the GRN-SO-RC model is effective and robust in solving different types of problems
Full Text from IEEE: PDF (2751KB); Contact the author by email for a copy.
Abstract: Neural circuits that route motor activity to sensory structures play a fundamental role in perception. Their purpose is to aid basic cognitive processes by integrating knowledge about an organism's actions and to predict the perceptual consequences of those actions. This work develops a biologically inspired model of a visual stimulus prediction circuit and proposes a mathematical formulation for a computational implementation. We consider an agent with a visual sensory area consisting of an unknown rigid configuration of light-sensitive receptive fields which move with respect to the environment and according to a given number of degrees of freedom. From the agent's perspective, every movement induces an initially unknown change to the recorded stimulus. In line with evidence collected from studies on ontogenetic development and the plasticity of neural circuits, the proposed model adapts its structure with respect to experienced stimuli collected during the execution of a set of exploratory actions. We discuss the tendency of the proposed model to organize such that the prediction function is built using a particularly sparse feedforward network which requires a minimum amount of wiring and computational operations. We also observe a dualism between the organization of an intermediate layer of the network and the concept of self-similarity.
Full Text from IEEE: PDF (2534KB); Contact the author by email for a copy.
Abstract: For robots to be accommodated in human spaces and in daily human activities, robots should be able to understand messages from their human conversation partner. In the same light, humans must also understand the messages that are being communicated to them by robots, including nonverbal messages. We conducted a Web-based video study wherein participants interpreted the iconic gestures and emblems produced by an anthropomorphic robot. Out of the 15 robotic gestures presented, we found 6 that can be accurately recognized by the human observer. These were nodding, clapping, hugging, expressing anger, walking, and flying. We review these gestures for their meaning from literature on human and animal behavior. We conclude by discussing the possible implications of these gestures for the design of social robots that are able to have engaging interactions with humans.
Full Text from IEEE: PDF (1336KB); Contact the author by email for a copy.
Abstract: Incorporating intrinsic motivation with reinforcement learning can permit agents to independently choose, which skills they will develop, or to change their focus of attention to learn different skills at different times. This implies an autonomous developmental process for skills in which a skill-acquisition goal is first identified, then a skill is learned to solve the goal. The learned skill may then be stored, reused, temporarily ignored or even permanently erased. This paper formalizes the developmental process for skills by proposing a goal-lifecycle using the option framework for motivated reinforcement learning agents. The paper shows how the goal-lifecycle can be used as a basis for designing motivational state-spaces that permit agents to reason introspectively and autonomously about when to learn skills to solve goals, when to activate skills, when to suspend activation of skills or when to delete skills. An algorithm is presented that simultaneously learns: 1) an introspective policy mapping motivational states to decisions that change the agent's motivational state, and 2) multiple option policies mapping sensed states and actions to achieve various domain-specific goals. Two variations of agents using this model are compared to motivated reinforcement learning agents without introspection for controlling non-player characters in a computer game scenario. Results show that agents using introspection can focus their attention on learning more complex skills than agents without introspection. In addition, they can learn these skills more effectively.
Full Text from IEEE: PDF (1788KB); Contact the author by email for a copy.
Abstract: For humans and robots, variable impedance control is an essential component for ensuring robust and safe physical interaction with the environment. Humans learn to adapt their impedance to specific tasks and environments; a capability which we continually develop and improve until we are well into our twenties. In this article, we reproduce functionally interesting aspects of learning impedance control in humans on a simulated robot platform. As demonstrated in numerous force field tasks, humans combine two strategies to adapt their impedance to perturbations, thereby minimizing position error and energy consumption: 1) if perturbations are unpredictable, subjects increase their impedance through cocontraction; and 2) if perturbations are predictable, subjects learn a feed-forward command to offset the perturbation. We show how a 7-DOF simulated robot demonstrates similar behavior with our model-free reinforcement learning algorithm PI2, by applying deterministic and stochastic force fields to the robot's end-effector. We show the qualitative similarity between the robot and human movements. Our results provide a biologically plausible approach to learning appropriate impedances purely from experience, without requiring a model of either body or environment dynamics. Not requiring models also facilitates autonomous development for robots, as prespecified models cannot be provided for each environment a robot might encounter.
Full Text from IEEE: PDF (2328KB); Contact the author by email for a copy.