This paper seeks to foster a discussion on whether experiments with robots can inform theory in infant motor development and specifically (1) how the interactions among the parts of a system, including the nervous and musculoskeletal systems and the forces acting on the body, induce organizational changes in the whole, and (2) how exploratory behaviour and selective informational signals at the timescale of skill learning may allow behaviour to become stabilized at the longer timescale of development. The paper describes how three generative principles, inspired from developmental biology and shown to underlie the dynamics of infants learning to bounce in a Jolly Jumper, were broken into a set of mechanisms suitable for controlling a robotic system and resulted in a similar developmental profile. A comparison of infant and robot data leads to a set of criteria for improving the usefulness of robotic studies.

In the present report, we describe a new theoretical framework for thinking about the dynamics of visual cognition—the dynamic field theory. This framework, grounded in neural principles, grew out of the dynamic systems approach to motor control and development. The initial development of the dynamic field theory extended concepts of the motor approach to decision making in a sensori-motor context, and, more recently, to the dynamics of spatial cognition. Here we extend these concepts still further to address topics in visual cognition, including visual working memory for non-spatial object properties, the processes that underlie change detection, and the ‘binding problem’ in vision. In each case, we demonstrate that the general principles of the dynamic field approach can unify findings in the literature and generate novel predictions. We contend that the application of these concepts to visual cognition avoids the pitfalls of reductionist approaches in cognitive science, and points toward a formal integration of brains, bodies, and behavior.

In this paper we discuss the concept of autobiographic agent and how memory may extend an agent's temporal horizon and increase its adaptability. These concepts are applied to an implementation of a scenario where agents are interacting in a complex virtual artificial life environment. We present computational memory architectures for autobiographic virtual agents that enable agents to retrieve meaningful information from their dynamic memories which increases their adaptation and survival in the environment. The design of the memory architectures, the agents, and the virtual environment are described in detail. Next, a series of experimental studies and their results are presented which show the adaptive advantage of autobiographic memory, i.e. from remembering significant experiences. Also, in a multi-agent scenario where agents can communicate via stories based on their autobiographic memory, it is found that new adaptive behaviours can emerge from an individual's reinterpretation of experiences received from other agents whereby higher communication frequency yields better group performance. An interface is described that visualizes the memory contents of an agent. From an observer perspective, the agents' behaviours can be understood as individually structured, and temporally grounded, and, with the communication of experience, can be seen to rely on emergent mixed narrative reconstructions combining the experiences of several agents. This research leads to insights into how bottom-up story-telling and autobiographic reconstruction in autonomous, adaptive agents allow temporally grounded behaviour to emerge. The article concludes with a discussion of possible implications of this research direction for future autobiographic, narrative agents.

Computational models have played a central role in the debate over language learnability. This article discusses how they have been used in different “stances”, from generative views to more recently introduced explanatory frameworks based on embodiment, cognitive development and cultural evolution. By digging into the details of certain specific models, we show how they organize, transform and rephrase defining questions about what makes language learning possible for children. Finally, we present a tentative synthesis to recast the debate using the notion of learning bias.

Robots and humans receive partial, fragmentary hints about the world’s state through their respective sensors. These hints – tiny patches of light intensity, frequency components of sound, etc. – are far removed from the world of objects we feel we perceive so effortlessly around us. The study of infant development and the construction of robots are both deeply concerned with how this apparent gap between the world and our experience of it is bridged. In this paper, we focus on some fundamental problems in perception that have attracted the attention of researchers in both robotics and infant development. Our goal is to identify points of contact already existing between the two fields, and also important questions identified in one field that could fruitfully be addressed in the other. We start with the problem of object segregation: how do infants and robots determine visually where one object ends and another begins? For object segregation, both fields have examined the idea of using “key events” where perception is in some way simplified and the infant or robot acquires knowledge that can be exploited at other times. We propose that the identification of the key events themselves constitutes a point of contact between the fields. And although the specific algorithms used in robots do not necessarily map directly to infant strategies, the overall “algorithmic skeleton” formed by the set of algorithms needed to identify and exploit key events may in fact form a basis for mutual dialogue. We then look more broadly at the role of embodiment in humans and robots, and see the opportunities it affords for development.

Robots and humans receive partial, fragmentary hints about the world’s state through their respective sensors. In this paper, we focus on some fundamental problems in perception that have attracted the attention of researchers in both robotics and infant development: object segregation, intermodal integration, and the role of embodiment. We concentrate on identifying points of contact between the two fields, and also important questions identified in one field and not yet addressed in the other. For object segregation, both fields have examined the idea of using “key events” where perception is in some way simplified and the infant or robot acquires knowledge that can be exploited at other times. We examine this parallel research in some detail. We propose that the identification of the key events themselves constitutes a point of contact between the fields. And although the specific algorithms used in robots are not easy to relate to infant development, the overall “algorithmic skeleton” formed by the set of algorithms needed to identify and exploit key events may in fact form a basis for mutual dialogue.

A central goal in cognitive and developmental science is to develop models of behavior that can generalize across both tasks and development while maintaining a commitment to detailed behavioral prediction. This paper presents tests of one such model, the Dynamic Field Theory (DFT). The DFT was originally proposed to capture delay-dependent biases in spatial recall and developmental changes in spatial recall performance. More recently, the theory was generalized to adults’ performance in a second spatial working memory task, position discrimination. Here, we use the theory to predict a specific, complex developmental pattern in position discrimination. Data with 3- to 6-year-old children and adults confirm these predictions, demonstrating that the DFT achieves generality across tasks and time scales, as well as the specificity necessary to generate novel, falsifiable predictions.

Within cognitive neuroscience, computational models are designed to provide insights into the organization of behavior while adhering to neural principles. These models should provide sufficient specificity to generate novel predictions while maintaining the generality needed to capture behavior across tasks and/or time scales. This paper presents one such model, the Dynamic Field Theory (DFT) of spatial cognition, showing new simulations that provide a demonstration proof that the theory generalizes across developmental changes in performance in four tasks—the Piagetian A-not-B task, a sandbox version of the A-not-B task, a canonical spatial recall task, and a position discrimination task. Model simulations demonstrate that the DFT can accomplish both specificity—generating novel, testable predictions—and generality—spanning multiple tasks across development with a relatively simple developmental hypothesis. Critically, the DFT achieves generality across tasks and time scales with no modification to its basic structure and with a strong commitment to neural principles. The only change necessary to capture development in the model was an increase in the precision of the tuning of receptive fields as well as an increase in the precision of local excitatory interactions among neurons in the model. These small quantitative changes were sufficient to move the model through a set of quantitative and qualitative behavioral changes that span the age range from 8 months to 6 years and into adulthood. We conclude by considering how the DFT is positioned in the literature, the challenges on the horizon for our framework, and how a dynamic field approach can yield new insights into development from a computational cognitive neuroscience perspective.

Infants gradually learn to predict the motion of moving targets and change from a strategy that mainly depends on saccades to one that depends on anticipatory control of smooth pursuit. A model is described that combines three types of mechanisms for gaze control that develops in a way similar to infants. Initially, gaze control is purely reactive, but as the anticipatory models become more accurate, the gain of the pursuit will increase and lead to a larger fraction of smooth eye movements. Finally, a third system learns to predict changes in target motion, which will lead to fast retuning of the parameters in the anticipatory model.

We investigate a computational model of the emergence of gaze following that is based on a generic basic set of mechanisms. Whereas much attention has been focused so far on the study of the infant's behavior, we systematically analyze the caregiver and show that he plays a crucial role in the development of gaze following in our model, especially for infant models with simulated developmental disorders such as autism and Williams syndrome. We first create two reference infant parameter sets and test their behavior with a simple standard caregiver. Based on these findings we then propose new caregiver models and evaluate them on normally developing infants and on infants with simulated developmental disorders. Further, we investigate if and how a pair of infants (with and without simulated developmental disorders) might learn gaze following from scratch, without a mature caregiver. The findings of this paper suggest the pivotal role the caregiver plays for the infant in developing gaze following, that his predictability is the most important criterion, and that different infant models require particular caregivers for gaze following to emerge optimally. Our simulations are consistent with Leekam's finding, that autistics can learn to follow gaze through a contingent presentation of rewarding visual stimuli, but that a lack of motivation may impede learning.

This survey presents an overview of the autonomous development of mental capabilities in computational agents. It does so based on a characterization of cognitive systems as systems which exhibit adaptive, anticipatory, and purposive goal-directed behavior. We present a broad survey of the various paradigms of cognition, addressing cognitivist (physical symbol systems) approaches, emergent systems approaches, encompassing connectionist, dynamical, and enactive systems, and also efforts to combine the two in hybrid systems. We then review several cognitive architectures drawn from these paradigms. In each of these areas, we highlight the implications and attendant problems of adopting a developmental approach, both from phylogenetic and ontogenetic points of view. We conclude with a summary of the key architectural features that systems capable of autonomous development of mental capabilities should exhibit.

This survey presents an overview of the autonomous development of mental capabilities in computational agents. It does so based on a characterization of cognitive systems as systems which exhibit adaptive, anticipatory, and purposive goaldirected behaviour. We present a broad survey of the various paradigms of cognition, addressing cognitivist (physical symbol systems) approaches, emergent systems approaches, encompassing connectionist, dynamical, and enactive systems, and also efforts to combine the two in hybrid systems. We then review several cognitive architectures drawn from these paradigms. In each of these areas, we highlight the implications and attendant problems of adopting a developmental approach, both from phylogenetic and ontogenetic points of view. We conclude with a summary of the key architectural features that systems capable of autonomous development of mental capabilities should exhibit.

Our exploratory research aims at suggesting design principles for educational software dedicated to high functioning autism. In order to explore the efficiency of educational games, we developed an experimental protocol to study the influence of the specific learning field constraints (spatial planning versus dialogue understanding) and Human Computer Interface modalities in this sector. We designed computer games that were tested with 10 teenagers diagnosed with high functioning autism, during 13 sessions, at the rate of one session per week. Participants' skills were assessed before and after a training period. A group of 10 typical children matched on academic level were also presented the experiment. A software platform was developed to manage interface modalities and log users' actions. Moreover, we annotated video recordings of two sessions. Results underline the influence of the task and interface modalities on executive functions.

We propose a model of intrinsic plasticity for a continuous activation model neuron based on information theory. We then show how intrinsic and synaptic plasticity mechanisms interact and allow the neuron to discover heavy-tailed directions in the input. We also demonstrate that intrinsic plasticity may be an alternative explanation for the sliding threshold postulated in the BCM theory of synaptic plasticity. We present a theoretical analysis of the interaction of intrinsic plasticity with different Hebbian learning rules for the case of clustered inputs. Finally, we perform experiments on the "bars" problem, a popular nonlinear independent component analysis problem.

Gaze following is the ability to redirect one's gaze to the location where another agent is looking. We present a computational model of how human infants or other agents may acquire gaze following by learning to predict the locations of interesting sights from the looking behavior of other agents through reinforcement learning. The model accounts for many findings about the development of gaze following in human infants. During learning, the model develops pre-motor representations that exhibit many properties characteristic of mirror neurons, but they are specific to looking behaviors. The existence of such a new class of mirror neurons is the main prediction of our model. The model also offers a parsimonious account of how these and possibly other mirror neurons may acquire their special response properties. In this account, visual representations of other agents' actions become associated with premotor neurons that represent the intention to perform corresponding actions. The model also demonstrates how this development may be obstructed in autism spectrum disorder, giving rise to specific physiological and anatomical differences in the mirror system.

This paper presents Incremental Hierarchical Discriminant Regression (IHDR) which incrementally builds a decision tree or regression tree for very high dimensional regression or decision spaces by an online, real-time learning system. Biologically motivated, it is an approximate computational model for automatic development of associative cortex, with both bottom-up sensory inputs and top-down motor projections. At each internal node of the IHDR tree, information in the output space is used to automatically derive the local subspace spanned by the most discriminating features. Embedded in the tree is a hierarchical probability distribution model used to prune very unlikely cases during the search. The number of parameters in the coarse-to-fine approximation is dynamic and data-driven, enabling the IHDR tree to automatically fit data with unknown distribution shapes (thus, it is difficult to select the number of parameters up front). The IHDR tree dynamically assigns long-term memory to avoid the loss-of-memory problem typical with a global-fitting learning algorithm for neural networks. A major challenge for an incrementally built tree is that the number of samples varies arbitrarily during the construction process. An incrementally updated probability model, called sample size dependent negative-log-likelihood (SDNLL) metric is used to deal with large-sample size cases, small-sample size cases, and unbalanced-sample size cases, measured among different internal nodes of the IHDR tree. We report experimental results for four types of data: synthetic data to visualize the behavior of the algorithms, large face image data, continuous video stream from robot navigation, and publicly available data sets that use human defined features. Keywords: online learning, incremental learning, cortical development, discriminant analysis, local invariance, plasticity, decision trees, high dimensional data, classification, regression, and autonomous development.

Currently, there is a lack of general-purpose, in-place learning engines that incrementally learn multiple tasks, to develop “soft” multi-task-shared invariances in the intermedi- ate internal representation while a developmental robot interacts with its environment. In-place learning is a biologically inspired concept, rooted in the genomic equivalence principle, meaning that each neuron is responsible for its own development while interact- ing with its environment. With in-place learning, there is no need for a separate learning network. Computationally, biologically inspired, in-place learning provides unusually effi- cient learning algorithms whose simplicity, low computational complexity, and generality are set apart from typical conventional learning algorithms. We present in this paper the multiple-layer in-place learning network (MILN) for this ambitious goal. As a key requirement for autonomous mental development, the network enables both unsupervised and supervised learning to occur concurrently, depending on whether motor supervision signals are available or not at the motor end (the last layer) during the agent’s interac- tions with the environment. We present principles based on which MILN automatically develops invariant neurons in different layers and why such invariant neuronal clusters are important for learning later tasks in open-ended development. From sequentially sensed sensory streams, the proposed MILN incrementally develops a hierarchy of inter- nal representations. The global invariance achieved through multi-layer invariances, with increasing invariance from early layers to the later layers. Experimental results with statistical performance measures are presented to show the effects of the principles. Keywords: In-place learning; positional invariance; size invariance; style invariance; incremental learning; internal representation; biologically inspired network.

Three different models of tactile shape perception inspired by the human haptic system were tested using an 8 d.o.f. robot hand with 45 tactile sensors. One model is based on the tensor product of different proprioceptive and tactile signals and a self-organizing map (SOM). The two other models replace the tensor product operation with a novel self-organizing neural network, the Tensor-Multiple Peak Self-Organizing Map (T-MPSOM). The two T-MPSOM models differ in the procedure employed to calculate the neural activation. The computational models were trained and tested with a set of objects consisting of hard spheres, blocks and cylinders. All the models learned to map different shapes to different areas of the SOM, and the tensor product model as well as one of the T-MPSOM models also learned to discriminate individual test objects.

Children seem to acquire new know-how in a continuous and open-ended manner. In this paper, we hypothesize that an intrinsic motivation to progress in learning is at the origins of the remarkable structure of children’s developmental trajectories. In this view, children engage in exploratory and playful activities for their own sake, not as steps toward other extrinsic goals. The central hypothesis of this paper is that intrinsically motivating activities correspond to expected decrease in prediction error. This motivation system pushes the infant to avoid both predictable and unpredictable situations in order to focus on the ones that are expected to maximize progress in learning. Based on a computational model and a series of robotic experiments, we show how this principle can lead to organized sequences of behavior of increasing complexity characteristic of several behavioral and developmental patterns observed in humans.We then discuss the putative circuitry underlying such an intrinsic motivation system in the brain and formulate two novel hypotheses. The first one is that tonic dopamine acts as a learning progress signal. The second is that this progress signal is directly computed through a hierarchy of microcortical circuits that act both as prediction and metaprediction systems.

This article presents recent research investigating how a robot equipped with an intrinsic motivation system can explore its environment and learn a sequence of tasks not initially specified by its programmer. A generic software architecture controls the robot, driving it towards situations where learning progress is maximal. These situations – called « progress niches » - depend on opportunities offered by the environment but also on the robot’s morphology, specific cognitive biases and past experiences. First results have been obtained in the field of locomotion, affordance discovery and prelinguistic communication. In each of these experiments, the robot explores situations that it evaluated as « interesting » given its learning capabilities and biases of its sensorimotor space. The article discusses the results of these initial experiments and concludes on the relevance of this research to offer neurosciences and psychology – that inspired these investigations in the first place – new kinds of concepts and explanations to think about developmental processes in the young child.

Robot imitation is a useful and promising alternative to robot programming. Robot imitation involves two crucial issues. The first is how a robot can imitate a human whose physical structure and properties differ greatly from its own. The second is how the robot can generate various motions from finite programmable patterns (generalization). This paper describes a novel approach to robot imitation based on its own physical experiences. We considered the target task of moving an object on a table. For imitation, we focused on an active sensing process in which the robot acquires the relation between the object's motion and its own arm motion. For generalization, we applied the RNNPB (recurrent neural network with parametric bias) model to enable recognition/generation of imitation motions. The robot associates the arm motion which reproduces the observed object's motion presented by a human operator. Experimental results proved the generalization capability of our method, which enables the robot to imitate not only motion it has experienced, but also unknown motion through nonlinear combination of the experienced motions.

We present a developmental perspective of robot learning that uses affordances as the link between sensory-motor coordination and imitation. The key concept is a general model for affordances able to learn the statistical relations between actions, object properties and the effects of actions on objects. Based on the learned affordances, it is possible to perform simple imitation games providing both task interpretation and planning capabilities. To evaluate the approach, we provide results of affordance learning with a real robot and simple imitation games with people.

In this paper, we present a strategy whereby a robot acquires the capability to learn by imitation following a developmental pathway consisting on three levels: 1) sensory-motor coordination; 2) world interaction; and 3) imitation. With these stages, the system is able to learn tasks by imitating human demonstrators. We describe results of the different developmental stages, involving perceptual and motor skills, implemented in our humanoid robot, Baltazar. At each stage, the system's attention is drawn toward different entities: its own body and, later on, objects and people. Our main contributions are the general architecture and the implementation of all the necessary modules until imitation capabilities are eventually acquired by the robot. Also, several other contributions are made at each level: learning of sensory-motor maps for redundant robots, a novel method for learning how to grasp objects, and a framework for learning task description from observation for program-level imitation. Finally, vision is used extensively as the sole sensing modality (sometimes in a simplified setting) avoiding the need for special data-acquisition hardware

We motivate and present a definition of an embodied, grounded individual sensorimotor interaction history, which captures the time-extended behavior characteristics of humans and many animals. We present an architecture that connects temporally extended individual experience with capacity for action, whereby a robot can develop over ontogeny through interaction. Central to this is an information theoretic metric space of sensorimotor experience, which is dynamically constructed and reconstructed as the robot acts. We present results of robotic experiments that establish the predictive efficacy of the space and we show the robot developing the capacity to play the simple interaction game peekaboo. A quantitative investigation of the appropriate horizon length of experience for the game reveals the relationship between the length of experience and the cycle time of interaction, and suggests the importance of multiple, and possibly self-adaptive, horizon lengths.

We motivate and present a definition of an embodied, grounded individual sensorimotor interaction history, which captures the time-extended behavior characteristics of humans and many animals. We present an architecture that connects temporally extended individual experience with capacity for action, whereby a robot can develop over ontogeny through interaction. Central to this is an information theoretic metric space of sensorimotor experience, which is dynamically constructed and reconstructed as the robot acts. We present results of robotic experiments that establish the predictive efficacy of the space and we show the robot developing the capacity to play the simple interaction game "peekaboo." A quantitative investigation of the appropriate horizon length of experience for the game reveals the relationship between the length of experience and the cycle time of interaction, and suggests the importance of multiple, and possibly self-adaptive, horizon lengths.

Intrinsic plasticity (IP) refers to a neuron's ability to regulate its firing activity by adapting its intrinsic excitability. Previously, we showed that model neurons combining a model of IP based on information theory with Hebbian synaptic plasticity can adapt their weight vector to discover heavy-tailed directions in the input space. In this paper we show how a network of such units can solve a standard non-linear independent component analysis (ICA) problem. We also present a model for the formation of maps of oriented receptive fields in primary visual cortex and compare our results with those from ICA. Together, our results indicate that intrinsic plasticity that tries to locally maximize information transmission at the level of individual neurons may play an important role for the learning of efficient sensory representations in the cortex.

Computational models have played a central role in the debate over language learnability. This article discusses how they have been used in different “stances”, from generative views to more recently introduced explanatory frameworks based on embodiment, cognitive development and cultural evolution. By digging into the details of certain specific models, we show how they organize, transform and rephrase defining questions about what makes language learning possible for children. Finally, we present a tentative synthesis to recast the debate using the notion of learning bias.

Intrinsic motivation, the causal mechanism for spontaneous exploration and curiosity, is a central concept in developmental psychology. It has been argued to be a crucial mechanism for open-ended cognitive development in humans, and as such has gathered a growing interest from developmental roboticists in the recent years. The goal of this paper is threefold. First, it provides a synthesis of the different approaches of intrinsic motivation in psychology. Second, by interpreting these approaches in a computational reinforcement learning framework, we ar- gue that they are not operational and even sometimes inconsistent. Third, we set the ground for a systematic operational study of intrinsic motivation by presenting a formal typology of possible computational approaches. This typology is partly based on existing computational models, but also presents new ways of conceptualizing intrinsic motivation. We argue that this kind of computational typology might be useful for opening new avenues for research both in psychology and developmental robotics.

This paper presents computational experiments that illustrate how one can precisely conceptualize language evolution as a Darwinian process. We show that there is potentially a wide diversity of replicating units and replication mechanisms involved in language evolution. Computational experiments allow us to study systemic properties coming out of populations of linguistic replicators: linguistic replicators can adapt to specific external environments; they evolve under the pressure of the cognitive constraints of their hosts, as well as under the functional pressure of communication for which they are used; one can observe neutral drift; coalitions of replicators may appear, forming higher level groups which can themselves become subject to competition and selection.

The problem of developing local reactive obstacle-avoidance behaviors by a mobile robot through online real-time learning is considered. The robot operated in an unknown bounded 2-D environment populated by static or moving obstacles (with slow speeds) of arbitrary shape. The sensory perception was based on a laser range finder. A learning-based approach to the problem is presented. To greatly reduce the number of training samples needed, an attentional mechanism was used. An efficient, real-time implementation of the approach was tested, demonstrating smooth obstacle-avoidance behaviors in a corridor with a crowd of moving students as well as static obstacles. Key words: Humanoid, Reactive Collision Avoidance, Range Finder, Perception

This paper focuses on three connected to each other fundamental beliefs that appear to be unconsciously taken for granted at the basis of modern cognitive neuroscience: (i) the attractor hypothesis, (ii) the neural code hypothesis, and (iii) the supervenience hypothesis about human subjective experience. Their precise understanding and experimental verification is an important challenge that calls for a scientific paradigm change and needs to be addressed for future progress in cognitive neuroscience. The work presents a theoretical analysis and preliminary experimental and modeling results that suggest a roadmap to solving the challenge.

Imitative learning and learning by observation are social mechanisms that allow a robot to acquire knowledge from a human or another robot. However to be able to obtain skills in this way the robot faces many complex issues, one of which is that of finding solutions to the correspondence problem. Evolutionary predecessors to observational imitation may have been self-imitation where an agent avoids the complexities of the correspondence problem by learning and replicating actions it has experienced through the manipulation of its body. We investigate how a robotic control and teaching system using self-imitation can be constructed with reference to psychological models of motor control and ideas from social scaffolding seen in animals. Within these scaffolded environments sets of competencies can be built by constructing hierarchical state/action memory maps of the robot's interaction within that environment. The scaffolding process provides a mechanism to enable learning to be scaled up. The resulting system allows a human trainer to teach a robot new skills and modify skills that the robot may possess. Additionally the system allows the robot to notify the trainer when it is being taught skills it already has in its repertoire and to direct and focus its attention and sensor resources to relevant parts of the skill being executed. We argue that these mechanisms may be a first step towards the transformation from self-imitation to observational imitation. The system is validated on a physical pioneer robot that is taught using self-imitation to track, follow and point to a patterned object.

Self-organization of connection patterns within brain areas of animals begins prenatally, and has been shown to depend on internally generated patterns of neural activity. The neural structures continue to develop postnatally through externally driven patterns, when the sensory systems are exposed to stimuli from the environment. The internally generated patterns have been proposed to give the neural system an appropriate bias so that it can learn reliably from complex environmental stimuli. This paper evaluates the hypothesis that complex artificial learning systems can benefit from a similar approach, consisting of initial training with patterns from an evolved pattern generator, followed by training with the actual training set. To test this hypothesis, competitive learning networks were trained for recognizing handwritten digits. The results demonstrate how the approach can improve learning performance by discovering the appropriate initial weight biases, thereby compensating for weaknesses of the learning algorithm. Due to the smaller evolutionary search space, this approach was also found to require much fewer generations than direct evolution of network weights. Since discovering the right biases efficiently is critical for solving large-scale problems with learning, these results suggest that internal training pattern generation is an effective method for constructing complex systems.

Self-organization of connection patterns within brain areas of animals begins prenatally, and has been shown to depend on internally generated patterns of neural activity. The neural structures continue to develop postnatally through externally driven patterns, when the sensory systems are exposed to stimuli from the environment. The internally generated patterns have been proposed to give the neural system an appropriate bias so that it can learn reliably from complex environmental stimuli. This paper evaluates the hypothesis that complex artificial learning systems can benefit from a similar approach, consisting of initial training with patterns from an evolved pattern generator, followed by training with the actual training set. To test this hypothesis, competitive learning networks were trained for recognizing handwritten digits. The results demonstrate how the approach can improve learning performance by discovering the appropriate initial weight biases, thereby compensating for weaknesses of the learning algorithm. Due to the smaller evolutionary search space, this approach was also found to require much fewer generations than direct evolution of network weights. Since discovering the right biases efficiently is critical for solving large-scale problems with learning, these results suggest that internal training pattern generation is an effective method for constructing complex systems.

To be autonomous, intelligent robots must learn the foundations of commonsense knowledge from their own sensorimotor experience in the world. We describe four recent research results that contribute to a theory of how a robot learning agent can bootstrap from the blooming buzzing confusion of the pixel level to a higher-level ontology including distinctive states, places, objects, and actions. This is not a single learning problem, but a lattice of related learning tasks, each providing prerequisites for tasks to come later. Starting with completely uninterpreted sense and motor vectors, as well as an unknown environment, we show how a learning agent can separate the sense vector into modalities, learn the structure of individual modalities, learn natural primitives for the motor system, identify reliable relations between primitive actions and created sensory features, and can define useful control laws for homing and path-following. Building on this framework, we show how an agent can use to self-organizing maps to identify useful sensory featurs in the environment, and can learn effective hill-climbing control laws to define distinctive states in terms of thos features, and trajectoryfollowing control laws to move from one distinctive state to another. Moving on to place recognition, we show how an agent can combine unsupervised learning, map-learning, and supervised learning to achieve high-performance recognition of places from rich sensory input. And finally, we take the first steps toward learning an ontology of objects, showing tha a bootstrap learning robot can learn to individuate objects through motion, separating them from the static environment and from each other, and learning properties that will be useful for classification. These are four key steps in a much larger research enterprise that lays the foundation for human and robot commonsense knowledge.

Primary visual cortex (V1) contains overlaid feature maps for orientation (OR), motion direction selectivity (DR), and ocular dominance (OD). Neurons in these maps are connected laterally in patchy, long-range patterns that follow the feature preferences. Using the LISSOM model, we show for the first time how realistic laterally connected joint OR/OD/DR maps can self-organize from Hebbian learning of moving natural images. The model predicts that lateral connections will link neurons of either eye preference and with similar DR and OR preferences. These results suggest that a single self-organizing system may underlie the development of spatiotemporal feature preferences and lateral connectivity.

When interacting with an object, the possible choices of grasp and manipulation operations are often limited by pick and place constraints. Traditional planning methods are analytical in nature and require geometric models of parts,

The authors introduce an ontology for the study of how animals think, as well as a comprehensive model of human and animal cognition utilizing the ontology. The IDA (Intelligent Distribution Agent) model of cognition, a computational and conceptual model derived from a working software agent, is described within the framework of the ontology. The model is built on functional needs of animals, relating it to the existing literature. The article provides testable hypotheses and a sample a model of decision-making processes in voles. The article closes with a brief comparison of the IDA model to other computational models of cognition, and a discussion of the strengths and weaknesses of the ontology and the model.

This paper reports about our investigation on action understanding in the brain. We review recent results of the neurophysiology of the mirror system in the monkey. Based on these observations we propose a model of the brain systems responsible for action recognition, in which the link between object affordances and action understanding is explicitly considered. To support our hypothesis we describe two experiments where some aspects of the model have been implemented. In the first experiment an action recognition system is trained by using data recorded from human movements which include kinesthetic, tactile, and visual information. In the second experiment, the model is partially implemented on a humanoid robot which learns to mimic simple actions performed by a human subject on different objects. These experiments show that motor information can have a significant role in interpretation of actions and that a mirror-like representation can be developed autonomously as a result of the interaction between an individual and the environment.

A major challenge in reinforcement learning research is to extend methods that have worked well on discrete, short-range, low-dimensional problems to continuous, high-diameter, high-dimensional problems, such as robot navigation using high-resolution sensors. Self-Organizing Distinctive-state Abstraction (SODA) is a new, generic method by which a robot in a continuous world can better learn to navigate by learning a set of high-level features and building temporally-extended actions to carry it between distinctive states based on those features. A SODA agent first uses a self-organizing feature map to develop a set of high-level perceptual features while exploring the environment with primitive, local actions. The agent then builds a set of high-level actions composed of generic trajectory-following and hill-climbing control laws that carry it between the states at local maxima of feature activations. In an experiment on a simulated robot navigation task, the SODA agent learns to perform a task requiring 300 small-scale, local actions using as few as 9 new, temporally-extended actions, significantly improving learning time over navigating with the local actions.

We propose a computational model of the emergence of gaze following skills in infant–caregiver interactions. The model is based on the idea that infants learn that monitoring their caregiver's direction of gaze allows them to predict the locations of interesting objects or events in their environment (Moore & Corkum, 1994). Elaborating on this theory, we demonstrate that a specific Basic Set of structures and mechanisms is sufficient for gaze following to emerge. This Basic Set includes the infant's perceptual skills and preferences, habituation and reward-driven learning, and a structured social environment featuring a caregiver who tends to look at things the infant will find interesting. We review evidence that all elements of the Basic Set are established well before the relevant gaze following skills emerge. We evaluate the model in a series of simulations and show that it can account for typical development. We also demonstrate that plausible alterations of model parameters, motivated by findings on two different developmental disorders – autism and Williams syndrome – produce delays or deficits in the emergence of gaze following. The model makes a number of testable predictions. In addition, it opens a new perspective for theorizing about cross-species differences in gaze following.

Artificial neural networks can model cortical local learning and signal processing, but they are not the brain, neither are many spe- cial purpose systems to which they contribute. Autonomous mental development models all or part of the brain (or the central nervous system) and how it develops and learns autonomously from infancy to adulthood. Like neural network research, such modeling aims to be biologically plausible. This paper discusses why autonomous develop- ment is necessary according to a concept called task muddiness. Then it introduces recent results for a series of research issues, including the new paradigm for autonomous development, mental architectures, developmental algorithm, a refined classification of types of machine learning, spatial complexity and time complexity. Finally, the paper presents some experimental results for applications, including: vision- guided navigation, object finding, object-based attention (eye-pan), and attention-guided pre-reaching, four tasks that infants learn to per- form early but very perceptually challenging for robots.

This paper reports about our investigation on action understanding in the brain. We review recent results of the neurophysiology of the mirror system in the monkey. Based on these observations we propose a model of the brain systems responsible for action recognition, in which the link between object affordances and action understanding is explicitly considered. To support our hypothesis we describe two experiments where some aspects of the model have been implemented. In the first experiment an action recognition system is trained by using data recorded from human movements which include kinesthetic, tactile, and visual information. In the second experiment, the model is partially implemented on a humanoid robot which learns to mimic simple actions performed by a human subject on different objects. These experiments show that motor information can have a significant role in interpretation of actions and that a mirror-like representation can be developed autonomously as a result of the interaction between an individual and the environment.

This article describes a developmental system based on information theory implemented on a real robot that learns a model of its own sensory and actuator apparatus. There is no innate knowledge regarding the modalities or representation of the sensory input and the actuators, and the system relies on generic properties of the robots world, such as piecewise smooth effects of movement on sensory changes. The robot develops the model of its sensorimotor system by first performing random movements to create an informational map of the sensors. Using this map, the robot then learns what effects the different possible actions have on the sensors. After this developmental process, the robot can perform basic visually guided movement.

What kind of motivation drives child language development? This article presents a computational model and a robotic experiment to articulate the hypothesis that children discover communication as a result of exploring and playing with their environment. The considered robotic agent is intrinsically motivated towards situations in which it optimally progresses in learning. To experience optimal learning progress, it must avoid situations already familiar but also situations where nothing can be learned. The robot is placed in an environment in which both communicating and non-communicating objects are present.As a consequence of its intrinsic motivation, the robot explores this environment in an organized manner focussing first on non-communicative activities and then discovering the learning potential of certain types of interactive behavior. In this experiment, the agent ends up being interested by communication through vocal interactions without having a specific drive for communication.

A major challenge in reinforcement learning research is to extend methods that have worked well on discrete, short-range, low-dimensional problems to continuous, high-diameter, high-dimensional problems, such as robot navigation using high-resolution sensors. Self-Organizing Distinctive-state Abstraction (SODA) is a new, generic method by which a robot in a continuous world can better learn to navigate by learning a set of high-level features and building temporally-extended actions to carry it between distinctive states based on those features. A SODA agent first uses a self-organizing feature map to develop a set of high-level perceptual features while exploring the environment with primitive, local actions. The agent then builds a set of high-level actions composed of generic trajectory-following and hill-climbing control laws that carry it between the states at local maxima of feature activations. In an experiment on a simulated robot navigation task, the SODA agent learns to perform a task requiring 300 small-scale, local actions using as few as 9 new, temporally-extended actions, significantly improving learning time over navigating with the local actions.

This paper overviews the interactivist model of representation and its applications in artificial intelligence (AI) and intelligent robotics. Selected examples from approaches in AI and robotics are contrasted with the generic interactivist architecture in order to illustrate specific features of it. Petitagé, an artificial agent that instantiates our interactivist-expectative theory of agency and learning (IETAL), is discussed in detail from the interactivist perspective.

Studies on lexicon acquisition systems are gaining attention in the search for a natural human-robot interface and a test environment to model the infant lexicon acquisition process. Although various lexicon acquisition systems that ground words to sensory experience have been developed, existing systems have clear limitations on the ability to autonomously associate words to objects. This limitation is due to the fact that categories for words are formed in a passive manner, either by teaching of caregivers or finding similarities in visual features. This paper presents a system for lexicon acquisition through behavior learning. Based on a modified multi-module reinforcement learning system, the robot is able to automatically associate words to objects with various visual features based on similarities in affordances or in functions. The system was implemented on a mobile robot acquiring a lexicon related to different rolling preferences. The experimental results are given and future issues are discussed.

Visual areas in primates are known to have reciprocal connections. While the feedforward bottom-up processing of visual information has been studied extensively for decades, little is known about the role of the feedback connections. Existing feedback models usually employ hand-coded connections, and do not address how these connections develop. The model described in this paper shows how feedforward and feedback connections between cortical areas V1 and V2 can be learned through self-organization simultaneously. Computational experiments show that both areas can form hierarchical representations of the input with reciprocal connections that link relevant cells in the two areas.

Three experiments tested whether geometric biases—biases away from perceived reference axes—reported in spatial recall tasks with pointing responses generalize to a recognition task that requires a verbal response. Seven-year-olds and adults remembered the location of a dot within a rectangle, and then either reproduced its location or verbally selected a matching choice dot from a set of colored options. Results demonstrate that geometric biases generalize to verbal responses; however, the spatial span of the choice set influenced performance as well. These data suggest that the same spatial memory process gives rise to both response types in this task. Simulations of a dynamic field model buttress this claim. More generally, these results challenge accounts that posit separate spatial systems for motor and verbal responses.

This paper overviews the interactivist model of representation and its applications in artificial intelligence (AI) and intelligent robotics. Selected examples from approaches in AI and robotics are contrasted with the generic interactivist architecture in order to illustrate specific features of it. Petitagé, an artificial agent that instantiates our interactivist-expectative theory of agency and learning (IETAL), is discussed in detail from the interactivist perspective.

This work aimed to unify the two famous hippocampal functions, spatial and memory function, based on their suggested common mechanism. The goal is achieved with a simple connectionist model that is used to give an account of hippocampal spatial navigation in rodents and episodic memory retrieval in humans. The model is essentially based on well-established experimental knowledge, including the numbers characterizing hippocampal connectivity, correlates of hippocampal place cell activity, e.g., the phase precession (PP) phenomenon, and properties of the sharp waves: see the color version of Figure 1 and its caption at the end of this document. The central part of the model (Figure 2) is the hippocampus, represented as a two-layer network of CA3 and CA1 place cells. Place cell firing during the theta mode represents the current location of the rat. During maze running, the probability of firing of a given place cell in a sharp wave is proportional to the recency of the last visit to its preferred location. This assumption results in specific learning rules (Equation 1). Next, the model assumes spontaneous, random alternations of the PP direction from one theta cycle to another. This means that during goal search, the rat is “mentally exploring” available directions of motion (or, more generally, available actions) at each step, using PP. Then it selects the direction producing the strongest modulation of the background activity of the goal-related CA1 place cell. This assumption results in a specific algorithm describing model dynamics. All the foregoing dynamical rules were implemented in several pieces of Matlab 6 code, each used to simulate a specific paradigm. Results are represented in various digital formats.

Long-range lateral connections in the primary visual cortex (V1) are known to link neurons with similar orientation preferences, but it is not yet known how color-selective cells are connected. Using a self-organizing model of V1 with natural color image input, we show that realistic color-selective receptive fields, color maps, and orientation maps develop. Connections between orientation-selective cells match previous experimental results, and the model predicts that color-selective cells will primarily connect to other cells with similar chromatic preferences. These findings suggest that a single self-organizing system may underlie the development of orientation selectivity, color selectivity, and lateral connectivity.

A computational model with three interacting components for context sensitive reinforcement learning, context processing and automation can autonomously learn a focus attention and a shift attention task. The performance of the model is similar to that of normal children, and when a single parameter is changed, the performance on the two tasks approaches that of autistic children.

We have shown in a computational model that a poor memory for context could result in some of the behaviors associated with ADHD, which is well in line with the dynamic developmental theory. Given the important role of context in extinction, a weaker context due to a steeper delay-of reinforcement gradient would result in impaired inhibition.

Cognitive explanations of autism often involve higher order cognitive functions developing late in childhood, such as theory of mind, executive functions or central coherence. In home videos of infants later diagnosed with autism, children display early signs of developmental disorders, for example impaired sensorimotor functions, attention to social and non-social stimuli and a lack of circadian regulation. We propose that these early signs need to be understood using a framework of context learning. It is also important to understand the role for context understanding in guiding the maturation of behavior. The role for inhibition in context learning as understood within learning theory provides us with helpful tools for this analysis. Our research aim is not to identify and explain early markers for autism, but to understand the cognitive developmental pathway set into rolling by an early impairment. This will help us understand the seemingly unrelated symptoms that define the complex syndrome of autism.

Conscious events interact with memory systems in learning, rehearsal and retrieval (Ebbinghaus 1885/1964; Tulving 1985). Here we present hypotheses that arise from the IDA computional model (Franklin, Kelemen and McCauley 1998; Franklin 2001b) of global workspace theory (Baars 1988, 2002). Our primary tool for this exploration is a flexible cognitive cycle employed by the IDA computational model and hypothesized to be a basic element of human cognitive processing. Since cognitive cycles are hypothesized to occur five to ten times a second and include interaction between conscious contents and several of the memory systems, they provide the means for an exceptionally fine-grained analysis of various cognitive tasks. We apply this tool to the small effect size of subliminal learning compared to supraliminal learning, to process dissociation, to implicit learning, to recognition vs. recall, and to the availability heuristic in recall. The IDA model elucidates the role of consciousness in the updating of perceptual memory, transient episodic memory, and procedural memory. In most cases, memory is hypothesized to interact with conscious events for its normal functioning. The methodology of the paper is unusual in that the hypotheses and explanations presented are derived from an empirically based, but broad and qualitative computational model of human cognition.

The speech code is a vehicle of language: it defines a set of forms used by a community to carry information. Such a code is necessary to support the linguistic interactions that allow humans to communicate. How then may a speech code be formed prior to the existence of linguistic interactions? Moreover, the human speech code is discrete and compositional, shared by all the individuals of a community but different across communities, and phoneme inventories are characterized by statistical regularities. How can a speech code with these properties form? We try to approach these questions in the paper, using the ‘‘methodology of the artificial’’. We build a society of artificial agents, and detail a mechanism that shows the formation of a discrete speech code without presupposing the existence of linguistic capacities or of coordinated interactions. The mechanism is based on a low-level model of sensory–motor interactions. We show that the integration of certain very simple and non-language-specific neural devices leads to the formation of a speech code that has properties similar to the human speech code. This result relies on the self-organizing properties of a generic coupling between perception and production within agents, and on the interactions between agents. The artificial system helps us to develop better intuitions on how speech might have appeared, by showing how self-organization might have helped natural selection to find speech. r 2004 Elsevier Ltd. All rights reserved.

Computational models have played a central role in the debate over language learnability. This article discusses how they have been used in different “stances”, from generative views to more recently introduced explanatory frameworks based on embodiment, cognitive development and cultural evolution. By digging into the details of certain specific models, we show how they organize, transform and rephrase defining questions about what makes language learning possible for children. Finally, we present a tentative synthesis to recast the debate using the notion of learning bias.

This paper shows how phonological structures can be culturally selected so as to become learnable and adapted to the ecological niche formed by the brains and bodies of speakers. A computational model of the cultural formation of syllable systems illustrates how general learning and physical biases can influence the evolution of the structure of vocalization systems. We use the artificial life methodology of building a society of artificial agents, equipped with motor, perceptual and cognitive systems that are generic and have a realistic complexity. We demonstrate that agents, playing the “imitation game,” build shared syllable systems and show how these syllable systems relate to existing human syllable systems. Detailed experiments study the learnability of the self-organized syllable systems. In particular, we reproduce the critical period effect and the artificial language learning effect without the need for innate biases which specify explicitly in advance the form of possible phonological structures. The ability of children agents to learn syllable systems is explained by the cultural evolutionary history of these syllable systems, which were selected for learnability.

Collerton et al. propose that one and the same mechanism (PAD) underlies recurrent complex visual hallucinations (RCVH) in various disorders, including schizophrenia, dementia, and eye disease. The present commentary offers an alternative account of RCVH and other recurrent complex hallucinations specific to schizophrenia and related disorders only. The proposed account is consistent with the bias of schizophrenic RCVH contents toward animate, socially active entities.

The goal of this work is to extend the theoretical understanding of the relationship between hippocampal spatial and memory functions to the level of neurophysiological mechanisms underlying spatial navigation and episodic memory retrieval. The proposed unifying theory describes both phenomena within a unique framework, as based on one and the same pathfinding function of the hippocampus. We propose a mechanism of reconstruction of the context of experience involving a search for a nearly shortest path in the space of remembered contexts. To analyze this concept in detail, we define a simple connectionist model consistent with available rodent and human neurophysiological data. Numerical study of the model begins with the spatial domain as a simple analogy for more complex phenomena. It is demonstrated how a nearly shortest path is quickly found in a familiar environment. We prove numerically that associative learning during sharp waves can account for the necessary properties of hippocampal place cells. Computational study of the model is extended to other cognitive paradigms, with the main focus on episodic memory retrieval. We show that the ability to find a correct path may be vital for successful retrieval. The model robustly exhibits the pathfinding capacity within a wide range of several factors, including its memory load (up to 30,000 abstract contexts), the number of episodes that become associated with potential target contexts, and the level of dynamical noise. We offer several testable critical predictions in both spatial and memory domains to validate the theory. Our results suggest that (1) the pathfinding function of the hippocampus, in addition to its associative and memory indexing functions, may be vital for retrieval of certain episodic memories, and (2) the hippocampal spatial navigation function could be a precursor of its memory function.

We start by assuming that the self is implemented in the brain as a functional unit, with a definite set of properties. We deduce the fundamental properties of the self from an analysis of neurological disorders and from introspection. We formulate a functionalist concept of the self based on these properties reduced to constraints. We use the formalism of schemas in our functionalist analysis, i.e. – a symbolic level description of brain dynamics. We then reformulate the functionalist model at a connectionist level and address the emergent "context shifting" problem. We suggest how the model might be mapped onto the functional neuroanatomy of the brain, and how it could be used to give an account of a range of neurological disorders, including hippocampal amnesia, various forms of schizophrenia, multiple personality, autism, PTSD, hemineglect, and reversible anosognosia. Finally, we briefly discuss future perspectives and possible applications of computer implementations of the model.

In most modern video games, character behavior is scripted; no matter how many times the player exploits a weakness, that weakness is never repaired. Yet, if game characters could learn through interacting with the player, behavior could improve as the game is played, keeping it interesting. This paper introduces the real-time Neuroevolution of Augmenting Topologies (rtNEAT) method for evolving increasingly complex artificial neural networks in real time, as a game is being played. The rtNEAT method allows agents to change and improve during the game. In fact, rtNEAT makes possible an entirely new genre of video games in which the player trains a team of agents through a series of customized exercises. To demonstrate this concept, the Neuroevolving Robotic Operatives (NERO) game was built based on rtNEAT. In NERO, the player trains a team of virtual robots for combat against other players' teams. This paper describes results from this novel application of machine learning, and demonstrates that rtNEAT makes possible video games like NERO where agents evolve and adapt in real time. In the future, rtNEAT may allow new kinds of educational and training applications through interactive and adapting games.

An approach bringing together space-variant vision through a simple color segmentation technique and learning is presented. The proposed approach is employed to control the movement of a 5 degree of freedom (d.o.f.) robotic head. Color information is used to determine the position of the object of interest in the image plane and, consequently, to track it during its motion. The distance of the target from the center of the image is used to feed both a closed-loop and an open-loop controller. Most important, the parameters of the controllers are learnt on-line in a self-supervised fashion. Experiments are presented to demonstrate empirically the feasibility of the approach and its application to a real world control problem.

Contour integration in low-level vision is believed to occur based on lateral interaction between neurons with similar orientation tuning. How such interactions could arise in the brain has been an open question. Our model suggests that the interactions can be learned through input-driven self-organization, i.e. through the same mechanism that underlies many other developmental and functional phenomena in the visual cortex. The model also shows how synchronized firing mediated by these lateral connections can represent the percept of a contour, resulting in performance similar to that of human contour integration. The model further demonstrates that contour integration performance can differ in different parts of the visual field, depending on what kinds of input distributions they receive during development. The model thus grounds an important perceptual phenomenon onto detailed neural mechanisms, so that various structural and functional properties can be measured, and predictions can be made to guide future experiments.

Fifty families participated in mother- and father-infant still-face interaction at infant ages 3 and 6 months as part of a study of affect in early parent-infant relationships. Infants' positive and negative affect and parents' positive affect and physical play were coded from videotapes. Consistent with previous research, during the normal condition, mothers displayed more positive affect than did fathers, and fathers were more likely than mothers to display physical play. Infants were more positive with mothers than with fathers. Parents’ positive affect but not parent gender predicted infants’ positive affect at 6 months. During the still-face condition, infants of parents with a lifetime history of depression were more likely to display negative affect and less likely to display positive affect than infants with no such parent history. Infants' affect was unrelated to parents' current level of depressive symptoms, which indicates the value of considering family history of psychopathology when examining individual differences in infants' affect.

An approach bringing together space-variant vision through a simple color segmentation technique and learning is presented. The proposed approach is employed to control the movement of a 5 degree of freedom (d.o.f.) robotic head. Color information is used to determine the position of the object of interest in the image plane and, consequently, to track it during its motion. The distance of the target from the center of the image is used to feed both a closed-loop and an open-loop controller. Most important, the parameters of the controllers are learnt on-line in a self-supervised fashion. Experiments are presented to demonstrate empirically the feasibility of the approach and its application to a real world control problem.

This paper presents autoassociator neural networks. A first section reviews the architecture of these models, common learning rules, and presents sample simulations to illustrate their abilities. In a second section, the ability of these models to account for learning phenomena such as habituation is reviewed. The contribution of these networks to discussions about infant cognition is highlighted. A new, modular approach is presented in a third section. In the discussion, a role for these learning models in a broader developmental framework is proposed.

Habituation and related procedures are the primary behavioral tools used to assess perceptual and cognitive competence in early infancy. This article introduces a neurally constrained computational model of infant habituation. The model combines the two leading process theories of infant habituation into a single functional system that is grounded in functional brain circuitry. The HAB model (for Habituation, Autoassociation, and Brain) proposes that habituation behaviors emerge from the opponent, complementary processes of hippocampal selective inhibition and cortical long-term potentiation. Simulations of a seminal experiment by Fantz [Visual experience in infants: Decreased attention familiar patterns relative to novel ones. Science, 146, 668–670, 1964] are reported. The ability of the model to capture the fine detail of infant data (especially age-related changes in performance) underlines the useful contribution of neurocomputational models to our understanding of behavior in general, and of early cognition in particular.

wo major goals in machine learning are the discovery and improvement of solutions to complex problems. In this paper, we argue that complexification, i.e. the incremental elaboration of solutions through adding new structure, achieves both these goals. We demonstrate the power of complexification through the NeuroEvolution of Augmenting Topologies (NEAT) method, which evolves increasingly complex neural network architectures. NEAT is applied to an open-ended coevolutionary robot duel domain where robot controllers compete head to head. Because the robot duel domain supports a wide range of strategies, and because coevolution benefits from an escalating arms race, it serves as a suitable testbed for studying complexification. When compared to the evolution of networks with fixed structure, complexifying evolution discovers significantly more sophisticated strategies. The results suggest that in order to discover and improve complex solutions, evolution, and search in general, should be allowed to complexify as well as optimize.

Autonomous Mental Development (AMD) of robots opened a new paradigm for developing machine intelligence, using neural network type of techniques and it fundamentally changed the way an intelligent machine is developed from manual to autonomous. The work presented here is a part of SAIL (Self-Organizing Autonomous Incremental Learner) project which deals with autonomous development of humanoid robot with vision, audition, manipulation and locomotion. The major issue addressed here is the challenge of high dimensional action space (5– 10) in addition to the high dimensional context space (hundreds to thousands and beyond), typically required by an AMD machine. This is the first work that studies a high dimensional (numeric) action space in conjunction with a high dimensional perception (context state) space, under the AMD mode. Two new learning algorithms, Direct Update on Direction Cosines (DUDC) and High- Dimensional Conjugate Gradient Search (HCGS), are developed, implemented and tested. The convergence properties of both the algorithms and their targeted applications are discussed. Autonomous learning of speech production under reinforcement learning is studied as an example.

Studies of orientation maps in primary visual cortex (V1) suggest that lateral connections mediate competition and cooperation between orientation-selective units, but their role in motion perception has not been established. Using a self-organizing model of V1 with moving oriented patterns, we show that (1) afferent weights of each neuron organize into Gabor-like spatiotemporal receptive fields with ON and OFF lobes, (2) these receptive fields form realistic joint direction and orientation maps, and (3) lateral connections develop between patches with similar orientation and direction preferences. These results suggest that a single self-organizing system may underlie the development of orientation selectivity, direction selectivity, and lateral connectivity.

Newborn humans preferentially orient to face-like patterns at birth, but months of experience with faces is required for full face processing abilities to develop. Several models have been proposed for how the interaction of genetic and evironmental influences can explain this data. These models generally assume that the brain areas responsible for newborn orienting responses are not capable of learning and are physically separate from those that later learn from real faces. However, it has been difficult to reconcile these models with recent discoveries of face learning in newborns and young infants. We propose a general mechanism by which genetically specified and environment-driven preferences can coexist in the same visual areas. In particular, newborn face orienting may be the result of prenatal exposure of a learning system to internally generated input patterns, such as those found in PGO waves during REM sleep. Simulating this process with the HLISSOM biological model of the visual system, we demonstrate that the combination of learning and internal patterns is an efficient way to specify and develop circuitry for face perception. This prenatal learning can account for the newborn preferences for schematic and photographic images of faces, providing a computational explanation for how genetic influences interact with experience to construct a complex adaptive system.

Experimentation is crucial to human progress at all scales, from society as a whole to a young infant in its cradle. It allows us to elicit learning episodes suited to our own needs and limitations. This paper develops active strategies for a robot to acquire visual experience through simple experimental manipulation. The experiments are oriented towards determining what parts of the environment are physically coherent|that is, which parts will move together, and which are more or less independent. We argue that following causal chains of events out from the robot’s body into the environment allows for a very natural developmental progression of visual competence, and relate this idea to results in neuroscience.

Vision and manipulation are inextricably intertwined in the primate brain. Tantalizing results from neuroscience are shedding light on the mixed motor and sensory representations used by the brain during reaching, grasping, and object recognition. We now know a great deal about what happens in the brain during these activities, but not necessarily why. Is the integration we see functionally important, or just a reflection of evolution’s lack of enthusiasm for sharp modularity? We wish to instantiate these results in robotic form to probe the technical advantages and to find any lacunae in existing models. We believe it would be missing the point to investigate this on a platform where dextrous manipulation and sophisticated machine vision are already implemented in their mature form, and instead follow a developmental approach from simpler primitives. We begin with a precursor to manipulation, simple poking and prodding, and show how it facilitates object segmentation, a long-standing problem in machine vision. The robot can familiarize itself with the objects in its environment by acting upon them. It can then recognize other actors (such as humans) in the environment through their effect on the objects it has learned about. We argue that following causal chains of events out from the robot's body into the environment allows for a very natural developmental progression of visual competence, and we relate this idea to results in neuroscience.

Appearance-based image analysis techniques require fast computation of principal components of high-dimensional image vectors. We introduce a fast incremental principal component analysis (IPCA) algorithm, called candid covariance-free IPCA (CCIPCA), used to compute the principal components of a sequence of samples incrementally without estimating the covariance matrix (so covariance-free). The new method is motivated by the concept of statistical efficiency (the estimate has the smallest variance given the observed data). To do this, it keeps the scale of observations and computes the mean of observations incrementally, which is an efficient estimate for some wellknown distributions (e.g., Gaussian), although the highest possible efficiency is not guaranteed in our case because of unknown sample distribution. The method is for real-time applications and, thus, it does not allow iterations. It converges very fast for high-dimensional image vectors. Some links between IPCA and the development of the cerebral cortex are also discussed.

Developmental robotics is an emerging field located at the intersection of robotics, cognitive science and developmental sciences. This paper elucidates the main reasons and key motivations behind the convergence of fields with seemingly disparate interests, and shows why developmental robotics might prove to be beneficial for all fields involved. The methodology advocated is synthetic and two-pronged: on the one hand, it employs robots to instantiate models originating from developmental sciences; on the other hand, it aims to develop better robotic systems by exploiting insights gained from studies on ontogenetic development. This paper gives a survey of the relevant research issues and points to some future research directions.

The dynamic field theory predicts that biases toward remembered locations depend on the separation between targets, and the spatial precision of interactions in working memory which become enhanced over development. This was tested by varying the separation between “A” and “B” locations in a sandbox. Children searched for an object six times at an A location, followed by three trials to B. Two and 4-year-olds’, but not 6-year-olds’, responses were biased toward A when A and B were 9” and 6” apart. When A and B were separated by 2”, however, 4- and 6-year-olds’ responses were biased toward A. Thus, the separation at which responses were biased toward A decreased across age groups, supporting the predictions of the theory.

This paper presents a new technique which incrementally builds a hierarchical discriminant regression (IHDR) tree for generation of motion based robot reactions. The robot learned the desired reactions from motion change images, without using other pre-defined features. The generation from training cases is accomplished through the automatically constructed IHDR tree, which automatically derives features that are most related to outputs and disregards subspaces that are not related, or less related, to outputs. The real-time speed is achieved through combination of feature space partition and a coarse-to-fine sample search, which results in a logarithmic time complexity in the number of nodes. The experiments showed that the IHDR method can interpolate the mapping between high dimensional input space and the output control signal space from a variety of objects of various shapes with different motion patterns, based on the sizedependent negative logarithmic distance measures in the hierarchical feature space. The trained robot is able to aim to its camera towards moving object and move toward or away according to the size of moving object.
2002 Elsevier Science B.V. All rights reserved.

This work addresses the problem of learning stabilization re exes in robots with moving eyes. Most essential in achieving efficient visual stabilization is the exploitation and integration of different motion related sensory information. In our robot, self-motion is measured inertially with an artificial vestibular system and visually using optic ow algorithms. The first sensory system provides short latency measurements of rotations and translations of the robot’s head, the second, a delayed estimate of the motion across the image plane. A self-tuning neural network learns to combine these two measurements and generates oculo-motor compensatory behaviors that stabilize the visual scene. We describe the network architecture and the learning scheme. The stabilization performance is evaluated quantitatively using direct measurements on the image plane.

In this paper we want to propose the idea that some techniques used for animal training might be helpful for solving human robot interaction problems in the context of entertainment robotics. We present a model for teaching complex actions to an animal-like autonomous robot based on ”clicker training”, a method used efficiently by professional trainers for animals of different species. After describing our implementation of clicker training on an enhanced version of AIBO, Sony’s four-legged robot, we argue that this new method can be a promising technique for teaching unusual behavior and sequences of actions to a pet robot.

The goal of this paper is to propose a biologically plausible, functional model of the acquisition of visual, acoustic and multi-modal motor responses. Within this context visual and acoustic spatial cues are considered, fused in a coherent percept and eventually employed to control the orienting behavior of a humanoid robot. The rationale of the approach lies in the possibility to test and empirically prove the correctness of the model through the embodiment and the real interaction of the system with the environment. The model takes into account the fact that: i) acoustic and visual cues are represented with respect to different coordinate frames (head- vs. retino-centric) and consequently they need to be “aligned” in order to be properly fused; ii) a teaching signal has to be generated in order to inform the system that the motor performance is not adequate to perform the task (i.e. orient toward the stimulus) and thus adaptation is required; iii) vision plays a major role in driving the acquisition of the appropriate map of space but other sources of feedback might be employed as well.

Research on infant cognition using habituation methods has sparked considerable controversy in recent years. At the core of the debates is the issue of whether infants have early (and possibly innate) conceptual understandings. This article reviews a range of computational models of habituation that might provide insights into such discussions. The models are assessed against key behavioral and neural features of habituation: temporal unfolding, exponential decrease, familiarity-to-novelty shift, habituation to repeated testing, discriminability of habitual items, selective inhibition and cortical–subcortical interactions. The review suggests that current models fail to offer comprehensive explanations of the behavioral phenomena.

Imitation is a powerful mechanism for efficient learning of novel behaviors that both supports and takes advantage of sociality. A fundamental problem for imitation is to create an appropriate (partial) mapping between the body of the system being imitated and the imitator. By considering for each of these two systems an associated automaton (respectively, transformation semigroup) structure, attempts at such mapping can be considered (partial) relational homomorphisms. This article shows how mathematical techniques can be applied to characterize how far a behavior is from a successful imitation and how to evaluate attempts at imitation arising from a particular correspondence between the imitator and model. For the imitator and the imitated, affordances in the agent-environment structural coupling are likely to be different, all the more so in the case of dissimilar embodiment. We argue that the use of what is afforded to the imitator to attain corresponding effects or, as in dance, sequences of effects, is necessary and sufficient for successful imitation. However, the judged degree of success or failure of an attempted behavioral match depends on some externally imposed or in the case ofautonomous agents internally determined criteria on effects of the attempted imitative behavior (including effects attained successively as well as final effects). These criteria correspond to metrics measures of difference which can guide the evaluation of a correspondence, the learning of a correspondence, or learning how to apply one. Metrics on states and sequences of action events in the system-environment coupling allow judgment of similarity for observer-dependent' purposes. This allows one to formally define successful imitation with respect to such criteria. The resulting measures can be used to compare various candidate mappings (e.g., body plan or perception-action correspondences). Additionally, this may be applied in the automated construction and learning of mappings to be used in imitation for artificial, hardware, and software systems.

How should biological behaviour be modelled? A relatively new approach is to investigate problems in neuroethology by building physical robot models of biological sensorimotor systems. The explication and justification of this approach are here placed within a framework for describing and comparing models in the behavioural and biological sciences. First, simulation models – the representation of a hypothesis about a target system – are distinguished from several other relationships also termed “modelling” in discussions of scientific explanation. Seven dimensions on which simulation models can differ are defined and distinctions between them discussed: 1. Relevance: whether the model tests and generates hypotheses applicable to biology. 2. Level: the elemental units of the model in the hierarchy from atoms to societies. 3. Generality: the range of biological systems the model can represent. 4. Abstraction: the complexity, relative to the target, or amount of detail included in the model. 5. Structural accuracy: how well the model represents the actual mechanisms underlying the behaviour. 6. Performance match: to what extent the model behaviour matches the target behaviour. 7. Medium: the physical basis by which the model is implemented. No specific position in the space of models thus defined is the only correct one, but a good modelling methodology should be explicit about its position and the justification for that position. It is argued that in building robot models biological relevance is more effective than loose biological inspiration; multiple levels can be integrated; that generality cannot be assumed but might emerge from studying specific instances; abstraction is better done by simplification than idealisation; accuracy can be approached through iterations of complete systems; that the model should be able to match and predict target behaviour; and that a physical medium can have significant advantages. These arguments reflect the view that biological behaviour needs to be studied and modelled in context, that is, in terms of the real problems faced by real animals in real environments.

How does one create an intelligent machine? This problem has proven difficult. Over the past several decades, scientists have taken one of three approaches: In the first, which is knowledge-based, an intelligent machine in a laboratory is directly programmed to perform a given task. In a second, learning-based approach, a computer is "spoon-fed" human-edited sensory data while the machine is controlled by a task-specific learning program. Finally, by a "genetic search," robots have evolved through generations by the principle of survival of the fittest, mostly in a computer-simulated virtual world. Although notable, none of the is powerful enough to lead to machines having the complex, diverse, and highly integrated capabilities of an adult brain, such as vision, speech, and language. Nevertheless, these traditional approaches have served as the incubator for the birth and growth of a new direction for machine intelligence: autonomous mental development. As Kuhn wrote (1), "Failure of existing rules is the prelude to a search for new ones."

A computational model of emergent syllable systems is developed based on a set of functional constraints on syllable systems and the assumption that language structure emerges through cumulative change over time. The constraints were derived from general communicative factors as well as from the phonetic principles of perceptual distinctiveness and articulatory ease. Through evolutionary optimization, the model generated mock vocabularies optimized for the given constraints. Several simulations were run to understand how these constraints might define the emergence of universals and variation in complex sound systems. The predictions were that (1) CV syllables would be highly frequent in all vocabularies evolved under the constraints; (2) syllables with consonant clusters, consonant codas and vowel onsets would occur much less frequently; (3) a relationship would exist between the number of syllable types in a vocabulary and the average word length in the vocabulary; (4) different syllable types would emerge according to, what we termed, an iterative principle of syllable structure and their frequency would be directly related to their complexity; and (5) categorical differences would emerge between vocabularies evolved under the same constraints. Simulation results confirmed these predictions and provided novel insights into why regularities and differences may occur across languages. Specifically, the model suggested that both language universals and variation are consistent with a set of functional constraints that are fixed relative to one another. Language universals reflect underlying constraints on the system and language variation represents the many different and equally-good solutions to the unique problem defined by these constraints.

The Spatial Semantic Hierarchy is a model of knowledge of large-scale space consisting of multiple interacting representations, both qualitative and quantitative. The SSH is inspired by the properties of the human cognitive map, and is intended to serve both as a model of the human cognitive map and as a method for robot exploration and map-building. The multiple levels of the SSH express states of partial knowledge, and thus enable the human or robotic agent to deal robustly with uncertainty during both learning and problem-solving. The control level represents useful patterns of sensorimotor interaction with the world in the form of trajectory-following and hill-climbing control laws leading to locally distinctive states. Local geometric maps in local frames of reference can be constructed at the control level to serve as observers for control laws in particular neighborhoods. The causal level abstracts continuous behavior among distinctive states into a discrete model consisting of states linked by actions. The topological level introduces the external ontology of places, paths and regions by abduction, to explain the observed pattern of states and actions at the causal level. Quantitative knowledge at the control, causal and topological levels supports a ``patchwork map'' of local geometric frames of reference linked by causal and topological connections. The patchwork map can be merged into a single global frame of reference at the metrical level when sufficient information and computational resources are available. We describe the assumptions and guarantees behind the generality of the SSH across environments and sensorimotor systems. Evidence is presented from several partial implementations of the SSH on simulated and physical robots.

Is attention a purely perceptual process or is it in any way related to motor control? The aim of this article is to show that attention puts similar demands on a cognitive system as motor control and present evidence supporting the view that similar mechanisms operate in the two processes. A computational model of attention is presented that uses habituation as well as classical and instrumental conditioning to explain a number of attentional processes. Evi- dence from neurophysiology is reviewed that suggest that attention is con- trolled in a way similar to actions. This view makes it possible to adapt tradi- tional learning theoretical mechanisms to the control of attention. Computer simulations are presented that illustrates the operation of the model.

Stabilization of gaze is a major functional prerequisite for robots exploring the environment. The main reason for a “steady-image” requirement is to prevent the robot’s own motion to compromise its “visual functions”. In this paper we present an artificial system, the LIRA robot head, capable of controlling its cameras/eyes to stabilize gaze. The system features a stabilization mechanism relying on principles exploited by natural systems: an inertial sensory apparatus and images of space-variant resolution. The inertial device measures angular velocities and linear acceleration along the vertical and horizontal fronto-parallel axes. The space-variant image geometry facilitates real-time computation of optic flow and the extraction of first-order motion parameters. Experiments which describe the performance of the LIRA robot head are presented. The results show that the stabilization mechanism improves the reactivity of the system to changes occurring suddenly at new spotted locations. Keywords: Inertial sensors; Image stabilization; Visuo-inertial integration; Space-variant binocular vision

THE STATE-BASED LEARNING METHOD PRESENTED HERE IS APPLICABLE TO VIRTUALLY ANY VISION-BASED CONTROL PROBLEM. THE AUTHORS USE NAVIGATION AS AN EXAMPLE. THE SHOSLIF-N NAVIGATION SYSTEM AUTOMATICALLY DERIVES, DURING TRAINING, THE VISUAL FEATURES THAT ARE BEST SUITED FOR NAVIGATION. USING SYSTEM STATES ENABLES SHOSLIF-N TO DISREGARD UNRELATED SCENE PARTS AND ACHIEVE BETTER GENERALIZATION.

Hippocampal-neocortical interactions in memory have typically been characterized within the ‘‘standard model’’ of memory consolidation. In this view, memory storage initially requires hippocampal linking of dispersed neocortical storage sites, but over time this need dissipates, and the hippocampal component is rendered unnecessary. This change in function over time is held to account for the retorgrade amnesia (RA) gradients often seen in patients with hippocampal damage. Recent evidence, however, calls this standard model into question, and we have recently proposed a new approach, the ‘‘multiple memory trace’’ (MMT) theory. In this view, hippocampal ensembles are always involved in storage and retrieval of episodic information, but semantic (gist) information can be established in neocortex, and will survive damage to the hippocampal system if enough time has elapsed. This approach accounts more readily for the very long RA gradients often observed in amnesia. We report the results of analytic and connectionist simulations that demonstrate the feasibility of MMT. We also report a neuroimaging study showing that retrieval of very remote (25-year-old) memories elicits as much activation in hippocampus as retrieval of quite recent memories. Finally, we report new data from the study of patients with temporal lobe damage, using more sensitive measures than previously the case, showing that deficits in both episodic and spatial detail can bed observed even for very remote memories. Overall, these findings indicate that the standard model of memory consolidation, which views the hippocampus as having only a temporary role in memory, is wrong. Instead, the data support the view that for episodic and spatial detail the hippocampal system is always necessary.

In this paper, we investigate vision-based navigation using the self-organizing hierarchical optimal subspace learning and inference framework (SHOSLIF) that incorporates states and a visual attention mechanism. With states to keep the history information and regarding the incoming video input as an observation vector, the vision-based navigation is formulated as an observation- driven Markov model (ODMM). The ODMM can be realized through recursive partitioning regression. A stochastic recursive partition tree (SRPT), which maps an preprocessed current input raw image and the previous state into the current state and the next control signal, is used for efficient recursive partitioning regression. The SRPT learns incrementally: each learning sample is learned or rejected “on-the-fly.” The purposed scheme has been successfully applied to indoor navigation.

This paper reviews a recent article suggesting that infants use a system of algebraic rules to learn an artificial grammar (Marcus, Vijayan, Bandi Rao & Vishton, Rule learning by seven?month?old infants. Science, 183 (1999), 77–80). In three reported experiments, infants exhibited increased responding to auditory strings that violated the pattern of elements they were habituated to. We argue that a perceptual interpretation is more parsimonious, as well as more consistent with a broad array of habituation data, and we report successful neural network simulations that implement this lower?level interpretation. In the discussion, we discuss how our model relates to other habituation research, and how it compares to other neural network models of habituation in general, and models of the Marcus et al. (1999) task specifically.

The main motivation of this paper is to propose a new classification and regression method for challenging highdimensional data. The proposed new technique casts classification problems (class labels as output) and regression problems (numeric values as output) into a unified regression problem. This unified view enables classification problems to use numeric information in the output space that is available for regression problems but are traditionally not readily available for classification problemsÐdistance metric among clustered class labels for coarse and fine classifications. A doubly clustered subspace-based hierarchical discriminating regression (HDR) method is proposed in this work. The major characteristics include: 1) Clustering is performed in both output space and input space at each internal node, termed ªdoubly clustered.º Clustering in the output space provides virtual labels for computing clusters in the input space. 2) Discriminants in the input space are automatically derived from the clusters in the input space. These discriminants span the discriminating subspace at each internal node of the tree. 3) A hierarchical probability distribution model is applied to the resulting discriminating subspace at each internal node. This realizes a coarse-to-fine approximation of probability distribution of the input samples, in the hierarchical discriminating subspaces. No global distribution models are assumed. 4) To relax the per class sample requirement of traditional discriminant analysis techniques, a sample-size dependent negative-log-likelihood (NLL) is introduced. This new technique is designed for automatically dealing with small-sample applications, large-sample applications, and unbalanced-sample applications. 5) The execution of HDR method is fast, due to the empirical logarithmic time complexity of the HDR algorithm. Although the method is applicable to any data, we report the experimental results for three types of data: synthetic data for examining the near-optimal performance, large raw face-image data bases, and traditional databases with manually selected features along with a comparison with some major existing methods, such as CART, C5.0, and OC1.