This paper seeks to foster a discussion on whether experiments with robots can inform theory in infant motor development and specifically (1) how the interactions among the parts of a system, including the nervous and musculoskeletal systems and the forces acting on the body, induce organizational changes in the whole, and (2) how exploratory behaviour and selective informational signals at the timescale of skill learning may allow behaviour to become stabilized at the longer timescale of development. The paper describes how three generative principles, inspired from developmental biology and shown to underlie the dynamics of infants learning to bounce in a Jolly Jumper, were broken into a set of mechanisms suitable for controlling a robotic system and resulted in a similar developmental profile. A comparison of infant and robot data leads to a set of criteria for improving the usefulness of robotic studies.

In the present report, we describe a new theoretical framework for thinking about the dynamics of visual cognition—the dynamic field theory. This framework, grounded in neural principles, grew out of the dynamic systems approach to motor control and development. The initial development of the dynamic field theory extended concepts of the motor approach to decision making in a sensori-motor context, and, more recently, to the dynamics of spatial cognition. Here we extend these concepts still further to address topics in visual cognition, including visual working memory for non-spatial object properties, the processes that underlie change detection, and the ‘binding problem’ in vision. In each case, we demonstrate that the general principles of the dynamic field approach can unify findings in the literature and generate novel predictions. We contend that the application of these concepts to visual cognition avoids the pitfalls of reductionist approaches in cognitive science, and points toward a formal integration of brains, bodies, and behavior.

In this paper we discuss the concept of autobiographic agent and how memory may extend an agent's temporal horizon and increase its adaptability. These concepts are applied to an implementation of a scenario where agents are interacting in a complex virtual artificial life environment. We present computational memory architectures for autobiographic virtual agents that enable agents to retrieve meaningful information from their dynamic memories which increases their adaptation and survival in the environment. The design of the memory architectures, the agents, and the virtual environment are described in detail. Next, a series of experimental studies and their results are presented which show the adaptive advantage of autobiographic memory, i.e. from remembering significant experiences. Also, in a multi-agent scenario where agents can communicate via stories based on their autobiographic memory, it is found that new adaptive behaviours can emerge from an individual's reinterpretation of experiences received from other agents whereby higher communication frequency yields better group performance. An interface is described that visualizes the memory contents of an agent. From an observer perspective, the agents' behaviours can be understood as individually structured, and temporally grounded, and, with the communication of experience, can be seen to rely on emergent mixed narrative reconstructions combining the experiences of several agents. This research leads to insights into how bottom-up story-telling and autobiographic reconstruction in autonomous, adaptive agents allow temporally grounded behaviour to emerge. The article concludes with a discussion of possible implications of this research direction for future autobiographic, narrative agents.

Computational models have played a central role in the debate over language learnability. This article discusses how they have been used in different “stances”, from generative views to more recently introduced explanatory frameworks based on embodiment, cognitive development and cultural evolution. By digging into the details of certain specific models, we show how they organize, transform and rephrase defining questions about what makes language learning possible for children. Finally, we present a tentative synthesis to recast the debate using the notion of learning bias.

Robots and humans receive partial, fragmentary hints about the world’s state through their respective sensors. These hints – tiny patches of light intensity, frequency components of sound, etc. – are far removed from the world of objects we feel we perceive so effortlessly around us. The study of infant development and the construction of robots are both deeply concerned with how this apparent gap between the world and our experience of it is bridged. In this paper, we focus on some fundamental problems in perception that have attracted the attention of researchers in both robotics and infant development. Our goal is to identify points of contact already existing between the two fields, and also important questions identified in one field that could fruitfully be addressed in the other. We start with the problem of object segregation: how do infants and robots determine visually where one object ends and another begins? For object segregation, both fields have examined the idea of using “key events” where perception is in some way simplified and the infant or robot acquires knowledge that can be exploited at other times. We propose that the identification of the key events themselves constitutes a point of contact between the fields. And although the specific algorithms used in robots do not necessarily map directly to infant strategies, the overall “algorithmic skeleton” formed by the set of algorithms needed to identify and exploit key events may in fact form a basis for mutual dialogue. We then look more broadly at the role of embodiment in humans and robots, and see the opportunities it affords for development.

Robots and humans receive partial, fragmentary hints about the world’s state through their respective sensors. In this paper, we focus on some fundamental problems in perception that have attracted the attention of researchers in both robotics and infant development: object segregation, intermodal integration, and the role of embodiment. We concentrate on identifying points of contact between the two fields, and also important questions identified in one field and not yet addressed in the other. For object segregation, both fields have examined the idea of using “key events” where perception is in some way simplified and the infant or robot acquires knowledge that can be exploited at other times. We examine this parallel research in some detail. We propose that the identification of the key events themselves constitutes a point of contact between the fields. And although the specific algorithms used in robots are not easy to relate to infant development, the overall “algorithmic skeleton” formed by the set of algorithms needed to identify and exploit key events may in fact form a basis for mutual dialogue.

A central goal in cognitive and developmental science is to develop models of behavior that can generalize across both tasks and development while maintaining a commitment to detailed behavioral prediction. This paper presents tests of one such model, the Dynamic Field Theory (DFT). The DFT was originally proposed to capture delay-dependent biases in spatial recall and developmental changes in spatial recall performance. More recently, the theory was generalized to adults’ performance in a second spatial working memory task, position discrimination. Here, we use the theory to predict a specific, complex developmental pattern in position discrimination. Data with 3- to 6-year-old children and adults confirm these predictions, demonstrating that the DFT achieves generality across tasks and time scales, as well as the specificity necessary to generate novel, falsifiable predictions.

Within cognitive neuroscience, computational models are designed to provide insights into the organization of behavior while adhering to neural principles. These models should provide sufficient specificity to generate novel predictions while maintaining the generality needed to capture behavior across tasks and/or time scales. This paper presents one such model, the Dynamic Field Theory (DFT) of spatial cognition, showing new simulations that provide a demonstration proof that the theory generalizes across developmental changes in performance in four tasks—the Piagetian A-not-B task, a sandbox version of the A-not-B task, a canonical spatial recall task, and a position discrimination task. Model simulations demonstrate that the DFT can accomplish both specificity—generating novel, testable predictions—and generality—spanning multiple tasks across development with a relatively simple developmental hypothesis. Critically, the DFT achieves generality across tasks and time scales with no modification to its basic structure and with a strong commitment to neural principles. The only change necessary to capture development in the model was an increase in the precision of the tuning of receptive fields as well as an increase in the precision of local excitatory interactions among neurons in the model. These small quantitative changes were sufficient to move the model through a set of quantitative and qualitative behavioral changes that span the age range from 8 months to 6 years and into adulthood. We conclude by considering how the DFT is positioned in the literature, the challenges on the horizon for our framework, and how a dynamic field approach can yield new insights into development from a computational cognitive neuroscience perspective.

Infants gradually learn to predict the motion of moving targets and change from a strategy that mainly depends on saccades to one that depends on anticipatory control of smooth pursuit. A model is described that combines three types of mechanisms for gaze control that develops in a way similar to infants. Initially, gaze control is purely reactive, but as the anticipatory models become more accurate, the gain of the pursuit will increase and lead to a larger fraction of smooth eye movements. Finally, a third system learns to predict changes in target motion, which will lead to fast retuning of the parameters in the anticipatory model.

We investigate a computational model of the emergence of gaze following that is based on a generic basic set of mechanisms. Whereas much attention has been focused so far on the study of the infant's behavior, we systematically analyze the caregiver and show that he plays a crucial role in the development of gaze following in our model, especially for infant models with simulated developmental disorders such as autism and Williams syndrome. We first create two reference infant parameter sets and test their behavior with a simple standard caregiver. Based on these findings we then propose new caregiver models and evaluate them on normally developing infants and on infants with simulated developmental disorders. Further, we investigate if and how a pair of infants (with and without simulated developmental disorders) might learn gaze following from scratch, without a mature caregiver. The findings of this paper suggest the pivotal role the caregiver plays for the infant in developing gaze following, that his predictability is the most important criterion, and that different infant models require particular caregivers for gaze following to emerge optimally. Our simulations are consistent with Leekam's finding, that autistics can learn to follow gaze through a contingent presentation of rewarding visual stimuli, but that a lack of motivation may impede learning.

This survey presents an overview of the autonomous development of mental capabilities in computational agents. It does so based on a characterization of cognitive systems as systems which exhibit adaptive, anticipatory, and purposive goal-directed behavior. We present a broad survey of the various paradigms of cognition, addressing cognitivist (physical symbol systems) approaches, emergent systems approaches, encompassing connectionist, dynamical, and enactive systems, and also efforts to combine the two in hybrid systems. We then review several cognitive architectures drawn from these paradigms. In each of these areas, we highlight the implications and attendant problems of adopting a developmental approach, both from phylogenetic and ontogenetic points of view. We conclude with a summary of the key architectural features that systems capable of autonomous development of mental capabilities should exhibit.

This survey presents an overview of the autonomous development of mental capabilities in computational agents. It does so based on a characterization of cognitive systems as systems which exhibit adaptive, anticipatory, and purposive goaldirected behaviour. We present a broad survey of the various paradigms of cognition, addressing cognitivist (physical symbol systems) approaches, emergent systems approaches, encompassing connectionist, dynamical, and enactive systems, and also efforts to combine the two in hybrid systems. We then review several cognitive architectures drawn from these paradigms. In each of these areas, we highlight the implications and attendant problems of adopting a developmental approach, both from phylogenetic and ontogenetic points of view. We conclude with a summary of the key architectural features that systems capable of autonomous development of mental capabilities should exhibit.

Our exploratory research aims at suggesting design principles for educational software dedicated to high functioning autism. In order to explore the efficiency of educational games, we developed an experimental protocol to study the influence of the specific learning field constraints (spatial planning versus dialogue understanding) and Human Computer Interface modalities in this sector. We designed computer games that were tested with 10 teenagers diagnosed with high functioning autism, during 13 sessions, at the rate of one session per week. Participants' skills were assessed before and after a training period. A group of 10 typical children matched on academic level were also presented the experiment. A software platform was developed to manage interface modalities and log users' actions. Moreover, we annotated video recordings of two sessions. Results underline the influence of the task and interface modalities on executive functions.

We propose a model of intrinsic plasticity for a continuous activation model neuron based on information theory. We then show how intrinsic and synaptic plasticity mechanisms interact and allow the neuron to discover heavy-tailed directions in the input. We also demonstrate that intrinsic plasticity may be an alternative explanation for the sliding threshold postulated in the BCM theory of synaptic plasticity. We present a theoretical analysis of the interaction of intrinsic plasticity with different Hebbian learning rules for the case of clustered inputs. Finally, we perform experiments on the "bars" problem, a popular nonlinear independent component analysis problem.

Gaze following is the ability to redirect one's gaze to the location where another agent is looking. We present a computational model of how human infants or other agents may acquire gaze following by learning to predict the locations of interesting sights from the looking behavior of other agents through reinforcement learning. The model accounts for many findings about the development of gaze following in human infants. During learning, the model develops pre-motor representations that exhibit many properties characteristic of mirror neurons, but they are specific to looking behaviors. The existence of such a new class of mirror neurons is the main prediction of our model. The model also offers a parsimonious account of how these and possibly other mirror neurons may acquire their special response properties. In this account, visual representations of other agents' actions become associated with premotor neurons that represent the intention to perform corresponding actions. The model also demonstrates how this development may be obstructed in autism spectrum disorder, giving rise to specific physiological and anatomical differences in the mirror system.

This paper presents Incremental Hierarchical Discriminant Regression (IHDR) which incrementally builds a decision tree or regression tree for very high dimensional regression or decision spaces by an online, real-time learning system. Biologically motivated, it is an approximate computational model for automatic development of associative cortex, with both bottom-up sensory inputs and top-down motor projections. At each internal node of the IHDR tree, information in the output space is used to automatically derive the local subspace spanned by the most discriminating features. Embedded in the tree is a hierarchical probability distribution model used to prune very unlikely cases during the search. The number of parameters in the coarse-to-fine approximation is dynamic and data-driven, enabling the IHDR tree to automatically fit data with unknown distribution shapes (thus, it is difficult to select the number of parameters up front). The IHDR tree dynamically assigns long-term memory to avoid the loss-of-memory problem typical with a global-fitting learning algorithm for neural networks. A major challenge for an incrementally built tree is that the number of samples varies arbitrarily during the construction process. An incrementally updated probability model, called sample size dependent negative-log-likelihood (SDNLL) metric is used to deal with large-sample size cases, small-sample size cases, and unbalanced-sample size cases, measured among different internal nodes of the IHDR tree. We report experimental results for four types of data: synthetic data to visualize the behavior of the algorithms, large face image data, continuous video stream from robot navigation, and publicly available data sets that use human defined features. Keywords: online learning, incremental learning, cortical development, discriminant analysis, local invariance, plasticity, decision trees, high dimensional data, classification, regression, and autonomous development.

Currently, there is a lack of general-purpose, in-place learning engines that incrementally learn multiple tasks, to develop “soft” multi-task-shared invariances in the intermedi- ate internal representation while a developmental robot interacts with its environment. In-place learning is a biologically inspired concept, rooted in the genomic equivalence principle, meaning that each neuron is responsible for its own development while interact- ing with its environment. With in-place learning, there is no need for a separate learning network. Computationally, biologically inspired, in-place learning provides unusually effi- cient learning algorithms whose simplicity, low computational complexity, and generality are set apart from typical conventional learning algorithms. We present in this paper the multiple-layer in-place learning network (MILN) for this ambitious goal. As a key requirement for autonomous mental development, the network enables both unsupervised and supervised learning to occur concurrently, depending on whether motor supervision signals are available or not at the motor end (the last layer) during the agent’s interac- tions with the environment. We present principles based on which MILN automatically develops invariant neurons in different layers and why such invariant neuronal clusters are important for learning later tasks in open-ended development. From sequentially sensed sensory streams, the proposed MILN incrementally develops a hierarchy of inter- nal representations. The global invariance achieved through multi-layer invariances, with increasing invariance from early layers to the later layers. Experimental results with statistical performance measures are presented to show the effects of the principles. Keywords: In-place learning; positional invariance; size invariance; style invariance; incremental learning; internal representation; biologically inspired network.

Three different models of tactile shape perception inspired by the human haptic system were tested using an 8 d.o.f. robot hand with 45 tactile sensors. One model is based on the tensor product of different proprioceptive and tactile signals and a self-organizing map (SOM). The two other models replace the tensor product operation with a novel self-organizing neural network, the Tensor-Multiple Peak Self-Organizing Map (T-MPSOM). The two T-MPSOM models differ in the procedure employed to calculate the neural activation. The computational models were trained and tested with a set of objects consisting of hard spheres, blocks and cylinders. All the models learned to map different shapes to different areas of the SOM, and the tensor product model as well as one of the T-MPSOM models also learned to discriminate individual test objects.

Children seem to acquire new know-how in a continuous and open-ended manner. In this paper, we hypothesize that an intrinsic motivation to progress in learning is at the origins of the remarkable structure of children’s developmental trajectories. In this view, children engage in exploratory and playful activities for their own sake, not as steps toward other extrinsic goals. The central hypothesis of this paper is that intrinsically motivating activities correspond to expected decrease in prediction error. This motivation system pushes the infant to avoid both predictable and unpredictable situations in order to focus on the ones that are expected to maximize progress in learning. Based on a computational model and a series of robotic experiments, we show how this principle can lead to organized sequences of behavior of increasing complexity characteristic of several behavioral and developmental patterns observed in humans.We then discuss the putative circuitry underlying such an intrinsic motivation system in the brain and formulate two novel hypotheses. The first one is that tonic dopamine acts as a learning progress signal. The second is that this progress signal is directly computed through a hierarchy of microcortical circuits that act both as prediction and metaprediction systems.

This article presents recent research investigating how a robot equipped with an intrinsic motivation system can explore its environment and learn a sequence of tasks not initially specified by its programmer. A generic software architecture controls the robot, driving it towards situations where learning progress is maximal. These situations – called « progress niches » - depend on opportunities offered by the environment but also on the robot’s morphology, specific cognitive biases and past experiences. First results have been obtained in the field of locomotion, affordance discovery and prelinguistic communication. In each of these experiments, the robot explores situations that it evaluated as « interesting » given its learning capabilities and biases of its sensorimotor space. The article discusses the results of these initial experiments and concludes on the relevance of this research to offer neurosciences and psychology – that inspired these investigations in the first place – new kinds of concepts and explanations to think about developmental processes in the young child.

Robot imitation is a useful and promising alternative to robot programming. Robot imitation involves two crucial issues. The first is how a robot can imitate a human whose physical structure and properties differ greatly from its own. The second is how the robot can generate various motions from finite programmable patterns (generalization). This paper describes a novel approach to robot imitation based on its own physical experiences. We considered the target task of moving an object on a table. For imitation, we focused on an active sensing process in which the robot acquires the relation between the object's motion and its own arm motion. For generalization, we applied the RNNPB (recurrent neural network with parametric bias) model to enable recognition/generation of imitation motions. The robot associates the arm motion which reproduces the observed object's motion presented by a human operator. Experimental results proved the generalization capability of our method, which enables the robot to imitate not only motion it has experienced, but also unknown motion through nonlinear combination of the experienced motions.

We present a developmental perspective of robot learning that uses affordances as the link between sensory-motor coordination and imitation. The key concept is a general model for affordances able to learn the statistical relations between actions, object properties and the effects of actions on objects. Based on the learned affordances, it is possible to perform simple imitation games providing both task interpretation and planning capabilities. To evaluate the approach, we provide results of affordance learning with a real robot and simple imitation games with people.

In this paper, we present a strategy whereby a robot acquires the capability to learn by imitation following a developmental pathway consisting on three levels: 1) sensory-motor coordination; 2) world interaction; and 3) imitation. With these stages, the system is able to learn tasks by imitating human demonstrators. We describe results of the different developmental stages, involving perceptual and motor skills, implemented in our humanoid robot, Baltazar. At each stage, the system's attention is drawn toward different entities: its own body and, later on, objects and people. Our main contributions are the general architecture and the implementation of all the necessary modules until imitation capabilities are eventually acquired by the robot. Also, several other contributions are made at each level: learning of sensory-motor maps for redundant robots, a novel method for learning how to grasp objects, and a framework for learning task description from observation for program-level imitation. Finally, vision is used extensively as the sole sensing modality (sometimes in a simplified setting) avoiding the need for special data-acquisition hardware

We motivate and present a definition of an embodied, grounded individual sensorimotor interaction history, which captures the time-extended behavior characteristics of humans and many animals. We present an architecture that connects temporally extended individual experience with capacity for action, whereby a robot can develop over ontogeny through interaction. Central to this is an information theoretic metric space of sensorimotor experience, which is dynamically constructed and reconstructed as the robot acts. We present results of robotic experiments that establish the predictive efficacy of the space and we show the robot developing the capacity to play the simple interaction game peekaboo. A quantitative investigation of the appropriate horizon length of experience for the game reveals the relationship between the length of experience and the cycle time of interaction, and suggests the importance of multiple, and possibly self-adaptive, horizon lengths.

We motivate and present a definition of an embodied, grounded individual sensorimotor interaction history, which captures the time-extended behavior characteristics of humans and many animals. We present an architecture that connects temporally extended individual experience with capacity for action, whereby a robot can develop over ontogeny through interaction. Central to this is an information theoretic metric space of sensorimotor experience, which is dynamically constructed and reconstructed as the robot acts. We present results of robotic experiments that establish the predictive efficacy of the space and we show the robot developing the capacity to play the simple interaction game "peekaboo." A quantitative investigation of the appropriate horizon length of experience for the game reveals the relationship between the length of experience and the cycle time of interaction, and suggests the importance of multiple, and possibly self-adaptive, horizon lengths.

Intrinsic plasticity (IP) refers to a neuron's ability to regulate its firing activity by adapting its intrinsic excitability. Previously, we showed that model neurons combining a model of IP based on information theory with Hebbian synaptic plasticity can adapt their weight vector to discover heavy-tailed directions in the input space. In this paper we show how a network of such units can solve a standard non-linear independent component analysis (ICA) problem. We also present a model for the formation of maps of oriented receptive fields in primary visual cortex and compare our results with those from ICA. Together, our results indicate that intrinsic plasticity that tries to locally maximize information transmission at the level of individual neurons may play an important role for the learning of efficient sensory representations in the cortex.

Computational models have played a central role in the debate over language learnability. This article discusses how they have been used in different “stances”, from generative views to more recently introduced explanatory frameworks based on embodiment, cognitive development and cultural evolution. By digging into the details of certain specific models, we show how they organize, transform and rephrase defining questions about what makes language learning possible for children. Finally, we present a tentative synthesis to recast the debate using the notion of learning bias.

Intrinsic motivation, the causal mechanism for spontaneous exploration and curiosity, is a central concept in developmental psychology. It has been argued to be a crucial mechanism for open-ended cognitive development in humans, and as such has gathered a growing interest from developmental roboticists in the recent years. The goal of this paper is threefold. First, it provides a synthesis of the different approaches of intrinsic motivation in psychology. Second, by interpreting these approaches in a computational reinforcement learning framework, we ar- gue that they are not operational and even sometimes inconsistent. Third, we set the ground for a systematic operational study of intrinsic motivation by presenting a formal typology of possible computational approaches. This typology is partly based on existing computational models, but also presents new ways of conceptualizing intrinsic motivation. We argue that this kind of computational typology might be useful for opening new avenues for research both in psychology and developmental robotics.

This paper presents computational experiments that illustrate how one can precisely conceptualize language evolution as a Darwinian process. We show that there is potentially a wide diversity of replicating units and replication mechanisms involved in language evolution. Computational experiments allow us to study systemic properties coming out of populations of linguistic replicators: linguistic replicators can adapt to specific external environments; they evolve under the pressure of the cognitive constraints of their hosts, as well as under the functional pressure of communication for which they are used; one can observe neutral drift; coalitions of replicators may appear, forming higher level groups which can themselves become subject to competition and selection.

The problem of developing local reactive obstacle-avoidance behaviors by a mobile robot through online real-time learning is considered. The robot operated in an unknown bounded 2-D environment populated by static or moving obstacles (with slow speeds) of arbitrary shape. The sensory perception was based on a laser range finder. A learning-based approach to the problem is presented. To greatly reduce the number of training samples needed, an attentional mechanism was used. An efficient, real-time implementation of the approach was tested, demonstrating smooth obstacle-avoidance behaviors in a corridor with a crowd of moving students as well as static obstacles. Key words: Humanoid, Reactive Collision Avoidance, Range Finder, Perception

This paper focuses on three connected to each other fundamental beliefs that appear to be unconsciously taken for granted at the basis of modern cognitive neuroscience: (i) the attractor hypothesis, (ii) the neural code hypothesis, and (iii) the supervenience hypothesis about human subjective experience. Their precise understanding and experimental verification is an important challenge that calls for a scientific paradigm change and needs to be addressed for future progress in cognitive neuroscience. The work presents a theoretical analysis and preliminary experimental and modeling results that suggest a roadmap to solving the challenge.

Imitative learning and learning by observation are social mechanisms that allow a robot to acquire knowledge from a human or another robot. However to be able to obtain skills in this way the robot faces many complex issues, one of which is that of finding solutions to the correspondence problem. Evolutionary predecessors to observational imitation may have been self-imitation where an agent avoids the complexities of the correspondence problem by learning and replicating actions it has experienced through the manipulation of its body. We investigate how a robotic control and teaching system using self-imitation can be constructed with reference to psychological models of motor control and ideas from social scaffolding seen in animals. Within these scaffolded environments sets of competencies can be built by constructing hierarchical state/action memory maps of the robot's interaction within that environment. The scaffolding process provides a mechanism to enable learning to be scaled up. The resulting system allows a human trainer to teach a robot new skills and modify skills that the robot may possess. Additionally the system allows the robot to notify the trainer when it is being taught skills it already has in its repertoire and to direct and focus its attention and sensor resources to relevant parts of the skill being executed. We argue that these mechanisms may be a first step towards the transformation from self-imitation to observational imitation. The system is validated on a physical pioneer robot that is taught using self-imitation to track, follow and point to a patterned object.