Bayesian mechanisms of induction in children

Listen to audio

The interest aroused by Bayesian theory in cognitive psychology is based on certain observations which suggest that very young children, from the very first months, are capable of plausible, probably unconscious, reasoning. For example, a 12-month-old baby, seeing the contents of an urn, can anticipate the result of a random draw: if the urn contains three blue objects and one yellow, the child shows his surprise by looking longer when it's the yellow object that comes out (Teglas, Girotto, Gonzalez & Bonatti, 2007 ; Teglas et al., 2011). And conversely, as described by Laura Schulz in the seminar, 8-month-old babies can infer the contents of an urn from a few samples (Gweon, Tenenbaum & Schulz, 2010 ; Xu & Garcia, 2008).

Joshua Tenenbaum and colleagues suggest that language learning provides one of the most compelling examples of Bayesian induction learning (Kemp & Tenenbaum, 2009 ; Perfors, Tenenbaum, Griffiths & Xu, 2011 ; Tenenbaum, Kemp, Griffiths & Goodman, 2011). Indeed, the theory provides elements of an answer to the " scandal of induction " debated since antiquity (Plato, Aristotle, Kant, Peirce, Quine, Russell), and which Tenenbaum summarizes as follows: " how do our minds manage to infer so much from so little ? " Children and adults alike make highly sophisticated inferences on a daily basis, when it seems obvious that the available data doesn't justify it. For example, every scientist knows that " correlation is not causation " - and yet humans regularly infer causal relationships, on the basis of so little data that it wouldn't even be enough to calculate a correlation coefficient.

Tenenbaum expands on the example, discussed by Quine (1960), of word learning. How does a child or adult infer the extension of a word on the basis of a few examples, whether an instance (" Fido ") or a more or less extended category (" basset ", " dog ", " animal ") ? This observation can be modeled in the Bayesian framework, assuming that the perceptual world is organized as a tree of hypotheses. A lower likelihood is automatically assigned to the broadest categories. The model converges on two or three examples : it assigns the highest plausibility to the correct meaning of the word, and correctly generalizes to new instances. In fact, the Bayesian model provides a universal theory of how learning generalizes (Shepard, 1987), a model that captures a huge amount of empirical data on the psychophysics of perceptual generalization.

Bayesian theory offers a vision of learning that assigns a well-defined place to innate knowledge : learning is constrained by the space of hypotheses considered, the a priori probability function and the likelihood function. In addition, a recent strand of the theory introduces the fundamental concept of the hierarchical Bayesian model : the hypothesis space at level n can , in turn, be generated by a generative model of higher level n+1, and its a priori probabilities determined by a small number ofhyper-parameters. Once the hierarchical model has been specified in detail, Bayesian theory then provides a simple way of updating plausibilities at each level.

Such hierarchical Bayesian models possess an interesting property that Tenenbaum calls " blessingof abstraction ": learning is often very rapid at the highest level, that of the very principles of organization of a domain, because each new observation contributes to selecting the most relevant model, and this generalizes to all new observations. This property is akin to what is known as " learning transfer " or " learning to learn ". To take an example from Nelson Goodman (1955), suppose a single blue ball is drawn from a single urn. What can we conclude from this about subsequent draws ? Nothing... unless, in the past, we've discovered that each bag contains only balls of a single color, in which case we can be almost certain that all subsequent balls will be blue. Tenenbaum (2011) shows how to model this situation using a hierarchical Bayesian model (Dirichlet-multinomial model).

The Bayesian induction model is directly applicable to Linda B. Smith's 17-19 month-old word learning experiments. Smith (2002). In this study, a child sees an adult calling shapes by new names (wif, dax, etc.). Systematically, the child sees that the same syllable applies to two objects of the same shape, but varying in texture, size and color. Subsequently, exposed to a single instance of a new word (" lug ") paired with a new object, he correctly generalizes the use of this word to these new objects of the same shape, while refusing to apply the word to objects of the same texture, size or color. The child has thus induced a general principle that governs most common nouns : each of them refers to an object of a given shape. Bayesian learning is able to explain and model this capacity for generalization in detail: " Each bit of individual learning changes the learner, and thus progressively changes what the learner finds easy to learn. " (Smith et al., 2002).

On the basis of a more complex computer algorithm, Kemp and Tenenbaum (2008) succeed in modeling the automatic induction of the deep structure of numerous domains : the tree structure of the family of living beings, the circular structure of the globe, or even the principle of causality itself. We are therefore not far from modelling the great principles of scientific discovery, even if cognitive psychology data are still lacking to validate this.

In short, Bayesian models include both very rich a priori hypotheses (on the space of mental representations under consideration and their a priori probability), and a simple yet powerful mechanism for learning by statistical inference. Until recently, cognitive psychology was very divided on the role of learning in the organization of children's skills. Connectionism, on the one hand, proposed sophisticated learning mechanisms, but focused on unstructured representations of knowledge (association matrices). The " representational " theories, on the other hand, assumed sophisticated, structured symbolic representations of knowledge, but lacked any real learning mechanism. The Bayesian approach has the advantage of reconciling these two camps, thus overcoming many caricatured and sterile oppositions (innate/acquired ; Chomskyan nativism versus Piagetian constructivism or Skinnerian associationism).