Dynamics of a birth–death process based on combinatorial innovation

Highlights

• We define and analyse a simple model of human cultural and technological evolution.

• In this stochastic model, subsets of items (objects, ideas, techniques) lead to new items.

• This model predicts a ‘hockey-stick’ behaviour observed in certain historical data.

Abstract

A feature of human creativity is the ability to take a subset of existing items (e.g. objects, ideas, or techniques) and combine them in various ways to give rise to new items, which, in turn, fuel further growth. Occasionally, some of these items may also disappear (extinction). We model this process by a simple stochastic birth–death model, with non-linear combinatorial terms in the growth coefficients to capture the propensity of subsets of items to give rise to new items. In its simplest form, this model involves just two parameters (P, α). This process exhibits a characteristic ‘hockey-stick’ behaviour: a long period of relatively little growth followed by a relatively sudden ‘explosive’ increase. We provide exact expressions for the mean and variance of this time to explosion and compare the results with simulations. We then generalise our results to allow for more general parameter assignments, and consider possible applications to data involving human productivity and creativity.

Introduction

In this paper, we introduce and analyse a new mathematical model for the broad process of the cumulative, combinatorial nature of technological evolution (Arthur, 2009, Kauffman, 2008, Kauffman, 2016, Kauffman, 2019, Ogburn, 1922, Read, Andersson, Valverde, 2016). Consider the course of technological evolution over the course of hominid evolution (Stringer, 2016, Stringer, Galway-Witham, 2017). In the Lower Paleolithic, about 2.6 million years ago, our ancestor Australopithecus first started shaping simple stone tools such as diggers and scrapers. The diversity of these tools was perhaps a dozen or so. During the Upper Paleolithic and Mesolithic, the diversity of stone tools increased at a glacial pace. Stone knife blades grew longer and sharper over hundreds of thousands of years. Compound tools such as a knife blade hafted to a bone handle by sinew emerged perhaps 300,000 years ago. Homo sapiens arose about 150,000 years ago. By the time of Cro Magnon, 50,000 to 15,000 years ago, the number of stone tools had increased to perhaps several hundred, ranging from bone needles and bone flutes to arrow heads, fluted fish hooks, and the spear thrower. Ten thousand years later, in the time of Mesopotamia about 5000 years ago, the number of tools had increased to perhaps thousands, ranging in complexity from needles and pots to war chariots.

Still another 5000 years later (today), the diversity of ‘tools’ has exploded into the billions, ranging from the 60,000-year-old needle to machine tools, televisions, computers, and the International Space Station (ISS). The Wright Brothers took their first flight in 1903, and a mere 66 years later, Apollo landed on the moon.

This history of human technological evolution shows two major features. At first, it proceeded at a glacial pace for a very long time as the complexity of goods and tools increased very slowly. But then the process exploded upward, creating an enormous array of tools, from simple to complex. In this late explosion the rate of change increased enormously. In less than a century we have gone from the advent of computers to word processing, the World Wide Web, smartphones with thousands of apps, and the Space Shuttle, to name just a few.

In this paper, we analyse a simple model that can explain this initial long and slow advance, followed by a sudden ‘hockey-stick’ upward trend in which an increasing number of distinct items of increasing complexity appear. Here increasing ‘complexity’ means that newly-arising items (goods, tools etc) combine features of several existing items, which in turn have resulting from combining features of earlier items, and so on. Thus, although M_t measures the number of distinct items, the growth of M_t is associated with increasing complexity of the items themselves in the model presented here.

This model develops the theme that “combinatorics is at the heart of innovation [and so] provides a possible rationale for the accelerating growth of innovations” Solé et al. (2013). It is based, in part on the notion of the ‘adjacent possible’, which was introduced some time ago to refer to the new things that could possibly arise next, given what is in existence now (Kauffman, 2008, Kauffman, 2019). For example, before the development of rocketry, the Space Shuttle and ISS could not arise. However, in the early 20th Century, when Robert H. Goddard was trying to invent rocketry, the space shuttle and ISS were already in the adjacent possible. What exists now does not necessarily cause, but certainly enables what could arise next.

Our model exhibits the phenomenon of an ‘explosion’ within finite time due to a non-linear (positive feedback) terms. The phenomenon is well known, both in the setting of deterministic differential equation modelling (see for example Goriely and Hyde, 2000), and continuous-time stochastic birth-processes (see e.g. Feller, 1968, Norris, 1999). Such processes can reach infinity in finite time when the (expected) rate of population increase grows sufficiently faster than linearly with population size. Indeed, a paper in Science from 1960 noted that a simple dynamical growth process (based on a first order non-linear differential equation with parameters estimated from historical demographic data) predicted that the world’s population would reach infinity in the year 2026 (von Foerster et al., 1960). Of course, this prediction was not meant to be taken literally; it instead suggested a sudden future upward growth some decades in the future, but not actually reaching infinity, due to factors such as the limited carrying capacity of the planet (as well as time-delays between generations). More refined studies that take such factors explicitly into account have since appeared (see e.g. Cohen, 1995, and Johansen and Sornette, 2001; this last paper also investigated applications to global GDP, a topic we consider also later in this paper). Similarly, in our model, reaching infinity in finite time should be regarded in applications as predicting only a sudden explosion towards a very large (but finite) value. Also, in this paper, our emphasis is on establishing results particular to our model (e.g. expressions for the expected time to extinction) which do not directly follow from more general results.

We propose that a simple cumulative combinatorial process underlies this pattern of human technological evolution, an idea that builds on earlier studies, for which some empirical data has been presented and discussed (see e.g. Ambrose, 2001, Arthur, 2009, Boyd, Richerson, Henrich, Lupp, 2013, Koppl, Devereaux, Herriot, Kauffman, Solé, Ferrer-Cancho, Montoya, Valverde, 2003). In this setting, we assume that humans take whatever lies at hand to fit a purpose and combine these in different possible ways, seeking combinations of them that might together serve the desired purpose. These (possibly arbitrary) combinations are then tested to see if any of the new artifacts work, thus accumulating new goods or tools that are useful in some way. Some of these inventions eventually become innovations (i.e. inventions that become economically or socially successful Erwin and Krakauer, 2004), though our model does not explicitly model this distinction.

This simple feature of this process of human inventive exploration suggests the following equation (from Koppl et al. (2018)):

$$M_{t+1}=M_t+\sum _{i=1}^{M_t}{\alpha }_i\left(\genfrac{}{}{0pt}{}{M_t}{i}\right),$$

where M_t is the number of goods or tools in the economy at time t and α_i, i ≥ 1 is a decreasing sequence of positive real numbers (each less than 1.0) which reflects the decreasing ease of finding and testing useful combinations among an increasing number of goods.

However, Eq. (1) has a number of shortcomings. Firstly, it will generally require M_t to take non-integer values, which is problematic for interpreting both the term $\left(\genfrac{}{}{0pt}{}{M_t}{i}\right)$ and the range of summation¹. Secondly, Eq. (1) is purely deterministic, whereas evolutionary processes are typically best modelled by a stochastic approach (Felsenstein, 2004, Yule, 1925). Thirdly, Eq. (1) allows items to be gained but not lost.

In the next section, we describe and analyse a stochastic process, which we call the Combinatorial Formation (CF) model, based on Eq. (1), which avoids these shortcomings. In Section 3 we then show the results from two simulation models, one based on the deterministic Eq. (1) and one based on the stochastic CF model. These simulation results agree well with the theoretical predictions derived from the CF model. Moreover, the deterministic simulation model accurately represents the average behaviour of the stochastic simulation model. We end by discussing the implications of our model and its results for describing technological evolution, and how it could have (indirect) consequences for human evolution.