Optimal search on a technology landscape

https://doi.org/10.1016/S0167-2681(00)00114-1Get rights and content

Abstract

We address the question of how a firm’s current location in the space of technological possibilities constrain its search for technological improvements. We formalize a quantitative notion of distance between technologies — encompassing the distinction between evolutionary changes (small distance) versus revolutionary change (large distance) — and introduce a technology landscape into an otherwise standard dynamic programming setting where the optimal strategy is to assign a reservation price to each possible technology. Technological search is modeled as movement, constrained by the cost of search, on a technology landscape. Simulations are presented on a stylized technology landscape while analytic results are derived using landscapes that are similar to Markov random fields. We find that early in the search for technological improvements, if the initial position is poor or average, it is optimal to search far away on the technology landscape; but as the firm succeeds in finding technological improvements it is optimal to confine search to a local region of the landscape.

Introduction

We address the question of how a firm’s current production practices and its location in the space of technological possibilities constrain its search for technological improvements. We formalize a quantitative notion of technological distance which encompasses the distinction between evolutionary change (small distance) versus revolutionary change (large distance). We are particularly interested in the relationship between the firm’s current location in the space of technological possibilities and the distance at which the firm should search for technological improvements. Our formalization results in a Landscape Search Model (LSM) search based upon a technology landscape which extends traditional search theory. In the present discussion we focus on a detailed application of the LSM to optimal firm search.2 Sufficient detail is provided so that readers unfamiliar with the LSM will find a self-contained treatment.

The starting point for our discussion is the representation of technology first presented in Auerswald and Lobo, 1996, Auerswald et al., 2000. In this framework a firm’s production plan is more than a point in input–output space; it also includes the production recipe used in the process of production. A configuration denotes a specific assignment of states for every operation in the production recipe. A production recipe is comprised of N distinct operations, each of which can occupy one of S discrete states. The productivity of labor employed by a firm is a summation over the labor efficiency associated with each of the N production operations. The labor efficiency of any given operation is dependent on the state that it occupies, as well as the states of e other operations. The parameter e represents the magnitude of production externalities among the N operations comprising a production recipe, what we refer to as “intranalities”. In the course of production during any given time period, the state of one or more operation is changed as a result either of spontaneous experimentation or strategic behavior. This change in the state of one or more operations of the firm’s production recipe alters the firm’s labor efficiency. The firm improves its labor efficiency — i.e., to say, the firm finds technological improvements — by searching over the space of possible configurations for its production recipe. When a firm finds a more efficient production recipe, it adopts that recipe in the next production period with certainty.

In order to explicitly consider the ways in which the firm’s technological search is constrained by the firm’s location in the search space, as well as the features of the space, we go beyond the standard search model (based on dynamic programming) and specify a technology landscape. The distance metric on the technology landscape is defined by the number of operations whose states need to be changed in order to turn one configuration into another. The firm’s search for more efficient, production recipes is studied here as a “walk” on a technology landscape. The cost of search paid by the firm when sampling a new configuration is a nondecreasing function of the number of operations in the newly sampled configuration whose states differ from those in the currently utilized production recipe.

The literature on technology management and organizational behavior emphasize that although firms employ a wide range of search strategies, firms tend to engage in local search — i.e., search that enables firms to build upon their established technology (see, e.g., Barney, 1991, Boeker, 1989, Helfat, 1994, Henderson and Clark, 1990, Lee and Allen, 1982, Sahal, 1985, Shan, 1990, Tushman and Anderson, 1986). As discussed in March, 1991, Stuart and Podolny, 1996, the prevalence of local search stems from the significant effort required for firms to achieve a certain level of technological competence, as well as from the greater risks and uncertainty faced by firms when they search for innovations far away from their current location in the space of technological possibilities. Using both numerical and analytical results we relate the optimal search distance to the firm’s initial productivity, the cost of search, and the correlation structure of the technology landscape. As a preview of our main result, we find that early in the search for technological improvements, if the firm’s initial technological position is poor or average, it is optimal to search far away on the technology landscape. As the firm succeeds in finding technological improvements, however, it is indeed optimal to confine search to a local region of the technology landscape. We also obtain the familiar result that there are diminishing returns to search but without having to assume that the firm’s repeated draws from the space of possible technologies are independent and identically distributed.

The outline of the paper is as follows. Section 2.1 presents a simple model of firm-level technology; production recipes are introduced in Section 2.1, production “intranalities” are defined in Section 2.2, and firm-level technological change is discussed in Section 2.3. The material in these sections draws heavily from Auerswald and Lobo, 1996, Auerswald et al., 2000 and further details can be found there. Section 3 develops the notion of a technology landscape, which is defined in Section 3.1. The correlation structure of the technology landscape is introduced in Section 3.2 as an important characteristic defining the landscape. Section 4 treats the firm’s search for improved production recipes as movement on its technology landscape. The cost of this search is considered in Section 4.1. Section 4.2 then presents simulation results of search for the Ne technology landscape model defined in Section 2.2. We then go on to develop an analytically tractable model of technology landscapes in Section 5. We also describe in this section how a landscape can be represented by a probability distribution under an annealed approximation. Section 6 considers search under this formal model. The firm’s search problem is formally defined in Section 6.1 and the important role of reservation prices is considered in Section 6.2. Section 6.3 determines the reservation price which determines optimal search and results are presented in Section 6.4. We conclude in Section 7 with a summary of results and some suggestions for further work.

Section snippets

Production recipes

A recent body of work, both empirical and theoretical, emphasizes the importance of firm specific characteristics for explaining technological change (for empirical contributions to this literature, see, e.g., Dunne, 1988, Dunne, 1989, Audretsch, 1991, Davis and Haltiwanger, 1992, Audretsch, 1994, Bailey et al., 1994, Dwyer, 1995, Dunne et al., 1996; for theoretical contributions, see, e.g., Jovanovic, 1982, Herriott et al., 1985, Hopenhayn, 1992, Kennedy, 1994, Ericson and Pakes, 1995). Our

Defining the technology landscape

To define a technology landscape we require a measure of distance between two different production recipes, ωi and ωj, each drawn from Ω. The distance metric used here is not based on the relative efficiencies of production recipes, but rather on the similarity between the operations constituting the recipes. More precisely, the distance d(ωi,ωj) between the production recipes ωi and ωj is the minimum number of operations which must be changed in order to convert ωi to ωj. Since changing

Search cost

The firm’s walk on a technology landscape is similar to a random search within a fixed population of possibilities (Stone, 1975).5 In the

Analytic approximation for the distribution of efficiencies

Technology landscapes are very complex entities, characterized by a neighborhood graph Γ and an exponential number of labor efficiencies SN. In any formal description of technology landscapes we have little hope of treating all of these details. Consequently we adopt a probabilistic approach focusing on the statistical regularities of the landscape and which is applicable to any landscape model.

To treat the technology landscape statistically we follow Macready (1999) and assume that the

The firm’s search problem

In order to determine the relationship between search cost and optimal search distance on a technology landscape, we recast the firm’s search problem into the familiar framework of dynamic programming Bellman, 1957, Bertsekas, 1976, Sargent, 1987. Recall that each production recipe ωi∈Ω is associated with a labor efficiency θi. Production recipes at different locations in the technology landscape — and therefore at different distances from each other — have different Gaussian distributions

Conclusion

In this discussion we have been concerned with the determination of the optimal distance at which a firm should seek technological improvements in a space of possible technologies. In our model the firm’s technology is determined by its organizational capital which in turn is represented by a production recipe whose N constituent operations can occupy S discrete states. Different configurations for a production recipe represent different technologies. Production recipes are also characterized

Acknowledgements

The authors thank Phil Auerswald, Richard Day, Richard Durrett, Alfred Nucci, Richard Schuler, Robert Solow, Nancy Stokey, Willard Zangwill and an anonymous referee for their helpful comments. The authors also gratefully acknowledge research support provided by the Santa Fe Institute and Bios Group LP.

References (71)

  • D. Audretsch

    Business survival and the decision to exit

    Journal of Business Economics

    (1994)
  • Auerswald, P., Lobo, J., 1996. Learning by doing, technological regimes and industry evolution. Presented at the 71st...
  • Bailey, M.N., Bartelsman, E.J., Haltiwanger, J., 1994. Downsizing and productivity growth: myth or reality? Center for...
  • J.B. Barney

    Firm resources and sustained competitive advantage

    Journal of Management

    (1991)
  • Bellman, R., 1957. Dynamic Programming. Princeton University Press, Princeton,...
  • Bertsekas, D.P., 1976. Dynamic Programming and Stochastic Control. Academic Press, New...
  • W. Boeker

    Strategic change: the effects of founding and history

    Academy of Management Journal

    (1989)
  • Cameron, P.J., 1994. Combinatorics: Topics, Techniques and Algorithms. Cambridge University Press, New...
  • H.B. Chenery

    Engineering production functions

    Quarterly Journal of Economics

    (1949)
  • R. Coase

    The nature of the firm

    Economica

    (1937)
  • W.M. Cohen et al.

    Innovation and learning: the two faces of R&D

    Economic Journal

    (1989)
  • R.M. Corless et al.

    On the Lambert W function

    Advances in Computational Mathematics

    (1996)
  • S.J. Davis et al.

    Gross job creation, gross job destruction, and employment reallocation

    Quarterly Journal of Economics

    (1992)
  • DeGroot, M.H., 1970. Optimal Statistical Decisions. McGraw-Hill, New...
  • B. Derrida et al.

    Random networks of automata: a simple annealed approximation

    Europhysics Letters

    (1986)
  • T. Dunne et al.

    Patterns of firm entry and exit in US manufacturing industries

    RAND Journal of Economics

    (1988)
  • T. Dunne et al.

    The growth and failure of US manufacturing plants

    Quarterly Journal of Economics

    (1989)
  • Dunne, T., Haltiwanger, J., Troske, K.R., 1996. Technology and jobs: secular change and cyclical dynamics. NBER Working...
  • Dwyer, D.W., 1995. Technology locks, creative, destruction and non-convergence in productivity levels. Center for...
  • Dyson, F., 1997. Imagined Worlds: The Jerusalem–Harvard Lectures. Harvard University Press, Cambridge,...
  • M. Eigen et al.

    The molecular quasispecies

    Advances in Chemical Physics

    (1989)
  • R. Ericson et al.

    Markov — perfect industry dynamics: a framework for empirical work

    Review of Economic Studies

    (1995)
  • R.E. Evenson et al.

    A stochastic model of applied research

    Journal of Political Economy

    (1976)
  • W. Fontana et al.

    RNA folding and combinatory landscapes

    Physical Review E

    (1993)
  • Griffeath, D., 1976. Introduction to random fields. In: Kemeny, J., Snell, J., Knapp, A. (Eds.), Denumerable Markov...
  • Cited by (0)

    1

    Tel.: +1-607-255-5385.

    View full text