How Software is Eating the World by Venkatesh Rao

How Software is Eating the World

• A great deal of product development is based on the assumption that products must adapt to unchanging human needs or risk being rejected. Yet, time and again, people adapt in unpredictable ways to get the most out of new tech.
• People change, then forget that they changed, and act as though they always behaved a certain way and could never change again. Because of this, unexpected changes in human behavior are often dismissed as regressive rather than as potentially intelligent adaptations.
• To "break smart" is to adapt intelligently to new technological possibilities.
• After written language and money, software is only the third major soft technology to appear in human civilization.
• Only a handful of general-purpose technologies--electricity, steam power, precision clocks, written language, token currencies, iron metallurgy, and agriculture among them--have impacted our world in the sort of deeply transformative way that deserves the description eating. And only two of these, written language and money, were soft technologies: seemingly ephemeral, but capable of being embodied in a variety of specific physical forms.
• Software has the same relationship to any specific sort of computing hardware as money does to coins or credit cards or writing to clay tables and paper books.
• This is breaking smart: an economic actor using early mastery of emerging technological leverage to wield disproportionate influence on the emerging future.
• Software eating the world is a story of the seen and the unseen: small, measurable effects that seem underwhelming or even negative, and large invisible and positive effects that are easy to miss, unless you know where to look.
• To traditionalists, particularly in the United States, the car is a motif for an entire way of life, and the smartphone just an accessory. To early adopters who have integrated ride-sharing deeply into their lives, the smartphone is the lifestyle motif, and the care is the accessory. To generations of Americans, owning a car represented freedom. To the next generation, not owning a car will represent freedom.
• Every aspect of the global industrial social order is being transformed by the impact of software.
• Partly as a consequence of how rarely soft, world-eating technologies erupt into human life, we have been systematically underestimating the magnitude of the forces being unleashed by software. While is might seem like software is constantly in the news, what we have already seen is dwarfed by what still remains unseen.
• The opportunities presented by software are expanding, and the risks of being caught on the wrong side of the transformation are dramatically increasing.
• Software-fueled victories in the past decade have tended to be overwhelming and irreversible faits accompli.
• We tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run.
• Technological change unfolds exponentially, like compound interest, and we humans seem wired to think about exponential phenomena in flawed ways.
• Change that often looks trivial or banal on the surface, but turns out to have been profound once the dust settles.
• We have shifted gears from what economic historian Carlota Perez called the installation phase of the software revolution, focused on basic infrastructure such as operating systems and networking protocols, to a deployment phase focused on consumer application such as social networks, ride-sharing and eBooks.
• In her landmark study of the history of technology, Perez demonstrates that the shift from installation to deployment phase for every major technology is marked by a chaotic transitional phase of wars, financial scandals and deep anxieties about civilization collapse. One consequence of the chaos is that attention is absorbed by transient crises in economic, political and military affairs, and the apocalyptic fears and Utopian dreams they provoke. As a result, momentous but quiet change passes unnoticed.
• A great deal of the impact of software today appears in a disguised form.
• The fourth reason we underestimate software, however, is a unique one: it is a revolution that is being led, in large measure, by brash young kinds rather than sober adults.
• This is perhaps the single most important thing to understand about the revolution that we have labeled software eating the world: it is being led by young people, and proceeding largely without adult supervision.
• But unlike most periods in history, young people today do not have to either "wait their turn" or directly confront a social order that is systematically stacked against them. Operating in the margins by a hacker ethos--a problem solving sensibility based on rapid trail-and-error and creative improvisation--they are able to use software leverage and loose digital forms of organization to create new economic, social and political wealth. In the process, young people are indirectly disrupting politics and economics and creating a new parallel social order. In stead of vying for control of venerable institutions that have already weathered several generational wars, young people are creating new institutions based on the new software and new wealth.
• Chris Dixon captured this guerrilla pattern of the ongoing shift in political power with a succinct observation: what the smartest people do on the weekend is what everyone else will do during the week in ten years.
• Software-driven transformations directly disrupt the middle-class life script, upon which the entire industrial social order is based.
• In its typical aspirational form, the traditional script is based on 12 years of regimented industrial schooling, and additional 4 years devoted to economic specialization, lifetime employment with predictable seniority-based promotions, and middle-class lifestyles.
• Instead of software, the traditional script runs on what we might call paper-ware: bureaucratic processes constructed from the older soft technologies of writing and money. Instead of the hacker ethos of flexible and creative improvisation, it is based on the credentialist ethos of degrees, certifications, licenses and regulations. Instead of being based on achieving financial autonomy early, it is based on taking on significant dept (for college and home ownership) early.
• One way to understand the shift from credentialist to hacker modes of social organization, via young people acquiring technological leverage, is through the mythological tale of Prometheus stealing fire from the heavens form human use.
• Technologies capable of eating the world typically have a Promethean character: they emerge within a mature social order, but their true potential is unreleased by an emerging one, which gains relative power as a result.
• As a result of a Promethean technology being unleashed, younger and older face a similar dilemma: should I abandon some of my investments in the industrial social order and join the dynamic new social order, or hold on to the status quo as long as possible?
• The decision is obviously easier if you are younger, with much less to lose. If you are an adult over 30, especially one encumbered with significant family obligations or debt, the decision is harder.
• Those with a Promethean mindset and an aggressive approach to pursuing a new path can break out of the credentialist life script at any age. Those who are unwilling or unable to do so are holding on to it more tenaciously than ever.
• Young or old, those who are unable to adopt the Promethean mindset end up defaulting to what we call a pastoral mindset: one marked by yearning for lost or unattained utopias.
• Breaking smart at the level of individual sis what leads to organizations and nations breaking smart, which in turn leads to societies succeeding or failing.
• Thanks to virtuous cycles already gaining in power, I believe almost all effective responses to the problems and opportunities of the coming decades will emerge out of the hacker ethos, despite its apparent peripheral role today. The credentialist ethos of extensive planning and scripting towards deterministic futures will play a minor supporting role at best.
• The nature of problem-solving in the hacker mode, based on trial-and-error, iterative improvement, testing and adaptation (both automated and human-driven) allows us to identify four characteristic of how the future will emerge.
• Despite current pessimism about the continued global leadership of the United States, the US remains the single largest culture that embodies the pragmatic hacker ethos, nowhere more so than in Silicon Valley. The United States in general, and Silicon Valley in particular, will therefore continue to serve as the global exemplar of Promethean technology-driven change.
• The future will unfold through very small groups having very large impacts.
• One piece of wisdom in Silicon Valley today is that the core of the best software is nearly always written by teams of fewer than a dozen people, not by huge committee-driven development teams.
• The future will unfold through a gradual and continuous improvement of well-being and qualify of life across the world, not through sudden emergence of a utopian software-enabled world (or sudden collapse into a dystopian world).
• The future will unfold through rapid declines in the costs of solutions to problems, including in heavily regulated sectors historically resistant to cost-saving innovations, such as healthcare and higher education.
• In improvements wrought by software, poor and expensive solutions have generally been replaced by superior and cheaper (often free) solutions, and these substitution effects will accelerate.
• A big implication is immediately clear: the asymptotic condition represents a consumer utopia. As consumers, we will enjoy far more for far less. This means that the biggest unknown today is our future as producers, which brings us to what many view as the central question today: the future of work.
• Purist approached, which rely on alluring visions, are like precious "good" china: mostly for display, and reserved exclusively for narrow uses like formal dinners. Damage through careless use can drastically lower the value of a piece. Broken or missing pieces must be replaced for the collection to retain its full display value. To purists, mixing an matching, either with humbler everyday tableware, or with different china patterns, is a kind of sacrilege.
• The pragmatic approach on the other hand, is like unrestricted and frequent use of hardier everyday dinnerware. damage through careless play does not affect value as much. Broken pieces may still be useful, and missing pieces need not be replaced if they are not actually needed. For pragmatists, mixing and matching available resources, far from being sacrilege, is something to be encourage, especially for collaborations such as neighborhood potlucks.
• To pragmatists, the [web] browser represented important software evolving as it should: in a pluralistic way, embodying many contending ideas, through what the Internet Engineering Task Force (IETF) calls "rough consensus and running code".
• Increasingly, the pragmatic, agile approach to building things has spread to other kinds of engineering and beyond, to business and politics.
• Agile philosophies are eating all kinds of building philosophies.
• Every field eaten by software experiences a migration of the creative part from visioning activities to hands-on activities, disrupting the social structure of all professions.
• The cost of this agility is a seemingly anarchic pattern of progress.
• Adopting agile models leads individuals an organizations to gradually increase their tolerance for anxiety in the face of apparent chaos. As a result, agile models can get more agile over time.
• Once a field become hacker-friendly, software beings to eat it.
• The IETF slogan of rough consensus and running code (RCRC) has emerged as the only workable doctrine for both technological development and associated economic models under these conditions.
• Software possesses an extremely strange property: it is possible to create high-value software products with effectively zero capital outlay. As Mozilla engineer Sam Penrose put it, software programming is labor that creates capital.
• Software though, is a medium that not only can, but must be approached with an abundance mindset. Without a level of extensive trail-and-error and apparent waste that would bankrupt both traditional engineering and art, good software does not take shape.
• Purist visions tend to arise when authoritarian architects attempt to concentrate and use scarce resources optimally, in ways they often sincerely believe is best for all. By contrast, tinkering is focused on steady progress rather than optimal end-states that realize a totalizing vision. It is usually driven by individual interests and not obsessively concerned with grand and paternalistic "best for all" objectives.
• Devoting skills and resources to playful tinkering still seems "wrong", when there are obvious and serious problems desperately waiting for skilled attention.
• Software engineers must unlearn habits born of scarcity before they can be productive in their medium.
• The principle of rough consensus and running code is perhaps the essence of the abundance mindset in software.
• Consensus in traditional organizations tends to be brokered by harmony-seeking individuals  attuned to the needs of others, sensitive to constraints, and good at creating "alignment" among competing autocrats.
• By contrast, software development favors individuals with an autocratic streak, driven by an uncompromising sense of how things ought to be designed and built which at first blush appears to contradict the idea of consensus.
• Conflicts are not sorted out through compromises that leave everybody unhappy. Instead they are sorted out through the principle futurist Bob Sutton identified as critical for navigating uncertainty: strong views, weakly held.
• Rough consensus favors people who, in traditional organizations, would be considered disruptive and stubborn: these are exactly the people prone to "breaking smart".
• In its most powerful form, rough consensus is about finding the most fertile directions in which to proceed rather than uncovering constraints.
• In a process reminiscent of the "rule of agreement" in improv theater, ideas that unleash the strongest flood of follow-on builds tend to be recognized as the most fertile and adopted as the consensus.
• Leaving more decisions for the future also leads to devolving authority to those who come later.
• It is generally smarter to assume that problems that seem difficult and important today might become trivial or be rendered moot in the future.
• Traditional processes of consensus-seeking drive towards clarity tin long-term visions but are usually fuzzy on immediate next steps. By contrast, rough consensus in software deliberately seeks ambiguity in long-term outcomes and extreme clarity in immediate next steps.
• At an ethical level, rough consensus is deeply anti-authoritarian, since it avoids constraining the freedoms of future stakeholders simply to allay present anxieties.
• In other words, true north in software is often the direction that combines ambiguity and evidence of fertility in the most alluring way: the direction of maximal interestingness.
• A pivot allows the direction of development to change rapidly, without a detailed long-term plan. It is enough to figure out experimental next steps. This ability to reorient and adopt new mental models quickly (what military strategists call a fast transient) is at the heart of agility.
• In software, waterfall processes fail in predictable ways, like classic Greek tragedies. Agile processes on the other hand, can lead to snowballing serendipity, getting luckier and luckier, and succeeding in unexpected ways. The reason is simple: waterfall plans constrain the freedom of future participants, leading them to resent and rebel against the grand plan in predictable ways. By contrast, agile models empower future participants in a project, catalyzing creativity and unpredictable new value.
• The best products use perpetual beta as a way to lead their users towards richer, more empowered behaviors, instead of following them through customer-driven processes. Backward compatibility is limited to situations of pragmatic need, rather than being treated as a religious imperative.
• Most software failures do not have life-threatening consequences. As a result, it is usually faster and cheaper to learn from failure than to attempt to anticipate and accommodate it via detailed planning (which is why the RERO principle is often restated in terms of failure as fail fast).
• Perhaps the most counter-intuitive consequence of the RERO principle is this: where engineers in other disciplines attempt to minimize the number of releases, software engineers today strive to maximize the frequency of releases.
• The only way for execution to track the changing direction of the rough consensus as it pivots is to increase the frequency of releases.
• If creating great software takes very little capital, copying great software takes even less.
• Where democratic processes would lead to gridlock and stalled development, conflicts under rough consensus and running code and release early, release often leads to competing, divergent paths of development that explore many possible worlds in parallel.
• Working code that prioritizes visible simplicity, catalyzing effective collaboration and rapid experimentation, tends to spread rapidly and unpredictably. Overwrought code that prioritizes authoritarian, purist concerns such as formal correctness, consistency, and completeness tends to die out.
• In the real world, teams form through self-selection around great code written by one or two linchpin programmers rather than contest challenges.
• While the precise size of an optimal team is debatable, Jeff Bezos' two-pizza rule suggests that the number is no more than about a dozen.
• In stark contrast to the quality code developed by "worse is better" processes, software developed by teams of anonymous, interchangeable programmers, with bureaucratic top-down staffing, tends to be of terrible quality.
• Today, its underlying concepts like rough consensus, pivot, fast failure, perpetual beta, promiscuous forking, opt-in, and worse is better are carrying over to domains beyond software and regions beyond Silicon Valley. Wherever they spread, limiting authoritarian visions and purist ideologies retreat.
• Internally, the software-eaten economy is even more driven by disruption: the time it takes for a disruptor to become a disruptee has been radically shrinking in the last decade--and startups today are highly aware of that risk. That awareness helps explain the raw aggressiveness that they exhibit.
• The Promethean force of technology is today, and always has been, the force that has rescued humanity from its worst problems just when it seemed impossible to avert civilizational collapse.
• A basic divide in the world of technology is between those who believe humans are capable of significant change, and those who believe they are not. Prometheanism is the philosophy of technology that follows from the idea that humans can, do, and should change. Pastoralism, on the other hand is the philosophy that change is profane. The tension between these two philosophies leads to a technology diffusion process characterized by a colloquial phrase popular in the startup world: first they ignore you, then they laugh at you, then they fight you, then you win.
• Science fiction writer Douglas Adams reduced the phenomena to a set of three sardonic rules from the point of view of users of technology:
• Anything that is in the world when you're born is normal and ordinary and is just a natural part of the way the world work.s
• Anything that's invented between when you're fifteen and thrifty-five is new and exciting and revolutionary and you can probably get a career in it.
• Anything invented after you're thirty-five is against the natural order of things.
• Technological evolution is path-dependent in the short term, but not in the long term.
• New technologies change our sense of proportions.
• The path-dependent phase of evolution of a technology can take centuries. But once it enters a collective invention phase, surplus and spillover effects gather momentum and further evolution becomes simultaneously unpredictable and inevitable. Once the inevitability is recognized, it is possible to bet on follow-on ideas without waiting for details to become clear.
• We get attached to pastorals because they offer a present condition of certainty and stability and a utopian future promise of absolutely perfected certainty and stability. Arrival at the utopia seems like a well-deserved reward for hard-won Promethean victories.
• When pastoral fantasies start to collapse under the weight of their own internal contradictions, long-repressed energies are unleashed. The result is a societal condition marked by widespread lifestyle experimentation based on previously repressed values.
• Because they serve as stewards of dominant pastoral visions, cultural elites are most prone to viewing unexpected developments as degeneracy.
• Viewed through any given pastoral lens, any unplanned development is more likely to subtract rather than add value.
• When more people speak a language or accept a currency, the potential of that language or currency increases in a non-zero-sum way. Shared languages and currencies allow more people to harmoniously co-exist, despite conflicting values, by allowing disputes to be settled through words or trade rather than violence.
• Even if one inventor chooses not to pursue a possibility, chances are, others will. As a result, all pastoralist forms of resistance are eventually overwhelmed.
• Prometheans who discover high-leverage unexpected possibilities enter a zone of serendipity. The universe seems to conspire to magnify their agency to superhuman levels. Pastoralists who reject change altogether as profanity turn lack of agency into a self-fulfilling prophecy, and enter a zone of zemblanity. The universe seems to conspire to diminish whatever agency they do have, resulting in the perception that technology diminishes agency.
• Power, unlike capability, is zero-sum, since it is defined in terms of control over other human beings.
• The broader lesson of the principle of generative pluralism is this: through technology, societies become intellectually capable of handling progressively more complex value-based conflicts. As societies gradually awaken to resolution mechanisms that do not require authoritarian control over the lives of others, they gradually substitute intelligence and information for power and coercion.
• Perhaps most important part of the change is that we are experiencing a systematic substitution of intelligence for brute authoritarian power in problem solving, allow a condition of vastly increased pluralism to emerge.
• Much of our collective sense of looming chaos and paradises being lost is in fact a clear and unambiguous sign of positive change in the world.
• Innovation can in fact be defined as ongoing moral progress achieved by driving directly towards the regimes of greatest moral ambiguity, where our collective demons lurk.
• Genuine progress feels like onrushing obscenity and profanity, and also requires new technological capabilities to drive it.
• We see the world through a rear-view mirror. We march backwards into the future.
• Our aesthetic and moral sensibilities are oriented by default towards romanticized memories of paradises lost. Indeed, this is the only way we can enter the future.
• The paradox of progress is that what seems like the path forward is in fact the reactionary path of retreat. What seems like the direction of decline is in fact the path forward.
• The politician's syllogism: Something must be done. This is something. This must be done.
• We may not be satisfied with the answers we find to timeless questions, but simply by asking the questions and attempting to answer them, we are bootstrapping our way to a more advanced society.
• The story of software eating the world is also the story of networks eating geography.
• Crashing storage costs and continuously upgraded data-center hardware allows corporations to indefinitely save all the data they generate. This turning out to be cheaper than deciding what to do with it in real time, resulting in the Big Data approach to business.
• Most code today, unlike fifty years ago, is in hardware-independent high-level programming languages rather than hardware-specific machine code.
• What we are living through today is a hardware and software upgrade for all of civilization.
• And of all the ways we are adapting, the single most important one is the adaption in our problem-solving behaviors.
• Acquiring resources means engaging in zero-sum competition to bring them into your boundary, as captive resources.
• In the last century, the most common outcome of goal-directed problem solving in complex cases has been failure.
• Goal-driven problem solving:
• Problem selection: Choose a clear and important problem.
• Resourcing: Capture resources by promising to solve it.
• Solution: Solve the problem within promised constraints.
• Tinkering-driven problem solving:
• Immersion in relevant streams of ideas, people and free capabilities.
• Experimentation to uncover new possibilities through trial and error.
• Leverage to double down on whatever works unexpectedly well.
• Tinkering is a process of serendipity-seeking that does not just tolerate uncertainty and ambiguity, it requires it. When conditions for it are right, the result is a snowballing effect where pleasant surprises lead to more pleasant surprises.
• From the inside, serendipitous problem solving feels like the most natural thing in the world. From the perspective of goal-driven problem solvers, however, it can look indistinguishable from waste and immoral priorities.
• Gall's Law: A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched to make it work. You have to start over with a working simple system.
• The idea that a new, simpler system can revitalize a complex system in a state of terminal crises is the essence of Promethean thinking.
• Free people and ideas can associate in arbitrary ways, creating interesting new combinations and exploring open-ended possibilities. They can make up their own minds about whether problems declared urgent by authoritarian leaders are actually the right focus for their talents. Free ideas are even more powerful, since unlike the talents of free individuals, they are not restricted to one use at a time.
• Free people and free ideas formed the "working simple system" that drove two centuries of disruptive industrial age innovation.
• Tinkering--the steady operation of this working simple system--is a much more subversive force than we usually recognize, since it poses an implicit challenge to authoritarian priorities.
• Tinkering is becoming much more than a minority activity pursued by the lucky few with access to well-stocked garages and junkyards. It is becoming the driver of a global mass flourishing.
• A stream is simply a life context formed by all the information flowing towards you via a set of trusted connections--to free people, ideas, and resources--from multiple networks.
• If the three most desirable things in a world defined by organizations are location, location, and location, in the networked world they are connections, connections, and connections.
• What makes streams ideal contexts for open-ended innovation through tinkering is that they constantly present interrelated people, ideas and resources in unexpected juxtapositions. This happens because streams emerge as the intersection of multiple networks.
• As a result of such unexpected juxtapositions, you might "solve" problems you didn't realize existed and do things that nobody realized were worth doing.
• People, ideas and things can have multiple, fluid meanings depending on what else appears in juxtaposition with them. Creative possibilities rapidly multiply, with every new network feeding into the stream.
• The most interesting place to be is usually the very edge, rather than the innermost sanctums.
• What we do not understand as instinctively is that streams are problem-solving and wealth-creation engines. We view streams as zones of play and entertainment, through the lens of the geographic-dualist assumption that play cannot also be work.
•  The first sustainable socioeconomic order of the networked world is just beginning to emerge, and the experience of being part of a system that is growing smarter at an exponential rate is deeply unsettling to pastoralists and immensely exciting to Prometheans.
• For Prometheans, the challenge is to explore how to navigate and live in this world. A growing non-geographic-dualist understanding of it is leading to a network culture view of the human condition. If the networked world is a planet-sized distributed computer, network culture is its operating system.