While much has been written about entrepreneurship in business and economics publications, much less has been written about entrepreneurial failure. Some writers in the popular press have interesting things to say, such as Megan McArdle’s The Up Side of Down: Why Failing Well Is the Key to Success. She finds that failing early on and understanding the cause of failure provides a powerful learning process that can lead to better opportunities in the future. Economists have also written numerous pieces about failure including F. A. Hayek, Israel Kirzner, Joseph Schumpeter, and, more recently, Russell Sobel in his Concise Encyclopedia of Economics entry.

Failure removes resources from poorly performing enterprises so they can be put into more useful ones. It is often a necessary precursor to the very things we view as successes. This self-healing process that constantly moves under-valued means to more preferred ends is a prime motivator of economic growth.

Still, there has been very little discussion about how entrepreneurs themselves feel about failure. During my career teaching economics of entrepreneurship in Silicon Valley, I have had the opportunity to question dozens of successful entrepreneurs. Asked how they became successful entrepreneurs, most initially said that they were lucky. But I would always probe further. Everyone has luck, after all. The more interesting question for economists is how entrepreneurs handle luck—how they anticipate and respond to unforeseen changes. When questioned more deeply, nearly all the entrepreneurs also admitted to having failed, often multiple times, before finding success. Sharing their stories of failure, rather than success, provides insight into how the real process of entrepreneurship functions, both at the individual and macro levels. At the individual level, failure spurs the learning process that is useful for an entrepreneur’s future endeavors. At the macro level, failure can lead to new products, processes, and knowledge that drive economic evolution. Below are two cases that illuminate the learning process and the creation of new knowledge at the heart of economic progress.

While nearly all speakers admitted failure, there was one who did not. He is an informative case. We will call him Ralph. I have known Ralph for over thirty years. We were partners in several multi-tenant industrial property developments starting in the 1970s. Ralph, as the managing partner, developed good relations with his tenants over the years and maintained his properties in excellent condition. However, at the height of the recession in 1982, many of the tenants were struggling to stay in business and some failed. The partnership was unable to make its loan payments and, eventually, had to give a couple of properties back to the lender. We were not alone, as many owners found themselves in a similar position in Silicon Valley. The partnership lost the remaining equity, though over the years it had had several distributions and the benefit of some tax shelter. The partners still considered it something of a failure. So, when I asked Ralph to talk about his failures, I was surprised when he seemed at a loss. He finally said he could not think of any. So, I brought up those properties in Santa Clara where we turned the keys back to the lender. He said, “Oh, those weren’t a failure. We had a good strategy. It is just that conditions changed so unpredictably that they could not have planned for. Our best business decision, given the circumstances, was to give the properties back.”

There are several interesting elements here. First, Ralph’s conception of failure meant making a bad business decision in real time. Though the partners lost some money, he believed that he made the most reasonable decision given the circumstances. He had great faith in his ability to see the best outcome. As David Harper discusses in his book The Foundations of Entrepreneurship and Economic Development, Ralph demonstrated high economic self-efficacy, the sense that he had the right tools to succeed in the field of development and management. Also, failure was not a reflection on him. If things radically changed that were out of his control, blaming himself served no value. He accepted uncertainty, but still maintained the general sense that he could respond effectively using that which he could control—what entrepreneurship scholars call a strong locus of control over the environment in which he was working.

There was an additional entrepreneurial recognition here. If properties were going back to lenders, Ralph asked himself how he could use this challenge as an opportunity. As he looked at his situation and that of others around him, he saw that he might use his local knowledge and reputational trust to create an option that others had not noticed. Many lenders took back property during this period. Most of those lenders were not local to the area and had little knowledge or experience leasing multi-tenant industrial space. Ralph realized that he could leverage his experience and good relations with local businesses to give him a head start when the business cycle improved. He began to reach out to lenders of other projects pointing out that, even if they foreclosed, they still had to fill their vacant spaces. He understood that many tenants who left under difficult economic conditions would be back when things improved. They would gravitate to the brokers and owners who served them well in the past and the properties those owners controlled would be the first to fill again. He set up meetings with his preferred lenders and created a set of workouts. These would allow current owners to extend their loans and keep the properties by renegotiating the terms, often more stringent, but also giving an equity position to the lender in exchange for the renegotiation so that the lender could share in the upside when conditions improved.

Lenders were initially resistant. Foreclosing demonstrated to other borrowers the lenders’ serious willingness to take on the expense and risk foreclosure entailed. However, as more buildings fell into foreclosure, some lenders began to realize that actually getting tenants was the ultimate solution for any building owner. That required local, subjective knowledge and reputation. As new owners with no connection to the properties’ rental landscape, lenders would require a local broker with contacts. But those brokers were often involved in the very properties that changed hands and were unlikely to use their best efforts. Ralph was able to demonstrate to several lenders that individual workouts with favorable equity positions would incentivize both borrowers and lenders to refill the properties as quickly and efficiently as possible. This proved so successful that Ralph spent the next couple of years working with owners and lenders as a consultant to create new loans on distressed properties where both sides benefited.

An important takeaway from this is that successful entrepreneurs often face what others would deem as failure. However, they view these situations, not as outcomes, but as challenges and opportunities to create something new. All the entrepreneurs I met have experienced these. Not all view them in the same way. Many are very honest about what they perceive as their own shortcomings and business blunders. However, they also viewed their experiences as learning opportunities. As one told me, there is no disgrace in making a mistake; it is only foolish if one makes the same mistake twice. It is one of the strengths of Silicon Valley that venture capitalists see this hard learning as a positive and many feel more comfortable supporting entrepreneurs who have had a “failure” as long as that person understands why they failed.

Certainly, as individuals, entrepreneurs and innovators face difficult barriers. Forbes estimates that 1 in 5 small businesses fail in the first year and 50% in 5 years. Startups are prone to numerous shortcomings including lack of advertising, products that do not entice customers, lack of adequate capital, pursuit of a single product with a short lifespan, and many more challenges. Some commentators such as the editorial staff at The Economist in “Entrepreneurs Anonymous” have complained that the high rate of failure among new entrepreneurial ventures leaves behind many broken would-be innovators. Jill Lepore in “The Disruption Machine” likens start-ups to a pack of ravenous hyenas that are “ruthless and leaderless and unrestrained” and “devastatingly dangerous.” For those who choose to pursue an innovative venture, the road can be treacherous. Even when successful, it has been estimated that entrepreneurs often earn no more and many less than had they pursued a corporate position in an established company. This reality invites a journey through another entrepreneur’s experience that did not work out as well as Ralph’s but still demonstrates an interesting facet of failure and success.

Susan was the product manager for a large software company, call them Apex. As a product manager, she got early notification of all her product glitches. After the introduction of a new software package, she received a growing number of complaints about a particular bug in the package. She had her team create a workaround, but this solution had customers jumping through several hoops. On her own, Susan investigated the problem and came up with a possible simplified and integrated solution, as well as a reasonable budget and schedule for implementation. She took it to her vice president and explained how the flaw could be fixed. He asked if customers were complaining about the work-around and she indicated that they were not. He killed the improvement saying that the potential improvement was not justified if customers were making do. In his view, the cost of the solution was too great given any potential increased revenue it might generate. Susan did not agree.

Rather than fight through the organization, and since she had already done all the foundational research, she decided to create the improvement on her own. Working outside of normal hours, she hired programmers familiar with the software application and developed an efficient solution that would work with her company’s existing software as an add-on. Once her modification was beta-tested, she quit Apex, formed her own company, and began offering the new package to companies that used the Apex suite. Those who tried the new add-on were pleased and word spread about its availability. Soon, Susan was hiring more people, renting space, and setting up an office. She created a management structure that included marketing, human resources, and quality assurance and control divisions. Her company grew to over one hundred employees and most of her time was consumed by internal management.

However, Apex noticed how many companies were using her software add-on and the revenue she was generating. As her company and revenue grew, Apex became more interested in solving the problem internally. With the substantial resources at its command, Apex reworked its package, removed the faulty element, and made Susan’s rival product unusable in the new software suite. Apex’s customers were encouraged to upgrade and, as they did so, Susan was so involved with her company internally, that she was slow to look outward at this competition and anticipate its ramifications. Susan’s revenue began to decline as customers moved to the new software. She cut costs but was reluctant to cut staff. As the yearend approached, revenue continued to drop and cash reserves dwindled. Susan borrowed where she could to pay bills and keep serving her remaining customers. By Christmas, the handwriting was on the wall, but she resisted laying off the rest of her people before the holiday. She was able to keep critical staff until the new year but was forced to declare bankruptcy immediately after the holiday, letting everyone go.

Reflecting on this, Susan said the hardest part of this process was telling her husband that she had borrowed all the money in their 401(k) to keep the company going as long as she did. The saving grace for her was that her husband forgave her and her marriage survived. Susan has since returned to corporate life and she is the CIO for an innovative, growing company. She learned several lessons for those who wish to start their own business. First, have a clear metric about when enough is enough. She strongly subscribes to McArdle’s advice of “failing well.” Second, the entrepreneur should always spend as much effort looking outward as inward at their company. When things do not go right the temptation is to concentrate on costs and lose sight of product development. Finally, every product has a life cycle. If one gets a product up and running successfully, it is time to create a new product. While Susan no longer has her own company, as a senior executive of a successful company, she has stressed these three insights with her executive team and has helped add value to her current company.

Economists have long recognized that entrepreneurial success leads to the reallocation of inefficient resources to more efficient uses, and that often these more successful ones are ones not recognized until they have been created through someone’s recognition of a profit opportunity. However, the above experiences expand our understanding of the entrepreneurial process by demonstrating how failure was critical in the above. This is true at both the individual level and the macro level.

At an individual level entrepreneurs are subject to a market process that has no mercy. Competition is always there, in one form or another. Every minute of every day, someone is trying to take the entrepreneur’s best customers from them. This can be harsh. But just because it is harsh, does not mean that it is not worthwhile. Even though failure is difficult, there is the benefit of hard lessons learned and the new knowledge that comes with them. At the macro level, one should not fall prey to the fallacy of composition. While some (most) entrepreneurs fail, that does not mean the system fails. Just the opposite is true. Failure is essential to the self-healing of the economic system. In the cases above, failure did cause resources to be reallocated to more productive uses. Buildings got revalued based on their reduced income streams, releasing value for other uses until the real estate market recovered. New models like owner-lender cooperative relationships evolved and moved markets forward. New products arose, like a new suite of software applications that better served customers. The software improvement in this case was driven by failure, not success.

That is only part of the story. More importantly, these entrepreneurs learned from their failures and brought their new knowledge with them to their next venture spilling the knowledge over to those around them in a virtuous circle. As in the cases above, the new knowledge was individual, subjective, and creative. Using their tacit alertness, they not only brought the knowledge of better products and processes but also products and processes no one else had imagined. It is this learning process that produces progress, better fulfilling unmet, and often unknown, wants with the limited resources we possess. They demonstrated that failure precedes success. It engenders a learning process that creates the new knowledge that is the lifeblood of economic growth.