All government regulations face a persistent problem. By their very nature they are trying to keep people from acting in their own self-interest. Either they try to keep people from doing what they want to do, or force them to do something they don’t want to do.

How the private sector responds to regulation

In a simple society in which people are not very creative, you might be able to choose a set of rules and regulations that work reasonably well for long periods of time. However, in a complex system populated by people who are highly entrepreneurial, it won’t be long before you get unanticipated private sector activity that partially undermines the original goal of the regulation. As time passes there will predictably be even more undermining. If the regulation never changes, eventually the private sector may innovate around it completely.

In the face of highly inefficient, bureaucratic taxi regulations, for example, eventually entrepreneurs circumvented the whole system and gave us Uber. In the face of bureaucratic hotel and occupancy regulations, entrepreneurs gave us Airbnb. In the face of an inefficient and bureaucratic Postal Service, we got FedEx and then email.

Health care regulation

The same thing happens in health care. The average new patient waits 26 days to see a doctor in the U.S. because demand exceeds supply and health delivery is tightly regulated. MinuteClinic found a way around those barriers and today there are about 2,000 walk-in clinics (staffed by nurses) in the U.S. Along the way, the innovators had to fight regulators and lawsuits filed by organized medicine. Rx.com began competing with local pharmacies online, thereby circumventing the bureaucratic triad of drug manufacturers, pharmacies and pharmacy benefit managers (PBMs). Mark Cuban is doing much the same things with Cost Plus Drugs.

In a typical hospital operation, the patient expects a health benefit, and the doctor gets income from the surgery. Insurance companies, by contrast, have an economic interest in avoiding the cost of unnecessary surgery. The resolution of these conflicts can be time-consuming and costly. Medici, a health management firm in Austin, Texas, found a better solution. It offers patients a second opinion from a list of specialists at top academic medical centers around the country. The second opinion causes the patients to avert surgery about 30 percent of the time.

Another way insurers (including Medicare and Medicaid) hold down costs is by insisting on preauthorization for certain procedures and denying the requests if there is any doubt about their appropriateness. The patient and doctor can appeal the denial, but that take many hours of the doctor’s time and requires citing studies in medical journals to back up the request.

Nearly half of doctors surveyed by the A.M.A. said that when they didn’t appeal a claim denial, it was at least in part because they didn’t have the time or resources for the lengthy appeals process. However, some physicians have found a new tool: an artificial intelligence (A.I.) program that completes the appeal for them. An Oklahoma doctor in obesity medicine said A.I. enabled his small practice to go from almost never appealing insurance denials to sending 10 to 20 appeals per week.

Under fee-for-service medicine, doctors have an economic interest in only performing those tasks for which there is a fee attached. In general, they get no money for keeping a diabetic patient healthy. They actually gain income if their patient’s condition worsens, because there will be more things they get paid to do.

One way to circumvent these perverse incentives is with a concierge doctor—one who is paid by the patient outside of the third-party payment system. Originally designed for the rich, this kind of service has become affordable for ordinary folks. Direct primary care (DPC) doctors provide all primary care to their patients 24/7 for very reasonable monthly fees. Along the way, DPC doctors have had to survive hostile regulators and menacing lawsuits.

Teledoc, which has been operating for more than two decades, is making telephone consultations available to more than 90 million people—worldwide. It was not easy. The Texas Medical Association went to court and tried to put the company out of business. Teledoc survived that challenge, but still must contend with another costly regulation: when a traveling patient calls to get a prescription filled, the doctor answering the call must be in the same state that the patient is calling from.

To recap, because entrepreneurs have circumvented burdensome regulations, people can buy low-cost drugs online; they can get same-day primary care from competent nurses in nearby drugstores; they can get prescriptions filled with a phone call to a doctor they have not previously met; and they can get 24/7 consultations from a DPC doctor and avoid going to the emergency room. Although most patients cannot get treated by doctors in other states, some patients can get second opinions from doctors in any state; and some patients are overcoming the costly obstacles created by preauthorization requirements with assistance from A.I.

Perverse incentives

All that said, there is one thing that makes health care different from a garden-variety regulated industry: The main actors in the system (patients, providers and insurers) all face perverse incentives. So a lot of entrepreneurial activity ends up making costs higher and quality lower for the system as a whole.

Patients have an interest in over-consuming, since someone else is paying the bill. If doctors get paid fee-for-service, their interest is in overproviding—since the more they do, the more they get paid. If doctors are paid a fixed sum, they have an interest in providing less care. If insurance premiums are unrelated to expected medical care, insurers have an interest in underproviding to the sick and maybe even overproviding to the healthy.

Of course, these divergent interests are well known. That is why the system is so highly regulated. In fact, almost every health care regulation is designed to thwart these very types of self-interested behavior in one way or another. However, no one in the system, including the regulators themselves, has a self-interest in making sure the system works the way almost everyone thinks it should work. So, regulations often make things worse.

Lessons

1. In all industries, regulations designed to achieve a social purpose will provoke private sector reactions that undermine the purpose of the regulation, and in time entrepreneurs may find ways of circumventing the regulations altogether.

2. For regulation of an industry to have a reasonable hope of success, it must be constantly updated to maintain its social purpose in response to private sector actors who find ways to undermine it and circumvent it.

3. One reason we have deregulated so many industries (airlines, trucking, banking, oil and gas, etc.) is because even imperfect markets often achieve social goals better than trying to achieve them with imperfect regulation. (There is no invisible hand at work in the government sector.)

4. The way we deliver health care in this country leaves all the principal actors with perverse incentives, and much health care regulation is designed to keep people from acting on those incentives.

5. Although entrepreneurial behavior designed to circumvent health care regulations has often made patients better off, it can also leave them worse off.

I will expand on these lessons in a subsequent column.