What?! The problem wasn’t that the financial models were allowed to “fly solo”. The problem was that the models kept getting changed to appease the developers’ bosses. The SciAm editors also try to make it sound like the developers should have just not made the changes to the models each time “overoptimistic assumptions and faulty data” was given. What should have they done? Just said, “I’m sorry, Dave. I’m afraid I can’t do that.” They would have been fired and then replaced. In some cases maybe they could make a stand and say that making the change is risky and shouldn’t be done without being fired. However, it would amount to the same efforts that Roger Boisjoly made on January 27, 1986, when he convinced his manager, McDonald, to warn the NASA managers not to launch the space shuttle, Challenger, until warmer weather. The push back from the bosses was such that another person was found to sign off on the launch.
I am getting a little off topic. Let us return to this financial airplane that we don’t want to allow to fly solo. Many disasters have been caused by people doing their best to “fix” the problem instead of allowing the established protocols (usually designed using mathematical models) to deal with the problem. I’ll give two examples: Three Mile Island and the incident aboard the drill rig Ocean Ranger in February 1982.
In short, it is wrong to blame the people who work on the product because they are not the people with the power to decide what work is done on the product regardless if the product is a car or a financial model.