Advanced Computing, Part 1: Markets and Skill Sets

For years, the major advance of corporate computing has been the dissemination of OO programming or ERD and relational databases. When it came to advanced computing, firms would turn to SAS to do a little statistical analysis. Data warehouses came on board with means to slice and dice data for decision making, yet their cost often out-weighed their value. It doesn’t take a rocket scientist to realize that we had come to a kind of standstill or market maturity phase.

The PhDs are finally breaking out of the research corridors and entering business. When advanced computing techniques first entered business, around 92- 95, they entered in the form of artificial intelligence, neural nets, and genetic algorithms. Initially, these techniques found their place in futile attempts to predict the stock market. As a result of the failure of these techniques to achieve above average returns in the stock market, many business leaders dismissed their credibility and went back to industry accepted OOD and ERD.

These days however are bringing a rebirth to advanced computing techniques as witnessed in the Tedesco questions.

One of the techniques generating a new found interest is matrix algebra, also known as linear algebra. Coupling linear algebra with basic statistics, SmartSignal has been able to create a tool that is able to predict system failures within complex systems. Financially, their product has been able to reap benefits in managing power plants and turbines, but
there are many more applications where the tool could be deployed, such as manufacturing. Both linear algebra and covariant statistics are taught in good mathematics undergraduate programs. Moreover, anyone who has studied undergraduate quantum mechanics in physics or chemistry has a firm footing in both of these areas.

A second technique that has regenerated some interest is optimal control. I first came across optimal control theory in David Tannor’s and Stuart Rice’s work on controlling molecular dynamics dating back to 1980s. Today, Adica Consulting, in conjunction with researchers from Argonne National Laboratories, are using optimal control with agent based systems to optimize power trading rules in the energy businesses. Another area where optimal control theory is applying itself to business is along with Monte Carlo simulations in exploring the phase-space of the economic variables of a business to determine which path and set of criteria produce the highest return on investment.

A third technique that is getting major press is the use of agent based systems for managing complex problems. Even Harvard Business Review recently posted an article about how Southwest Airlines saved money and improved service by using an agent based model to redesign their cargo shipping process. Agent based models rely upon both artificial intelligence and statistics to generate realistic models of behavior of complex systems. Although I had to do independent research to learn about agent based theory, I suspect that the pool of likely candidates to perform this kind of work is sufficiently large to warrant the expectation of finding satisfactory employees with an appropriate skill set.

The difficulty with exploiting these new techniques is that the skill set of computer scientists or database architects do not neatly match the skill set required to reap the benefits of the advanced computing techniques. The individuals that understand advanced computing techniques are available, but their background is likely to be highly academic and low in actual job experience. While many business people consider multivariate regression analysis to be hard, over 1000 physics PhDs, who graduate each year with pathetic job prospects, consider statistics to be child’s play. If we throw in the Chemists and Mathematicians, we will quickly find a pool of 5000 PhDs with experience, at least tangential, in these areas. Yet hiring them would require managers to forego requiring 10 years of job experience for a 30 something year old and accept that education and post-doctoral research might actually be worth more than a stint at Procter and Gamble or Accenture. Hence, if a firm wanted to capture the next wave of computing, it may have to make a risky investment in acquiring a new skill set, not to mention marketing it.

The May Report, TECH BUSINESS BRIEFS, May 22, 2002