The Advanced Technology Program (ATP) is a com-petitive cost-sharing program in which the Federal Government works in partnership with industry to foster the development and broad dissemination of challeng-ing, high-risk technologies that offer the potential for significant, broad-based economic benefits for the nation. This unique government-industry research part-nership fosters the acceleration not only of dramatic gains in existing industries, but also of the development of emerging technologies leading to revolutionary new products, industrial processes and services for the world's markets while working to spawn industries of the 21st century. ATP's focus on civilian technologies offers the potential for substantial increases in produc-tivity and competitiveness of firms; provides consumers with new, better, and lower-cost products and services; and increases high-wage employment in the United States.

From 1990 through 1999, the ATP made multi-year awards for a total of 468 projects, including over 157 joint ventures, involving more than 1,000 participants (not including the many subcontractors and informal partners and collaborators that participate in many of the projects). These projects entail approximately $3.0 billion of research, of which industry committed slightly more than half ($ 1.53 billion).

Since its inception, the Program's direct and indirect impacts have been substantial. In its brief existence, the ATP has helped shape the face of long-term enabling technology development undertaken by industry. It has also developed, by necessity, a wide variety of new tools for evaluating the impact of technology investment on the economy. Furthermore, it has often provided expertise at an important juncture— staff members, who are often leaders in their field, provide the critical insights to their peers necessary to assess how a certain technology has evolved and describe its potential future trajectories.

The ATP Proposal Preparation Kit [1] was the means by which the program has been introduced to the public. Its publication in 1990, as well as subsequent editions, spelled out the ATP's selection criteria and provided a manual for submitting a proposal. It was developed by the ATP specifically to help the public understand the program and addresses ATP's selection criteria. In the first year, nearly 1,000 copies of these guidelines were distributed to potential proposers; by the time the competition began for the year 2000, requests topped 50,000 copies. With such wide distribution, the kit serves many purposes: it informs the public of the nature of the program, answers questions about the program, and supplies all of the necessary information to complete an application, offering "one-stop shop-ping" for the potential proposal writer. From the outset, the Proposal Preparation Kit has undergone annual revisions to reflect both the needs of the public and the changes to the program.

The Proposal Preparation Kit has had widespread impact; to date, the rules contained within it have influ-enced the formation of almost 4,500 project teams. As such, it has changed the face of long-term technology development considered by industry. While the ATP has funded approximately 10 % of these proposals, anecdo-tal evidence suggests that the kit was influential in fostering collaboration even when proposals were not funded.

A critical part of the ATP program management has been the development of methodologies and tools for assessment of impact and broad-based economic benefits. The ATP relies on the presence of expected private returns to induce companies to plan, propose, and cost-share research with the government. If the research is successful in overcoming its technical hurdles, the ATP relies on awardees to pursue commer-cial development of the new technology with private capital. The ATP applies its criteria to the proposals it receives to identify those projects expected to accom-plish ATP's public-interest mission and objectives. Selected projects must demonstrate that they have the potential to generate social rates of return (the return to the nation) far in excess of the private rates of return on investment. In addition, selected projects must demon-strate that, without ATP funding, the private sector would be unwilling to do the research in either a timely fashion or a scale needed to realize the social benefit potential.

A study conducted by economist William F. Long [2], an ATP contractor, provides the most detailed examina-tion to date of the outcomes of the earliest ATP projects. The study covers all 38 ATP projects completed by the end of March 1997. It documents research accomplish-ments and highlights subsequent work by the partici-pants to commercialize the results. It also provides near-term outlooks for the technologies. In the high-risk envi-ronment in which the ATP-funded projects operate, fail-ure is to be expected. Thus the report also highlights the reasons for failure of 12 terminated projects that had been selected between 1991 and March 1997.

The 38 projects surveyed by Long cover a broad range of ATP investments. The technologies were distributed over seven broad areas— chemicals and chemical pro-cessing; materials; discrete manufacturing; energy and environment; biotechnology; information, computers and communication; and electronics— with the majority in electronics. Industry participants provided $65.7 million, a little more than half the funding, while the ATP contributed $64.6 million to the 38 projects.

Notable new technologies highlighted by Long include a merger of tissue-engineering and textile-weaving to help regenerate lost or damaged tissue in the body, an application of high-temperature superconduc-tors to improve cellular phone service, and a suite of process-monitoring and control technologies that are cutting costs and improving quality throughout much of the U. S. auto industry. Other new technologies devel-oped under the projects ran the gamut from a desktop bioreactor capable of growing large amounts of human stem cells isolated from bone marrow for cell replace-ment therapy, a device now in clinical trials; to a com-puter programming tool to simplify the task of writing software for parallel-processing computers, which already is in commercial use; to a new navigation system for mobile robots which is being used to guide delivery robots in hospitals.

This analysis shows that technical success has not always led to commercial success. In a few cases, finan-cial reverses or corporate takeovers left technically successful projects in abeyance. In other cases, com-mercialization activities are expected to take longer. For example, projects with medical applications generally had yet to be widely applied because they must go through a long process of clinical trials.

Other projects, however, have shown strong early returns. The process-monitoring and control technolo-gies for the auto body industry, developed by a consor-tium of large and small companies, have been adopted in more than half of the Chrysler and General Motors plants in the United States and Canada. When the tech-nologies are fully implemented, annual savings in pro-duction costs are expected to range from $65 million to $160 million for one of the most important U. S. industrial sectors.

A project pursued by one small start-up company developed a novel technology for processing very large semiconductor wafers, allowing the United States to be the first in the market with processing equipment for the next generation of 300 mm semiconductor wafers. A small New Jersey firm developed a new laser light source, which is the most powerful tunable source of laser light over much of the ultraviolet spectrum, and already has incorporated it in three new products for laser surgery and other applications.

The Long study is influential because it is among the first to demonstrate through a systematic assessment of the ATP portfolio the breath of the impact of the public investment undertaken by the ATP. In his introductory comments for the report, former Commerce Secretary William Daley characterized the report's influence by stating, "this new study fills in the details behind the statistical analyses of the ATP and demonstrates the overall success of the program. These pages say that the ATP is working. Industries as diverse as biotechnology, electronics, manufacturing and software have new tech-nologies in place, today, that they wouldn't have had here and now without the ATP. The projected benefits to the nation's economy from just three of these early ATP projects would pay for every ATP project funded to date." Daley concluded by stating, "this report is a portrait of a program that works."

Since the ATP funds the research underlying the development of enabling technologies that are expected to have benefits extending substantially beyond the direct ATP award recipient, impact evaluation remains a challenge. Even under the best of circumstances, tech-nology diffusion takes time, and tracking and measuring externalities, or spillover effects, is often complicated and difficult. As such, program and project assessment requires that the ATP go well beyond such traditional measures as return-on-investment, firm profitability, or increased tax revenues. Available evaluation tools often are insufficient to meet the task.

A special issue of the Journal of Technology Transfer [3] highlights a collection of ATP evaluation studies and illustrates the variety of evaluation issues ATP faces. The works in the publication are among the first to demonstrate the conceptual challenges ATP is faced with and the wide variety of tools developed by the ATP. That publication features articles on methodology, the evaluation program, results from the path-breaking ATP "Business Reporting System" database, survey-based research, and a macroeconomic impact study. Other papers in the volume examine the effects of collabora-tion, special concepts that need to be considered in examining social benefits, and counterpart programs in the industrialized world.

In the Journal of Technology Transfer issue [3], Adam Jaffe, of Brandeis University, reflects on the im-portance to the ATP of generating and measuring "economic spillovers." It illustrates with simple models how the benefits of ATP projects may extend beyond the direct ATP award recipients through market and knowledge spillover effects. The paper also discusses network spillovers. Jaffe recommends that the ATP fund projects for which expected social benefits are large and substantially exceed expected benefits to the awardees. He recommends that evaluation efforts include measures of spillover effects. This paper is influential in that it affected the way in which the Advanced Technology Program, as well as others, undertakes economic analysis. Jaffe's paper codified the economic concepts used by the ATP and has served as a springboard for many other research projects underway.

Rosalie Ruegg the Director of the Economic Assessment Office of the ATP, provides an overview of the ATP, explains how the program operates, how it seeks to accomplish its mission, and what it has funded to date. She identifies the major components of ATP's evaluation program and identifies research areas of particular interest. The program is internationally recognized and has made a substantial impact in the evaluation community. In the United States, Congress has directed other agencies to model their evaluation programs on the ATP. Furthermore, state governments seek assistance from ATP in establishing their evaluation programs.

Jeanne Powell from the ATP presents and analyzes data from ATP's "Business Reporting System" database to evaluate short and medium term project effects. She describes ATP's principal data collection tool and identifies a number of pathways through which ATP-funded technologies are generating impact. The enabling nature of the funded technologies is suggested by the many potential applications that have thus far been identified. The survey-based database employed in this study is unique because it is among the first in the world that allows a researcher to explore the inner-work-ings of firms and technology projects at such a magni-tude. As such, the tools used in this paper serve as the basis for future evaluation efforts.

ATP 's effects on the speed with which research is conducted and technology is commercialized are exam-ined by Frances Laidlaw. Laidlaw questions not only whether, and by how much, the ATP accelerates research, but also the value of acceleration; whether saving time during the research stage translates into saving time downstream; and whether project partici-pants realize any time-related benefits outside the walls of the project. This paper demonstrates that acceleration of technology development and commercialization is one of several ways that the ATP can affect economic outcomes of the projects it funds.

The researchers at CONSAD Research Corporation report on a detailed case analysis performed for an ATP-funded research joint venture whose focus was on new dimensional control technology for discrete manufactur-ing. They analyze the multiple impacts of the new technology applied in the automobile manufacturing sector and apply a macroeconomic model to project national economic impacts resulting from quality improvements in vehicles made by U. S. producers. This paper demonstrates the feasibility of a macroeconomic input-output model dealing with the impact of techno-logical innovations.

Other relevant issues are discussed in this series. Albert Link, a Professor of Economics at the University of North Carolina at Greensboro investigates the effects on research efficiency of collaboration in an ATP-sponsored joint venture. Also, Andrew Wang examines the special considerations that need to be undertaken in modeling the social benefits of medical technologies. Finally, Connie Chang of the ATP office, signals ATP's interest in counterpart programs that operate in most industrialized countries.

Prepared by Robert Sienkiewicz and Richard N. Spivack.

Bibliography

[1] Advanced Technology Program Proposal Preparation Kit, National Institute of Standards and Technology, Gaithersburg, MD, Novem-ber 1999.

[2] William F. Long, Advanced Technology Program Performance of Completed Projects: Status Report Number 1, NIST Special Publication 950-1, National Institute of Standards and Technology, Gaithersburg, MD (1999).

[3] Symposium on Evaluating A Public-Private Partnership: The Advanced Technology Program, Special issue of J. Technol. Transfer , Vol. 23, no. 2, Summer 1998.

Fig. 1. ATP Proposal Preparation Kit.