Uwe Naumann
Skip Abstract Section
Abstract
The availability of first derivatives of vector functions is crucial for the robustness and efficiency of a large number of numerical algorithms. An upcoming new version of the differentiation-enabled NAGWare Fortran 95 compiler is described that uses programming language extensions and a semantic code transformation known as automatic differentiation to provide Jacobians of numerical programs with machine accuracy. We describe a new user interface as well as the relevant algorithmic details. In particular, we focus on the source transformation approach that generates locally optimal gradient code for single assignments by vertex elimination in the linearized computational graph. Extensive tests show the superiority of this method over the current overloading-based approach. The robustness and convenience of the new compiler-feature is illustrated by various case studies.
- Aho, A., Sethi, R., and Ullman, J. 1986. Compilers. Principles, Techniques, and Tools. Addison-Wesley, Reading, MA. Google Scholar
- Averik, B., Carter, R., and Moré, J. 1991. The MINPACK-2 test problem collection (preliminary version). Tech. mem. ANL/MCS-TM-150. Mathematics and Computer Science Division, Argonne National Laboratory, Argonne, IL.Google Scholar
- Balay, S., Buschelman, K., Gropp, W., Kaushik, D., Knepley, M., Curfman-McInnes, L., Smith, B., and Zhang, H. 2003. PETSc 2.0 users manual. Tech. rep. ANL-95/11---revision 2.1.6. Mathematics and Computer Science Division, Argonne National Laboratory, Argonne, IL. Go online to http://www.mcs.anl.gov/petsc.Google Scholar
- Benson, S., McInnes, L., and Moré, J. 2000. TAO users manual. Tech. rep. ANL/MCS-TM-242, Mathematics and Computer Science Division, Argonne National Laboratory, Argonne, IL. Go online to www.mcs.anl.gov/tao.Google Scholar
- Benson, S. and Moré, J. 2001. A limited memory variable metric algorithm for bound constrained minimization. Tech. rep. ANL/MCS-P909-0901. Mathematics and Computer Science Division, Argonne National Laboratory, Argonne, IL.Google Scholar
- Berz, M., Bischof, C., Corliss, G., and Griewank, A., Eds. 1996. Computational Differentiation: Techniques, Applications, and Tools. Proceedings Series. SIAM Press, Philadelphia, PA.Google Scholar
- Bischof, C., Carle, A., Khademi, P., and Maurer, A. 1996. The ADIFOR 2.0 system for automatic differentiation of Fortran 77 programs. IEEE Comp. Sci. Eng. 3, 3, 18--32. Google Scholar
- Bischof, C., Roh, L., and Mauer, A. 1997. ADIC---An extensible automatic differentiation tool for ANSI-C. Softw. Pract. Exper. 27, 12, 1427--1456. Google Scholar
- Cohen, M., Naumann, U., and Riehme, J. 2003. Toward differentiation-enabled Fortran 95 compiler technology. In Proceedings of the 2003 ACM Symposium on Applied Computing. 143--147. Google Scholar
- Corliss, G., Faure, C., Griewank, A., Hascoet, L., and Naumann, U., Eds. 2002. Automatic Differentiation of Algorithms---From Simulation to Optimization. Springer, New York, NY. Google Scholar
- Corliss, G. and Griewank, A., Eds. 1991. Automatic Differentiation: Theory, Implementation, and Application. Proceedings Series. SIAM Press, Philadelphia, PA.Google Scholar
- Curtis, A., Powell, M., and Reid, J. 1974. On the estimation of sparse Jacobian matrices. J. Inst. Math. Appl. 13, 117--119.Google Scholar
- Forth, S., Tadjouddine, M., Pryce, J., and Reid, J. 2004. Jacobian code generated by source transformation and vertex elimination can be as efficient as hand-coding. ACM Trans. Math. Softw. 3, 30 (Sep.), 266--299. Google Scholar
- Garey, M. and Johnson, D. 1979. Computers and Intractability---A Guide to the Theory of NP-Completeness. W. H. Freeman and Company, San Francisco, CA. Google Scholar
- Griewank, A. 2000. Evaluating Derivatives. Principles and Techniques of Algorithmic Differentiation. Number 19 in Frontiers in Applied Mathematics. SIAM Press, Philadelphia, PA. Google Scholar
- Griewank, A. and Reese, S. 1991. On the calculation of Jacobian matrices by the Markovitz rule. In Corliss and Griewank {1991}, 126--135.Google Scholar
- Hascoët, L., Naumann, U., and Pascual, V. 2004. TBR analysis in reverse-mode automatic differentiation. In Future Generation Computer Systems---Special Issue on Automatic Differentiation, M. Bücker, Ed. Elsevier, Amsterdam, The Netherlands.Google Scholar
- Hopkins, T. and Phillips, C. 1988. Numerical Methods in Practice Using the NAG Library. International Computer Science Series. Addison-Wesley, Reading, MA.Google Scholar
- Hovland, P., Norris, B., and Smith, B. 2002. Making automatic differentiation truly automatic: Coupling PETSc with ADIC. In Computational Science---ICCS 2002, Proceedings of the International Conference on Computational Science, Amsterdam, The Netherlands, April 21--24, 2002. Part II, P. M. A. Sloot, C. J. K. Tan, J. J. Dongarra, and A. G. Hoekstra, Eds. Lecture Notes in Computer Science, vol. 2330. Springer, Berlin, Germany, 1087--1096. Google Scholar
- Moré, J. 2001. Automatic differentiation tools in optimization software. In Corliss et al. {2002}, 25--34.Google Scholar
- Naumann, U. 2002. On optimal Jacobian accumulation for single-expression-use programs. Preprint ANL-MCS/P944-0402. Mathematics and Computer Science Division, Argonne National Laboratory, Argonne, IL.Google Scholar
- Naumann, U. 2004. Optimal accumulation of Jacobian matrices by elimination methods on the dual computational graph. Math. Prog. 99, 3 (Apr.), 399--421. Google Scholar
- Naumann, U. and Riehme, J. 2005. Computing adjoints with the NAGWare Fortran 95 compiler. In Automatic Differentiation: Applications, Theory, and Tools, H. Bücker, G. Corliss, P. Hovland, U. Naumann, and B. Norris, Eds. Lecture Notes in Computational Science and Engineering, vol. 50. Springer, Berlin, Germany, 159--169.Google Scholar
- Newsam, G. and Ramsdell, J. 1983. Estimation of sparse Jacobian matrices. SIAM J. Alg. Dis. Meth. 4, 404--417.Google Scholar
- Roe, P. 1981. Approximating Riemann solvers, parameter vectors, and difference schemes. J. Comp. Physics 43, 357--372. Google Scholar
- Tadjouddine, M., Forth, S., and Pryce, J. 2001. AD tools and prospects for optimal AD in CFD flux calculations. In Corliss et al. {2002}, 255--261.Google Scholar
- Tadjouddine, M., Forth, S., Pryce, J., and Reid, J. 2002. Performance issues for vertex elimination methods in computing Jacobians using automatic differentiation. In Computational Science---ICCS 2002, Proceedings of the International Conference on Computational Science, Amsterdam, The Netherlands, April 21--24, 2002. Part II, P. Sloot, C. Tan, J. Dongarra, and A. Hoekstra, Eds. Lecture Notes in Computer Science, vol. 2330. Springer, Berlin, Germany, 1077--1086. Google Scholar
- The Numerical Algorithms Group, 2000. NAG Fortran 90 library. Online documentation, Oxford, U.K., at http://www.nag.co.uk/numeric/FN/manual/html/FNlibrarymanual.asp.Google Scholar
- The Numerical Algorithms Group, 2002. The NAG Fortran library manual, mark 20. Online documentation at http://www.nag.co.uk/numeric/fl/manual/html/FLlibrarymanual.asp.Google Scholar
- Utke, J. 2005. Flattening basic blocks. In Automatic Differentiation: Applications, Theory, and Tools, H. Bücker, G. Corliss, P. Hovland, U. Naumann, and B. Norris, Eds. Lecture Notes in Computational Science and Engineering, vol. 50. Springer, Berlin, Germany, 121--133.Google Scholar
- Utke, J. and Naumann, U. 2004a. Optimality-preserving elimination of linearities in Jacobian accumulation. Presentation at SIAM Workshop on Combinatorial Scientific Computing, San Francisco, February 2004. Article is under review for special issue on combinatorial scientific computing of the Electronic Transactions on Numerical Analysis (ETNA), dedicated to Professor Alan George.Google Scholar
- Utke, J. and Naumann, U. 2004b. Separating language-dependent und independent tasks for the semantic transformation of numerical programs. In Software Engineering and Applications, M. Hamza, Ed. ACTA Press, Calgary, Alta., Canada (To appear).Google Scholar
- Wengert, R. 1964. A simple automatic derivative evaluation program. Comm. ACM 7, 463--464. Google Scholar
Index Terms
- A differentiation-enabled Fortran 95 compiler
Recommendations
Implementation of automatic differentiation tools
Automatic differentiation is a semantic transformation that applies the rules of differential calculus to source code. It thus transforms a computer program that computes a mathematical function into a program that computes the function and its ...
Implementation of automatic differentiation tools
PEPM '02: Proceedings of the 2002 ACM SIGPLAN workshop on Partial evaluation and semantics-based program manipulationAutomatic differentiation is a semantic transformation that applies the rules of differential calculus to source code. It thus transforms a computer program that computes a mathematical function into a program that computes the function and its ...
Comments