|
|
DiffSharp: Automatic Differentiation Library-
Technical report
- | |
|
Author(s)
Atilim Gunes Baydin
, Barak A. Pearlmutter
, Jeffrey Mark Siskind
|
Institution arXiv |
Year 2015 |
Abstract In this paper we introduce DiffSharp, an automatic differentiation (ad) library designed with machine learning in mind. ad is a family of techniques that evaluate derivatives at machine precision with only a small constant factor of overhead, by systematically applying the chain rule of calculus at the elementary operator level. DiffSharp aims to make an extensive array of ad techniques available, in convenient form, to the machine learning community. These including arbitrary nesting of forward/reverse ad operations, ad with linear algebra primitives, and a functional API that emphasizes the use of higher-order functions and composition. The library exposes this functionality through an API that provides gradients, Hessians, Jacobians, directional derivatives, and matrix-free Hessian- and Jacobian-vector products. Bearing the performance requirements of the latest machine learning techniques in mind, the underlying computations are run through a high-performance BLAS/LAPACK backend, using OpenBLAS by default. GPU support is currently being implemented. |
AD Tools DiffSharp |
AD Theory and Techniques Implementation Strategies |
BibTeX
@TECHREPORT{
Baydin2015DAD,
title = "DiffSharp: Automatic Differentiation Library",
author = "Atilim Gunes Baydin and Barak A. Pearlmutter and Jeffrey Mark Siskind",
year = "2015",
number = "arXiv:1511.07727",
institution = "arXiv",
abstract = "In this paper we introduce DiffSharp, an automatic differentiation (AD) library
designed with machine learning in mind. AD is a family of techniques that evaluate derivatives at
machine precision with only a small constant factor of overhead, by systematically applying the
chain rule of calculus at the elementary operator level. DiffSharp aims to make an extensive array
of AD techniques available, in convenient form, to the machine learning community. These including
arbitrary nesting of forward/reverse AD operations, AD with linear algebra primitives, and a
functional API that emphasizes the use of higher-order functions and composition. The library
exposes this functionality through an API that provides gradients, Hessians, Jacobians, directional
derivatives, and matrix-free Hessian- and Jacobian-vector products. Bearing the performance
requirements of the latest machine learning techniques in mind, the underlying computations are run
through a high-performance BLAS/LAPACK backend, using OpenBLAS by default. GPU support is currently
being implemented.",
ad_tools = "DiffSharp",
ad_theotech = "Implementation Strategies"
}
| |
back
|
|