www.Autodiff.org - Publication: Event-Based Automatic Differentiation of OpenMP with OpDiLib

Event-Based Automatic Differentiation of OpenMP with OpDiLib

- Article in a journal -

Author(s)
Johannes Blühdorn , Max Sagebaum , Nicolas Gauger

Published in
ACM Trans. Math. Softw.

Year
2023

Publisher
Association for Computing Machinery

Abstract
We present the new software OpDiLib, a universal add-on for classical operator overloading ad tools that enables the automatic differentiation (ad) of OpenMP parallelized code. With it, we establish support for OpenMP features in a reverse mode operator overloading ad tool to an extent that was previously only reported on in source transformation tools. We achieve this with an event-based implementation ansatz that is unprecedented in ad. Combined with modern OpenMP features around OMPT, we demonstrate how it can be used to achieve differentiation without any additional modifications of the source code; neither do we impose a priori restrictions on the data access patterns, which makes OpDiLib highly applicable. For further performance optimizations, restrictions like atomic updates on adjoint variables can be lifted in a fine-grained manner. OpDiLib can also be applied in a semi-automatic fashion via a macro interface, which supports compilers that do not implement OMPT. We demonstrate the applicability of OpDiLib for a pure operator overloading approach in a hybrid parallel environment. We quantify the cost of atomic updates on adjoint variables and showcase the speedup and scaling that can be achieved with the different configurations of OpDiLib in both the forward and the reverse pass.

AD Theory and Techniques
Parallelism

BibTeX
@ARTICLE{
         Bluhdorn2023EBA,
       author = "Bl\"{u}hdorn, Johannes and Sagebaum, Max and Gauger, Nicolas",
       title = "Event-Based Automatic Differentiation of {OpenMP} with {OpDiLib}",
       year = "2023",
       issue_date = "March 2023",
       publisher = "Association for Computing Machinery",
       address = "New York, NY, USA",
       volume = "49",
       number = "1",
       issn = "0098-3500",
       url = "https://doi.org/10.1145/3570159",
       doi = "10.1145/3570159",
       abstract = "We present the new software OpDiLib, a universal add-on for classical operator
         overloading AD tools that enables the automatic differentiation (AD) of OpenMP parallelized code.
         With it, we establish support for OpenMP features in a reverse mode operator overloading AD tool to
         an extent that was previously only reported on in source transformation tools. We achieve this with
         an event-based implementation ansatz that is unprecedented in AD. Combined with modern OpenMP
         features around OMPT, we demonstrate how it can be used to achieve differentiation without any
         additional modifications of the source code; neither do we impose a priori restrictions on the data
         access patterns, which makes OpDiLib highly applicable. For further performance optimizations,
         restrictions like atomic updates on adjoint variables can be lifted in a fine-grained manner.
         OpDiLib can also be applied in a semi-automatic fashion via a macro interface, which supports
         compilers that do not implement OMPT. We demonstrate the applicability of OpDiLib for a pure
         operator overloading approach in a hybrid parallel environment. We quantify the cost of atomic
         updates on adjoint variables and showcase the speedup and scaling that can be achieved with the
         different configurations of OpDiLib in both the forward and the reverse pass.",
       journal = "ACM Trans. Math. Softw.",
       month = "mar",
       articleno = "3",
       numpages = "31",
       keywords = "Algorithmic differentiation, OpenMP, OMPT, high performance computing, AD tool
         design, reusable software"
,
       ad_theotech = "Parallelism"}

back

autodiff.org
Username:
Password: