Programme of the 26th Euro AD Workshop

Lessons to be learned from adjoint AD when implementing Newton steps.; potential savings in computational cost by an order of magnitude.

We present a method for transforming directed acyclic graphs, induced by functions with invertible Jacobians, into equivalent uniformly layered graphs. These computational graphs can be transformed into layered graphs using edge splitting. To achieve an equal number of vertices in each layer (uniformity), we selectively partially preaccumulate local Jacobians. The transformed graphs allow for the application of a system matrix-free Newton method that potentially yields a significant speed-up compared to standard methods. In this context, challenges arise in efficiently leveraging the structural pattern and minimizing fill-in in the decomposition of each local Jacobian.

It is commonly advised not to brute-force differentiate through linear solvers, and to instead use a high-level formulation that involves solving a tangent or adjoint linear system.

We ignore this good advice and differentiate through some of the most commonly used iterative linear solvers. We present the results for some matrices from the suitesparse matrix collection.

We demonstrate why it is necessary to provide a support for small scale function integration in AD tapes. In addition we present a generator for integrating function into AD tapes and some results.

The presentation shows how minor adaptions to the control unit and datapath of a RISC-V processor enable us to apply forward-mode automatic differentiation in hardware to arithmetic instructions. The proposed processor design extends an existing implementation of the RV32I RISC-V ISA, written for high-level synthesis in C. With minor adaptions to the existing architecture, we can enable automatic differentiation in hardware. To that extent, we introduce custom RISC-V instructions. The synthesized design runs on an FPGA.

Derivgrind is a novel AD tool applicable to the machine code of compiled programs, implemented in the Valgrind framework for dynamic binary instrumentation. On the machine-code level, real-arithmetic operations are usually performed by means of the corresponding floating-point instructions, which are straightforward to identify. However, while applying Derivgrind to various primal programs, we encountered a few alternative, hard-to-spot ways to implement certain real-arithmetic operations. In this talk, we present a new feature in Derivgrind that allows to heuristically detect, and roughly localize, part of these 'bit-tricks'.

Using reverse accumulation and the theory of fixed points, Bruce Christianson showed that first order derivative of fixed point iterations can be computed also with a fixed point iteration inherting the convergence rate. In this talk we examine the calculation of second order derivatives for a fixed point iteration.

pyhonOCC provides Python wrappers for the widely-used C++ geometric kernel Open CASCADE Technology (OCCT) that facilitate the use of the C++ library for many complex industrial workflows. To employ it in a CAD-enabled gradient-based shape optimization, one requires the calculation of the so-called geometric sensitivities. For this purpose, pythonOCC is differentiated with ADOL-C by integrating a Python wrapper of the adouble class into its source code. Moreover, it is linked against the AD-enabled OCCT C++ kernel that is also differentiated with ADOL-C. The approach enables the propagation of derivatives from Python to C++ and vice-versa in the context of CAD-based shape optimization for complex engineering configurations such as aircraft and engine models.

The Generalized Jacobian Chain Problem tries to minimize the cost of accumulating the Jacobian of a function, given the tangent and adjoint models of all concatenated subprograms. For real world applications the main difficulty is not the solution of the Jacobian Chain Problem but the identification of suitable subprograms and the extraction of the corresponding tangent and adjoint models. We are solving this issue via a profiling step which analyzes the set of active variables throughout the execution and selects cuts in the DAG which naturally divide the program into suitable subprograms, with implicit tangent and adjoint models which can be evaluated via forward and reverse tape interpreations.

Our work with differentiable quantum dynamics requires automatic differentiation of a complex-valued initial value problem, which numerically integrates a system of ordinary differential equations from a given initial condition. We explored several automatic differentiation frameworks for this task, finding that no framework natively supports our application requirements. We therefore raise a call for broader support of complex-valued, differentiable numerical integration in scientific computing libraries. As a first step in this direction, we document the minor change required to enable this capability in JAX.

Differential Machine Learning, aka Sobolev Training, augments the typical neural network supervised learning process with differential data labels obtained via automatic differentiation. This additional loss factor results in an unbiased form of regularization. We extend the learning process of neural network-based surrogate models with second-order derivative information. In this talk, we present effective ways to find relevant directions for hessian-vector products and evaluate the techniques for common option pricing models (Bachelier and Heston). Beyond finance, Second-Order Differential Machine Learning could be a generally applicable tool for mean prediction of stochastic numerical models using efficient yet accurate surrogate models.

We are considering differential-algebraic equations with embedded optimization criteria (DAEOs) in which the embedded optimization problem is solved by global optimization. This actually leads to differential inclusions for cases in which there are multiple global optimizer at the same time. Jump events from one global optimum to another result in nonsmooth DAEs and thus reduction of the convergence order of the numerical integrator to first-order. Implementation of event detection and location as introduced in this work preserves the higher-order convergence behavior of the integrator. This allows to compute discrete tangents and adjoint sensitivities for optimal control problems.

AD.Net offers a practical solution for language-independent automatic differentiation, empowering community to work with multiple .NET-supported languages while efficiently handling complex linear algebraic problems with matrices value functions at Intermediate Language Level. Our approach has the potential to simplify and streamline the differentiation process across a diverse range of programming languages in the .NET ecosystem.