Automatisch Differenzierung (AD) is a computational technique used to evaluate the derivative of a function specified by a computer program. Unlike numerical differentiation, which approximates derivatives using finite differences, and symbolic differentiation, which manipulates mathematical expressions, automatic differentiation provides exact derivatives using the Kettenregel of calculus. This makes it particularly valuable in fields like maschinellem Lernen, where optimization Probleme hängen oft von der genauen Berechnung der Gradienten ab.
AD funktioniert, indem es aufteilt complex functions into simpler components, allowing derivatives to be computed in a systematic way. There are two primary modes of automatic differentiation: Vorwärtsmodus and Rückwärtsmodus. In forward mode, the derivatives are propagated alongside the function evaluation, which is efficient for functions with fewer inputs than outputs. Conversely, reverse mode is more suited for functions with many inputs and fewer outputs, as it computes derivatives in a single Rückwärtsdurchlauf nachdem die Funktion ausgewertet wurde.
Diese Technik wird in verschiedenen Anwendungen weit verbreitet eingesetzt, einschließlich Optimierungsalgorithmen like gradient descent, where knowing the gradient is essential for updating model parameters. It is also a fundamental component of many machine learning frameworks, enabling efficient training of neural networks. By providing a robust and accurate means of computing derivatives, automatic differentiation plays a crucial role in modern computational science and artificial intelligence.