Forward mode AutoDiff in R
08/09/21
Forward mode AutoDiff (FAD) involves propagating gradients alongside the computation of an expression. For example consider the function $$ f(x) = \sin(\cos(x)) $$ With FAD we evaluate \(f(x)\) and at the same time evaluate \(f'(x)\). We can implement this in R as follows
# R
x <- list(val=1.2, grad=1)
cos. <- function(x)
list(val = cos(x$val), grad = x$grad * -sin(x$val))
sin. <- function(x)
list(val = sin(x$val), grad = x$grad * cos(x$val))
sin.(cos.(x))
Noting that the gradient computation involves the application of the chain rule. Calling the above code gives
$val
[1] 0.3544799
$grad
[1] -0.8715159
Similarly in R we can use the deriv
function, which applies symbolic and algorithmic differentiation,
# R
fx <- deriv(~ sin(cos(x)), "x", function.arg=T)
fx(1.2)
deriv
works for a range of simple functions; however it's always good to know how to do this yourself.