Forward mode AutoDiff in R
08/09/21

Forward mode AutoDiff (FAD) involves propagating gradients alongside the computation of an expression. For example consider the function $$ f(x) = \sin(\cos(x)) $$ With FAD we evaluate \(f(x)\) and at the same time evaluate \(f'(x)\). We can implement this in R as follows

# R
x <- list(val=1.2, grad=1)

cos. <- function(x)
    list(val = cos(x$val), grad = x$grad * -sin(x$val))

sin. <- function(x)
    list(val = sin(x$val), grad = x$grad * cos(x$val))

sin.(cos.(x))

Noting that the gradient computation involves the application of the chain rule. Calling the above code gives

$val
[1] 0.3544799

$grad
[1] -0.8715159

Similarly in R we can use the deriv function, which applies symbolic and algorithmic differentiation,

# R
fx <- deriv(~ sin(cos(x)), "x", function.arg=T)
fx(1.2)

deriv works for a range of simple functions; however it's always good to know how to do this yourself.

🡨 Prev
Index
Next 🡪