Consider the function \(f:\mathbb{R} \to \mathbb{R}\) defined by \(f(x) = x^2\).
The derivative of \(f\) at \(x_0\), as presented in single-variable calculus, is the number \(f'(x_0)\) such that \[ f'(x_0) = \lim_{h \to 0} \frac{f(x_0+h) - f(x_0)}{h}. \]
As long as \(h\) is small, we can drop the limit, replace “\(=\)” with “\(\approx\)”, and rearrange to get \[ f(x_0 + h) - f(x_0) \approx f'(x_0)h. \]
In order for all of this to make sense, we need to think of \(h\) as: \[ h = (x_0 + h) - x_0. \]
In this way of thinking, \(h\) is a “step vector” in the domain, i.e., a vector along the \(x\)-axis with its tail at the point \(x_0\) and its head at the point \(x_0 + h\).
Single-variable calculus
Let \(f:\mathbb{R} \to \mathbb{R}\) be a function and let \(x_0\) be a point in the domain of \(f\). The derivative \(f'(x_0)\) in single-variable calculus is:
Multi-variable calculus
Let \(f:\mathbb{R}^n \to \mathbb{R}^m\) be a function and let \(\mathbf{v}_0\) be (the position vector of) a point in the domain of \(f\). The derivative \(f'(\mathbf{v}_0)\) in multivariable calculus is:
So… does that mean \(f(\mathbf{v}_0 + \mathbf{h}) - f(\mathbf{v}_0) \approx f'(\mathbf{v}_0)\mathbf{h}\)?
ABSOLUTELY!
One clever way to make a step vector \(\mathbf{h}\) go to \(0\) is to consider a scalar multiple \(\lambda\mathbf{h}\), and let \(\lambda\to 0\).
Then, our approximation becomes \[ f(\mathbf{v}_0 + \lambda\mathbf{h}) - f(\mathbf{v}_0) \approx f'(\mathbf{v}_0)(\lambda\mathbf{h}). \]
But, because \(f'(\mathbf{v}_0)(\lambda\mathbf{h})\) is the product of a matrix and a scalar multiple of a vector, the \(\lambda\) can be pulled out: \[ f'(\mathbf{v}_0)(\lambda\mathbf{h}) = \lambda f'(\mathbf{v}_0)\mathbf{h}. \]
Then, our approximation becomes \[ f(\mathbf{v}_0 + \lambda\mathbf{h}) - f(\mathbf{v}_0) \approx \lambda f'(\mathbf{v}_0)\mathbf{h}. \]
Dividing both sides by \(\lambda\) and flipping sides gives \[ f'(\mathbf{v}_0)\mathbf{h} \approx \frac{f(\mathbf{v}_0 + \lambda\mathbf{h}) - f(\mathbf{v}_0)}{\lambda}. \]
This approximation is supposed to become exact as \(\lambda\to0\), which leads us to.
Definition.
Let \(f:\mathbb{R}^n \to \mathbb{R}^m\) be a function and let \(\mathbf{v}_0\) be (the position vector of) a point in the domain of \(f\). The derivative of \(f\) at \(\mathbf{v}_0\) is the \(m\times n\) matrix \(f'(\mathbf{v}_0)\) such that \[ f'(\mathbf{v}_0)\mathbf{h} = \lim_{\lambda \to 0} \frac{f(\mathbf{v}_0 + \lambda\mathbf{h}) - f(\mathbf{v}_0)}{\lambda} \] for all vectors \(\mathbf{h}\) in \(\mathbb{R}^n\). (Provided the limit exists.)
Consider the function
\[ f:\mathbb{R}^2 \to \mathbb{R}^2, \quad f(x,y) = (y^2+x, y). \]
Compute the derivative \(f'(x,y)\) using the definition, following the steps below.
Identify the size of the derivative matrix \(f'(x,y)\).
Identify the position vector \(\mathbf{v}_0\) and its size.
Identify the step vector \(\mathbf{h}\) and its size.
Now compute \(f'(x,y)\) using the above information and the definition of the derivative.
Use your answer to approximate the change in the function \(f(1.1, 0.9) - f(1,1)\).
Consider the function
\[ f:\mathbb{R} \to \mathbb{R}^2, \quad f(t) = (t^2, t^3), \]
Compute the derivative \(f'(t)\) using the definition, following the steps below.
Identify the size of the derivative matrix \(f'(t)\).
Identify the position vector \(\mathbf{v}_0\) and its size.
Identify the step vector \(\mathbf{h}\) and its size.
Now compute \(f'(t)\) using the above information and the definition of the derivative.
Use your answer to approximate the change in the function \(f(1.9) - f(2)\).
Consider the function
\[ f:\mathbb{R} \to \mathbb{R}^2, \quad f(t) = (\cos{t}, \sin{t}), \]
Compute the derivative \(f'(t)\) using the definition, following the steps below.
Identify the size of the derivative matrix \(f'(t)\).
Identify the position vector \(\mathbf{v}_0\) and its size.
Identify the step vector \(\mathbf{h}\) and its size.
Now compute \(f'(t)\) using the above information and the definition of the derivative.
Use your answer to approximate the change in the function \(f(1.7) - f(1.8)\).
Consider the function
\[ f:\mathbb{R}^2 \to \mathbb{R}, \quad f(x,y) = x^2 + y^2, \]
Compute the derivative \(f'(x,y)\) using the definition, following the steps below.
Identify the size of the derivative matrix \(f'(x,y)\).
Identify the position vector \(\mathbf{v}_0\) and its size.
Identify the step vector \(\mathbf{h}\) and its size.
Now compute \(f'(x,y)\) using the above information and the definition of the derivative.
Use your answer to approximate the change in the function \(f(10.2, -4.3) - f(10.05,-4.15)\).
Let \(f:\mathbb{R}^n \to \mathbb{R}^m\) be a function and let \(\mathbf{v}_0\) be (the position vector of) a point in the domain of \(f\).
As long as the step vector \(\mathbf{h}\) is small, the derivative \(f'(\mathbf{v}_0)\) was designed so that \[ f(\mathbf{v}_0 + \mathbf{h}) \approx f'(\mathbf{v}_0)\mathbf{h} + f(\mathbf{v}_0). \]
Now, if \(\mathbf{v}\) is the position vector of a (variable) point in the domain of \(f\) that is close to \(\mathbf{v}_0\), then we can write \(\mathbf{v} = \mathbf{v}_0 + \mathbf{h}\) for some small step vector \(\mathbf{h}\).
Then, we can write \[ f(\mathbf{v}) \approx f'(\mathbf{v}_0)(\mathbf{v} - \mathbf{v}_0) + f(\mathbf{v}_0). \]
The right-hand side of this approximation has a name:
Definition.
Let \(f:\mathbb{R}^n \to \mathbb{R}^m\) be a differentiable function and let \(\mathbf{v}_0\) be (the position vector of) a point in the domain of \(f\). The tangent approximation to \(f\) at \(\mathbf{v}_0\) is the function
\[ L_{\mathbf{v}_0}(\mathbf{v}) = f'(\mathbf{v}_0)(\mathbf{v} - \mathbf{v}_0) + f(\mathbf{v}_0), \]
where \(\mathbf{v}\) is the position vector of a (variable) point in the domain of \(f\).
Definition.
Let \(f:\mathbb{R}^n \to \mathbb{R}^m\) be a differentiable function and let \(\mathbf{v}_0\) be (the position vector of) a point in the domain of \(f\). The tangent space to \(f\) at \(\mathbf{v}_0\) is the range of the tangent approximation \(L_{\mathbf{v}_0}\).
A couple concrete cases to keep in mind:
Consider the function
\[ f:\mathbb{R}^2 \to \mathbb{R}, \quad f(x,y) = x^2 + y^2, \]
Compute the tangent approximation to \(f\) at the point \((x_0,y_0) = (1,1)\).
Plot the graph of \(f\) along with its tangent plane at the point \((x_0,y_0) = (1,1)\).
Consider the function
\[ f:\mathbb{R} \to \mathbb{R}^2, \quad f(t) = (t^2, t^3), \]
Compute the tangent approximation to \(f\) at the point \(t_0 = 1\).
Visualizing \(f\) as a physical transformation embedding the \(t\)-line into the \(xy\)-plane, plot the tangent line to the curve at the point \(t_0 = 1\).
Consider the function
\[ f:\mathbb{R} \to \mathbb{R}^2, \quad f(t) = (\cos{t}, \sin{t}), \]
Compute the tangent approximation to \(f\) at the point \(t_0 = 1\).
Visualizing \(f\) as a physical transformation embedding the \(t\)-line into the \(xy\)-plane, plot the tangent line to the curve at the point \(t_0 = 1\).
Consider the function
\[ f:\mathbb{R}^2 \to \mathbb{R}, \quad f(x,y) = (y^3 +x, y), \]
Compute the tangent approximation to \(f\) at the point \((x_0,y_0) = (1,1)\).
What is the tangent space to \(f\) at the point \((x_0,y_0) = (1,1)\)? Is it a line, a plane, or something else?