Differentiability Formalism and Sufficient Conditions for Weak or Locally Weak Minima

If you only care for the sufficient conditions, but not for the differentiability formalism, proceed through the 1st bullet, then skip to eqn (3)

Assume L ∈ C^{2}([a, b] × G × ℝ^{n} → ℝ), where G ⊂ ℝ^{n} is open, and consider the functional I defined by

I[y] := ∫_{a}^{b} L(t, y(t), y′(t)) dt , I : C^{1}([a, b] → G ⊂ ℝ^{n}) → ℝ  (0)

is in C^{2}. More precisely we claim that the 1st and 2nd derivatives are

DI[y]φ = ∫_{a}^{b}{L_{y}(t, y(t), y′(t))φ(t) + L_{y′}(t, y(t), y′(t))φ′(t)} dt  (1)

and

\begin{matrix}D^{2}I[y](φ, ψ) = ∫_{a}^{b}\{ & φ(t)^{T} L_{yy}(t, y(t), y′(t))ψ(t) + \\ + φ(t)^{T} L_{yy′}(t, y(t), y′(t))ψ′(t) + ψ(t)^{T} L_{yy′}(t, y(t), y′(t))φ′(t) + \\ + φ′(t)^{T} L_{y′y′}(t, y(t), y′(t))ψ′(t)\} dt \end{matrix} (2)

respectively.

• First we must show that the linear mapping φ ↦ DI[y]φ defined by (1) is continuous from the space X := C^{1}([a, b] → G) to ℝ. This we need to show because the very definition of ‘derivative’, requires a continuous linear map as the analog of the Jacobi matrix. This step is therefore preliminary to differentiability. It has nothing to do with continuous differentiability, which is about continuity of the map y ↦ DI[y].

Note that the range R := {(t, y(t), y′(t)) : a ≤ t ≤ b} ⊂ [a, b] × G × ℝ^{n} is compact for each fixed function y. For sufficiently small δ, a closed δ-neighbourhood R_{δ} := {(t, u, v) : a ≤ t ≤ b , |u − y′(t)| ≤ δ , |u − y(t)| ≤ δ} of this range will therefore still lie in [a, b] × G × ℝ^{n}, and R_{δ} is still a compact set. In particular, given any such function y, the set

{(t, z(t), z′(t)) : a ≤ t ≤ b , ∥z − y∥ ≤ δ}

lies in the compact set R_{δ}. Here ∥ · ∥ of course refers to the norm in C^{1}([a, b] → ℝ^{n}), namely ∥φ∥ := max{|φ(t)|, |φ′(t)| : a ≤ t ≤ b}. The continuous functions L_{y}, L_{y′}, L_{yy}, L_{yy′}, L_{y′y′} are uniformly continuous on the compact set R_{δ}.

The continuity of the linear map DI[y] follows from

|DI[y]φ| ≤ |b − a|(max_{R_{δ}} |L_{y}| + max_{R_{δ}} |L_{y′}|)∥φ∥

(and the modulus of continuity is locally uniform).

The continuity (with locally uniform modulus of continuity) of the bilinear map (φ, ψ) ↦ D^{2}I[y](φ, ψ) that is defined by (2) follows analogously.

• At this stage, we have not shown yet that DI[y] and D^{2}I[y] indeed are the derivatives which by name they claim to be, even though it is clear from the directional derivative arguments that they are the only candidates for the job.

• We next show the continuity of the maps y ↦ DI[y] and y ↦ D^{2}I[y].

By uniform continuity of L_{y} and L_{y′} on R_{δ}, we conclude that, given any ε, we can find η < δ such that |L_{y}(t, y(t), y′(t)) − L_{y}(t, z(t), z′(t))| < ε (and a similar formula with L_{y′}) provided ∥y −z∥ < η. Then |DI[y]φ−DI[z]φ| < 2ε|b−a|∥φ∥, i.e. the norm of the linear maps satisfies ∥DI[y] − DI[z]∥ < 2ε|b − a|, provided ∥y − z∥ < η < δ. An analogous argument can be made for D^{2}I.

• Now we show that DI[y] is indeed the derivative of I at y, i.e., that

|I[y + φ] − I[y] − DI[y]φ|/∥φ∥ → 0 as ∥φ∥ → 0

Indeed

I[y + φ] − I[y] − DI[y]φ =

= ∫_{a}^{b}(L(t, y(t) + φ(t), y′(t) + φ′(t)) − L(t, y(t), y′(t))

−L_{y}(t, y(t), y′(t))φ(t) − L_{y′}(t, y(t), y′(t))φ′(t)) dt

= ∫_{a}^{b} ∫_{0}^{1} \frac{d}{ds}(L(t, y(t) + sφ(t), y′(t) + sφ′(t))

−sL_{y}(t, y(t), y′(t))φ(t) − sL_{y′}(t, y(t), y′(t))φ′(t)) ds dt

= ∫_{a}^{b} ∫_{0}^{1}{(L_{y}(… y + sφ …) − L_{y}(… y …))φ(t)

+ (L_{y′}(… y + sφ …) − L_{y′}(… y …))φ′(t)} ds dt

Now if ∥φ∥ is sufficiently small, then all occurring arguments to L_{y} and L_{y′} lie in the set R_{δ}, where the functions L_{y} and L_{y′} are uniformly continuous. Then for every ε, there exists some η such that the differences in the big parentheses will be uniformly smaller than ε provided only ∥φ∥ < η. We conclude that |I[y + φ] − I[y] − DI[y]φ| ≤ 2(b − a)ε∥φ∥, which was to be shown.

By the same method it can be shown that D^{2}I[y] is indeed the second derivative it claims to be. Namely, the estimate needs to prove that

|DI[y + ψ]φ − DI[y]φ − D^{2}I[y](ψ, φ)|/∥φ∥ ∥ψ∥ → 0

as ∥ψ∥ → 0, and two invocations of the fundamental theorem of calculus and the uniform continuity of the 2nd derivatives of L on R_{δ} do the trick.

• So we have seen that I ∈ C^{2}(X → ℝ), where X is itself a Banach space, namely X = C^{1}([a, b] → ℝ^{n}). This implies the same approximation of I by a 2nd degree Taylor ‘polynomial’ as in multivariable calculus, namely:

|I[y + φ] − I[y] − DI[y]φ − \frac{1}{2}D^{2}I[y](φ, φ)|/∥φ∥^{2} → 0 as ∥φ∥ → 0 .  (3)

Proof of (3):

I[y + φ] − I[y] = ∫_{0}^{1} \frac{d}{ds}I[y + sφ] ds = ∫_{0}^{1} DI[y + sφ]φ ds

Hence

I[y + φ] − I[y] − DI[y]φ = ∫_{0}^{1}(DI[y + sφ] − DI[y])φ ds

= ∫_{0}^{1} ∫_{0}^{s} \frac{d}{dσ} DI[y + σφ]φ dσ ds

= ∫_{0}^{1} ∫_{σ}^{1} D^{2}I[y + σφ](φ, φ) ds dσ

= ∫_{0}^{1}(1 − σ)D^{2}I[y + σφ](φ, φ) dσ

and

\begin{matrix}I[y + φ] − I[y] − DI[y]φ − \frac{1}{2}D^{2}I[y](φ, φ) = \\ = ∫_{0}^{1}(1 − σ)(D^{2}I[y + σφ](φ, φ) − D^{2}I[y](φ, φ)) dσ \end{matrix}  (4)

Using formula (2) for D^{2}I and the uniform continuity of L_{yy}, L_{yy′}, L_{y′y′} on R_{δ} again, claim (3) is immediate.

We therefore have a rigorous analog of the MV-Calculus argument for relative minima. Having shown that, if L ∈ C^{2}([a, b] × G × ℝ^{n} → ℝ), then I ∈ C^{2}(X → ℝ) (where X = C^{1}([a, b] → ℝ^{n})), we obtain from (3) the 1st part of the following result immediately:

Theorem:

(a) If L ∈ C^{2}([a, b] × G × ℝ^{n} → ℝ) and y_{∗} satisfies the EL-eqn and if D^{2}I[y_{∗}](φ, φ) ≥ c∥φ∥^{2} with c > 0 for all φ ∈ C_{0}^{1}([a, b] → ℝ^{n}) in case of fixed boundary conditions, or else for all φ ∈ C^{1}([a, b] → ℝ^{n}) in case of free boundary, then y_{∗} is a weak minimum of I given in (0).

(b) If however we only have the positive definiteness of L_{y′y′}(t, y_{∗}(t), y′_{∗}(t)) for all t ∈ [a, b] (which implies the uniform positive definiteness in a neighbourhood R_{δ}), then y_{∗} is locally a weak minimum, i.e., sufficiently short segments of y_{∗} are weak minima.

As far as proving the 2nd part is concerned, we suppose that the support of φ has (small) length h, and recall that by continuity, there is an upper bound M for L_{yy} and L_{yy′}, and by uniform positive definiteness, there is a lower bound m > 0 such that φ′(t)^{T} L_{y′y′}(…)φ′(t) ≥ m|φ′(t)^{2}|.

For such φ, we estimate taking a worst-case (negative) scenario for L_{yy} and L_{yy′} relying only on L_{y′y′} to give something positive in the end (more explanations after the formula):

\begin{aligned}D^{2}I[y_{∗}](φ, φ) & ≥ −M ∫ φ^{2} − 2M ∫ |φ| |φ′| + m ∫ φ^{′2} \\ & ≥ −M ∫ φ^{2} − M(\frac{2M}{m} ∫ φ^{2} + \frac{m}{2M} ∫ φ^{′2}) + m ∫ φ^{′2} \\ & ≥ −M̂ ∫ φ^{2} + \frac{m}{2} ∫ φ^{′2} ≥ (\frac{m}{2} − \frac{M̂h^{2}}{4} ) ∫ φ^{′2} \\ & ≥ (\frac{m}{2} − \frac{M̂h^{2}}{4} )(1 + h^{2}/4)^{−1}∥φ∥_{W^{1,2}}^{2} \end{aligned}  (5)

In line 2, we have estimated 2|φ| φ′| under the integral using the famous and immensely useful inequality 2uv ≤ Au^{2} + v^{2}/A. This inequality is true because (u\sqrt{A} − v/\sqrt{A})^{2} ≥ 0

In line 3 we have introduced the abbreviation M̂ := M + 2M^{2}/m and then used the fact that for φ supported on an interval of length h, the estimate ∫ φ(t)^{2} dt ≤ h^{2}/4 ∫ φ′(t)^{2} dt holds (as mentioned in the notes on pg 18, and reproved the pedestrian way in a lemma below). This is why ‘small support’ always makes the tiniest platoon of ∫ φ^{′2} win against the largest army of ∫ φ^{2}.

In line 4, we have introduced a new norm ∥φ∥_{W^{1,2}}^{2} := ∫ φ^{2} + ∫ φ^{′2}.

We need to (and can) take h so small that \frac{m}{2} − \frac{M̂h^{2}}{4} > 0, and then (3), together with the EL eqn DI[y_{∗}]φ = 0, ought to imply I[y_{∗} + φ] > I[y_{∗}] (but doesn’t, yet…).

A little nuisance arises here: Our positivity estimate for D^{2}I[y_{∗}] hinges on the fact ∫ L_{y′y′}φ^{′2} ≥ ∫ mφ^{′2}, and we cannot do better than this, under the assumption L_{y′y′} > 0. All the other terms merely bite a bit off the constant m, so D^{2}I[y_{∗}](φ, φ) ≥ m̃ ∫ φ^{′2}. So we haven’t squanderend anything essential in estimating the L_{yy} and L_{yy′} terms and therefore cannot gain by improving there. We were thus forced to introduce the new norm ∥φ∥_{W^{1,2}}, because an estimate in terms of ∥φ∥ is just not available. But the error term in the Taylor polynomial estimate (3) is ≤ ε∥φ∥^{2}, where ∥φ∥ denotes our old C^{1} norm max{|φ|, |φ′|}. Now there is no way how we could prove m̃ ∫ φ^{′2} − ε∥φ∥^{2} > 0, because this estimate is simply not true. Whatever small ε/m̃ is, we can still find φ such that m̃ ∫ φ^{′2} − ε∥φ∥^{2} < 0. It seems we have labored in vain.

All the estimates up to (3) were routine estimates in the sense that you just write down what ought to be true in analogy to multivariable calculus, and it is indeed true by a very straightforward argument. (Fair enough if it doesn’t seem routine or straightforward to you, if you are still struggling to absorb the formalism, but trust me it is routine, once you master the formalism.) But then it turns out, as it often happens when lofty function spaces are used for down-and-dirty real problems, that the automatic machinery doesn’t quite do the expected job, because the norm that the machinery likes is not the norm the question likes. So we have to return and do some adjustments manually. Luckily, in this problem this is easy: We need an improved variant of (3), namely:

|I[y + φ] − I[y] − DI[y]φ − \frac{1}{2}D^{2}I[y](φ, φ)|/∥φ∥_{W^{1,2}}^{2} Note new norm ↗ → 0 as ∥φ∥↖ still old C^{1} norm! → 0 .  (6)

This is much stronger than (3) because we divide by a potentially much smaller expression. It is also non-routine, because it blends different norms. Luckily the proof follows readily from (4): For all ε > 0 there exists η < δ such that

|I[y + φ] − I[y] − DI[y]φ − \frac{1}{2}D^{2}I[y](φ, φ)| ≤ ∫_{0}^{1}(1 − σ)(εφ^{2} + 2ε|φφ′| + εφ^{′2})dσ ≤ ε∥φ∥_{W^{1,2}}^{2}

if ∥φ∥ < η < δ. Now we can indeed use (5) with (6) instead of (3) and argue I[y_{∗} + φ] > I[y_{∗}] for φ with small C^{1} norm and small support. Hence y_{∗} is locally a weak minimum, as claimed.

Let’s review the small parameters we have used: Given y_{∗} (solution of the EL eqn) we called R its ‘range’ in [a, b] × G × ℝ^{n}. First we chose δ so that still R_{δ} ⊂ [a, b] × G × ℝ^{n}, and we have uniform continuity of L and all its derivatives up to order 2 on R_{δ}. We get a large constant M bounding the continuous functions L_{yy} and L_{yy′} above, and a small constant m giving a lower bound for L_{y′y′}. From the onset, we commit at least to ∥φ∥ < δ, but we will later require a smaller bound ∥φ∥ < η. From M and m we choose a small h, so that m/2 − M̂h^{2}/4 > 0, and then we choose an ε so that m̃ := (m/2 − M̂h^{2}/4)(1 + h^{2}/4)^{−1} > ε. For this ε, we find η < δ so that ∥φ∥ < η guarantees that the ratio in (6) is less than ε. This gives I[y_{∗} + φ] − I[y_{∗}] ≥ (m̃ − ε)∥varphi∥_{W^{1,2}}^{2} > 0.

Let’s prove the lemma we used: Without loss of generality, we take the interval of length h to be [0, h].

Lemma: If φ(0) = 0 = φ(h), then ∫_{0}^{h} φ^{2} ≤ h^{2}/4 ∫_{0}^{h} φ^{′2}.

Proof: Since φ(0) = 0, we get by menas of the Cauchy Schwarz inequality (refer to it below)

|φ(t)| = | ∫_{0}^{t} 1 · φ′(s) ds| ≤ (∫_{0}^{t} 1^{2} ds)^{1/2}(∫_{0}^{t} φ^{′2}(s) ds)^{1/2} ≤ t^{1/2}(∫_{0}^{h} φ^{′2}(s) ds)^{1/2}

We use this for 0 ≤ t ≤ h/2. For h/2 ≤ t ≤ h, we integrate from t to h and get

|φ(t)| ≤ (h − t)^{1/2}(∫_{0}^{h} φ^{′2}(s) ds)^{1/2}

instead. We conclude

φ(t)^{2} ≤ min{t, h − t} × A where A := ∫_{0}^{h} φ^{′2}(t) dt

Integrating this over [0, h] we get

∫_{0}^{h} φ(t)^{2} dt ≤ \frac{h^{2}}{4} ∫_{0}^{h} φ^{′2}(t) dt

Note: A better constant is possible using a more sophisticated proof technology. Indeed, ∫_{0}^{h} φ(t)^{2} dt ≤ \frac{h^{2}}{π^{2}} ∫_{0}^{h} φ^{′2}(t) dt. But the best possible constant was not our concern.

The Cauchy Schwarz inequality: For any inner product, the following inequality holds (and is called the Cauchy Schwarz inequality):

|⟨u, v⟩| ≤ ∥u∥ ∥v∥ with ∥u∥ := \sqrt{⟨u, u⟩}

It has a very simple albeit tricky proof: Since ⟨tu+v, tu+v⟩ ≥ 0 for every number t, and since we can calculate ⟨tu+v, tu+v⟩ = t^{2}⟨u, u⟩+2t⟨u, v⟩+⟨v, v⟩, we have a quadratic polynomial in t that is never negative. Therefore, if you try to find zeros t_{1}, t_{2} with the quadratic formula, the term under the square root in the quadratic formula must be ≤ 0. This latter condition is the Cauchy Schwarz inequality. For continuous (or merely square integrable) functions defined on an interval (or any domain in ℝ^{n}), we have the inner product ⟨u, v⟩ := ∫ u(x)v(x) dx, and then Cauchy Schwarz reads

|∫ u(x)v(x) dx| ≤ (∫ u(x)^{2} dx)^{1/2} (∫ v(x)^{2} dx)^{1/2}

Jochen Denzler, Feb 2006, Lecture on Calculus of Variations