8. Interchanging operators

• \(I \subseteq \mathbb{R}\) interval

• \(Y\) set with accumulation point \(y_0\)

• \(f: \ I \times Y \rightarrow \mathbb{R}\)

Let the pointwise limit

\[g(x) := \lim_{y \rightarrow y_0} f(x,y) \ \]

exists for \(\forall x \in I\), i.e. analogously \(\forall \varepsilon > 0\), \(\forall x \in I\), \(\exists \delta > 0\) so that \(|f(x,y) - g(x)| < \delta\) for \(\forall y \in U_{y_0, \delta}\).

8.1. Interchange limits - Moore-Osgood theorem

Let \(x_0\) a limit point of \(I\). Suppose

1. for \(\forall y \in V_{y_0}\) (punctured neighborhood of \(y_0\)), \(\varphi(y) := \lim_{x \rightarrow x_0} f(x,y)\) exists. (Here \(y\) is taken as fixed. If \(y\) is taken as a parameter, these are common limits of function of one variable, for all the possible parameters \(y\); can it be interpreted as pointwise convergence as \(x \rightarrow x_0\) for all the points \(y \in V_{y_0}\)?)

2. \(f(x,y) \rightarrow g(x)\) as \(y \rightarrow y_0\) uniformly for \(x \in U_{x_0}\) (punctured)

Then

• \(\lim_{y \rightarrow y_0} \varphi(y)\) exists and

  \[\lim_{y \rightarrow y_0} \varphi(y) = \lim_{x \rightarrow x_0} g(x) = \lim_{(x,y) \rightarrow (x_0, y_0)} f(x,y) \ .\]

Proof

Step 1. By 2. (see definition of uniform convergence), for \(\forall \varepsilon > 0\), \(\exists \delta_1 > 0\) s.t. \(| f(x,y) - g(x) | < \frac{\varepsilon}{3}\) for \(\forall y \in V_{y_0, \delta_1}\), \(\forall x \in U_{x_0}\). Take \(y_1, y_2 \in V_{y_0, \delta_1}\), then

\[|f(x,y_1) - f(x,y_2)| \le |f(x,y_1) - g(x)| + |g(x) - f(x,y_2)| < \frac{2}{3} \varepsilon \ .\]

As \(x \rightarrow x_0\), from 1.,

\[\lim_{x \rightarrow x_0} |f(x,y_1) - f(x,y_2)| = | \varphi(y_1) - \varphi(y_2) | \le \frac{2}{3} \varepsilon \ .\]

Thus, as \(\varphi(y)\) satisfies the Cauchy condition, the limit \(\lim_{y \rightarrow y_0} \varphi(y) =: L\) exists.

Step 2. Shrinking \(V_{y_0, \delta_1}\) to \(V_{y_0, \delta_2}\) so that \(| \varphi(y) - L | < \frac{\varepsilon}{3}\) for \(\forall y \in V_{y_0, \delta_2}\). From the definition of \(\varphi(y)\), then \(\exists \delta_x > 0\) s.t.

\[|f(x,y) - \varphi(y)| < \frac{\varepsilon}{3} \quad , \quad \forall 0 < | x - x_0 | < \delta_x \ .\]

Thus

\[| g(x) - L | \le | g(x) - f(x,y) | + | f(x,y) - \varphi(y) | + | \varphi(y) - L | < \frac{\varepsilon}{3} + \frac{\varepsilon}{3} + \frac{\varepsilon}{3} = \varepsilon \ , \]

i.e. \(g(x) \rightarrow L\) as \(x \rightarrow x_0\).

Counter-example. \(f(x,y) = \frac{x^2}{x^2 + y^2}\) on \((x,y) \ne (0,0)\) for \(x_0 = y_0 = 0\). This function is not uniformly convergent as \(y \rightarrow y_0\).

…

Pointwise and uniform convergence

\[\lim_{y \rightarrow y_0} f(x,y) = g(x) \ .\]

Pointwise. For \(\forall \varepsilon > 0\), \(\forall x \in X\), \(\exists \delta > 0\) s.t. \(| f(x,y) - g(x) | < \varepsilon\) for \(\forall y \in Y_{y_0, \delta}\). Here \(\delta( \varepsilon, x)\).

Uniform. For \(\forall \varepsilon > 0\), \(\exists \delta > 0\) s.t. \(| f(x,y) - g(x) | < \varepsilon\) for \(\forall y \in Y_{y_0, \delta}\), \(\forall x \in X\), . Here \(\delta( \varepsilon)\). This can be written in terms of \(\sup | f(x,y) - g(x) |\) as well, as it must hold for \(\forall x \in X\).

Cauchy criterion

Let \(I \subseteq \mathbb{R}\) be an interval containing a limit point \(x_0\), and let \(f: I \setminus \{x_0\} \to \mathbb{R}\).

The limit \(\lim_{x\to x_0} f(x)\) exists in \(\mathbb{R}\) if and only if for every \(\varepsilon > 0\), there exists a deleted neighborhood \(U = (x_0 - \delta, x_0 + \delta) \setminus \{x_0\}\) such that for all \(x_1, x_2 \in U\),

\[\vert{}f(x_1) - f(x_2)\vert{} < \varepsilon \ .\]

Proof.

Direction \((\Rightarrow)\): if the limit exists, the Cauchy condition holds. Assume \(\lim_{x \to x_0} f(x) = L\) exists. Fix \(\varepsilon > 0\). By definition of a functional limit, there exists \(\delta > 0\) such that for all \(x \in I\) with \(0 < \vert{}x - x_0\vert{} < \delta\):

\[\vert{}f(x) - L\vert{} < \frac{\varepsilon}{2}\]

Take any \(x_1, x_2 \in I\) such that \(0 < \vert{}x_1 - x_0\vert{} < \delta\) and \(0 < \vert{}x_2 - x_0\vert{} < \delta\). By the triangle inequality:

\[\vert{}f(x_1) - f(x_2)\vert{} = \vert{}(f(x_1) - L) - (f(x_2) - L)\vert{} \le \vert{}f(x_1) - L\vert{} + \vert{}f(x_2) - L\vert{}\]

Substitution gives \(\vert{}f(x_1) - f(x_2)\vert{} < \frac{\varepsilon}{2} + \frac{\varepsilon}{2} = \varepsilon\). Thus, the condition holds. \(\square\)

Direction \((\Leftarrow)\): if the Cauchy condition holds, then the limit exists.

…todo…

Bolzano-Weierstrass theorem

Every bounded sequence of real numbers has a convergence subsequence, i.e. let \(( x_n )_{x=1}^{+\infty}\) be a sequence in \(\mathbb{R}\). If \(\exists a, b \in \mathbb{R}\) s.t. \(a \le x_n \le b\), then \(( x_n )_{n=1}^{+\infty}\) contains a convergent subsequence \((x_{n_k})_{k=1}^{+\infty}\)

Proof. Take the original interval \(I_0 = [a,b]\), and split it with its midpoint \(c_0 = \frac{a+b}{2}\). (At least) one of the two sub-intervals contains infinitey many terms of the original sequence. Choose this sub-interval \(I_1\), and repeat the process. Select the sub-sequence with these rules: pick \(x_{n_1} \in I_1\); pick \(x_{n_2} \in I_2\) s.t. \(n_2 > n_1\) (always possible, as there’s a infinite number of terms); pick \(x_{n_3} \in I_3\) s.t. \(n_3 > n_2\);… The sequence of the left endpoints \(( a_k )\) is non-decreasing and bounded above by \(b\), so by monotone convergence theorem, then \(\lim_{k \rightarrow +\infty} a_k = L\), for some \(L \in \mathbb{R}\). The series of right endpoints satisfies \(b_k = a_k + \frac{b-a}{2^k}\) and thus \(\lim_{k \rightarrow +\infty} b_k = L\) as well. Thus, as \(a_k \le x_{n_k} \le b_k\), it follows (squeeze theorem) that \(\lim_{k \rightarrow +\infty} x_{n_k} = L\). \(\blacksquare\)

From \(\ < \ \) to \(\ \le \ \) in limits

Let \(f(x) < C\) for \(x \in U_{x_0}\). Then \(L := \lim_{x \rightarrow x_0} f(x) \le C\).

Proof. By contradiction, let’s assume \(L > C\). 1) Define \(\varepsilon := L - C > 0\); 2) apply the definition of limit. For \(\forall \varepsilon > 0\), \(\exists \delta > 0\) s.t. \(| f(x) - L | < \varepsilon\) for \(\forall y \in U_{x_0, \delta}\); 3) this implies \(L - \varepsilon < f(x) < L + \varepsilon\); 4) replacing \(\varepsilon = L - C\), it follows \(C < f(x) < - C\), and this contains a contradiction with the assumption of the theorem \(f(x) < C\). It follows that the premise \(L > C\) is false, and thus \(L \le C\). \(\blacksquare\)

8.2. Interchange limit and integral

8.2.1. Riemann integral

If

• \([a,b]\) compact

• \(f(\cdot, y)\) Riemann integrable on \([a,b]\) for each \(y\)

• \(f(x,y) \rightarrow g(x)\) uniformly as \(y \rightarrow y_0\) uniformly for \(x \in [a,b]\)

Then

• \(g\) is Riemann integrable and

  \[\lim_{y \rightarrow y_0} \int_{x=a}^{b} f(x,y) \, dx = \int_{x=a}^{b} g(x) \, dx \ .\]

Proof

Integrability of \(g(x)\). Given \(\varepsilon > 0\), there exists a neighborhood of \(y_0\) so that

\[\sup_{x \in [a,b]} |f(x,y) - g(x)| < \frac{\varepsilon}{4(b-a)} \ .\]

Since \(f(\cdot, y)\) is Riemann integrable, it’s possible to choose a partition \(P\) so that \(U(f(\cdot,y),P) - L(f(\cdot,y),P) < \frac{\varepsilon}{2}\), with

\[\begin{split}\begin{aligned} L(f(x),P) & = \sum_{n} \inf_{x\in I_n} f(x) ( x_{n} - x_{n-1} ) \\ U(f(x),P) & = \sum_{n} \sup_{x\in I_n} f(x) ( x_{n} - x_{n-1} ) \\ \end{aligned}\end{split}\]

As \(\sup g = \sup ( f + g - f ) \le \sup f + \sup |f-g|\), \(\inf g = \inf ( f + g - f ) \le \inf f - \sup | f-g |\),

\[U(g,P) - L(g,P) \le U(f,P) - U(f,P) + 2 \sup|f-g| (b-a) < \frac{\varepsilon}{2} + \frac{\varepsilon}{2} = \varepsilon \ .\]

For Riemann crieterion, \(g\) is Riemann integrable. Then

\[\left| \int_{a}^{b} f(x,y) \, dx - \int_{a}^{b} g(x) \, dx \right| \le \int_{a}^{b} \left| f(x,y) - g(x) \right| \, dx \le (b-a) \sup_{x \in [a,b]} | f(x,y) - g(x) | < \frac{varepsilon}{4} \ ,\]

and thus

\[\lim_{y \rightarrow y_0} \int_{a}^{b} f(x,y) \, dx = \int_{a}^{b} \lim_{y \rightarrow y_0} f(x,y) \, dx \ .\]

Counter-example. \(f(x,y) = y^2 x ( 1 - x )^y\), for \(x \in [0,1]\), for \(y \rightarrow +\infty\).

Details

As \(y \rightarrow + \infty\), \(f(x,y) \rightarrow 0\) for all \(x \in [0,1]\), and thus \(\int_{0}^{1} \lim_{y \rightarrow +\infty} f(x,y) \, dx = 0\). On the other hand,

\[\begin{split}\begin{aligned} \int_{0}^{1} y^2 x ( 1 - x )^y \, dx & = (u = 1 - x, \ du = -dx) \\ & = \int_{0}^{1} y^2 ( 1 - u ) u^y du = \\ & = y^2 \left.\left( \frac{1}{y+1} u^{y+1} - \frac{1}{y+2} u^{y+2} \right)\right|_{u=0}^{1} = \\ & = \frac{y^2}{(y+1)(y+2)} \ , \end{aligned}\end{split}\]

and thus \(\lim_{y \rightarrow +\infty} \int_{0}^{1} f(x,y) \, dx = 1\).

This occurs because \(f(x,y)\) doesn’t converge uniformly as \(y \rightarrow +\infty\). This can be easily proved as:

• the function \(f(x,y)\) converges pointwise to \(0\) as \(y \rightarrow +\infty\)

• but

  \[\sup_{x \in [0,1]} | f(x,y) - 0 | = y^2 \left( 1 - \frac{1}{1+y} \right)^{y+1} \rightarrow +\infty \, \text{as $y \rightarrow + \infty$}\]

since

\[\partial_x f(x,y) = y^2 \left[ (1-x)^y - y x ( 1 - x )^{y-1} \right] = y^2 ( 1 - x )^{y-1} ( 1 - x - x y )\]

\[0 = \partial_x f(\overline{x}, y) \quad \Rightarrow \quad \overline{x}_1 = 1 \ , \quad \overline{x}_2 = \frac{1}{1+y}\]

\[f(\overline{x}_2, y) = y^2 \left( 1 - \frac{1}{1+y} \right)^{y+1} \]

as \(\lim_{y \rightarrow +\infty} \left( 1 - \frac{1}{1+y} \right)^{y+1} = e^{-1}\), it follows that \(\lim_{y \rightarrow +\infty} f(\overline{x}_2, y) = +\infty\). Thus \(\lim_{y \rightarrow + \infty} \sup_{x \in [0,1]} | f(x,y) - 0 | = +\infty\), and \(f(x,y)\) doesn’t converge uniformly to \(0\).

Riemann integrable function

\[\int_{x=a}^{b} f(x) \,\ dx = \lim_{\Delta x \rightarrow +\infty} \sum_{n=1}^{N} f(\xi_n) ( x_{n} - x_{n-1} ) \ ,\]

with \(\xi_n \in [x_n, x_{n+1}]\) with \(x_0 = a\), \(x_N = b\).

Riemann criterion for integrability

8.2.2. Lebesgue integral

…

Counter-example. \(f(x,y) = y \cdot 1_{\left(0,\frac{1}{y}\right)}(x)\) for \(x \in [0,1]\), for \(y \rightarrow +\infty\).

8.3. Interchange limit and derivative

• \(I\) open interval

• \(f(\cdot,y)\) differentaible on \(I\) for \(\forall y \in V_{y_0}\), punctured

Suppose:

1. \(f(x_1,y)\) converges as \(y\rightarrow y_0\) for at least one point \(x_1 \in I\)

2. \(f_x(x,y) \rightarrow h(x)\) uniformly as \(y\rightarrow y_0\) for \(x\) in every compact subinterval \(J \subseteq I\) (This is required for differentiation as \(I\) is open)

Then:

• \(f(x,y)\) converges locally uniformly for \(\forall x \in I\) to some \(g(x)\), as \(y \rightarrow y_0\)

• \(g(x)\) is differentiable on \(I\) and

  \[\frac{d}{dx} \left( \lim_{y \rightarrow y_0} f(x,y) \right) = g'(x) = h(x) = \lim_{y \rightarrow y_0} \frac{\partial f}{\partial x}(x,y) \ .\]

Proof

Let (and fix) \(J \subset I\) compact, with \(x_1 \in J\), and length \(|J|\).

Step 1. 1. and 2. implies that \(f(\cdot, y)\) converges uniformly to some \(g(x)\) for \(x \in J\). By 2., for all \(\varepsilon > 0\), \(\exists V_{y_0, \delta}\) so that (Cauchy criterion)

(8.1)\[\sup_{\xi \in J}|f_x(\xi,y) - f_x(\xi,y')| < \frac{\varepsilon}{2 (|J| + 1)} \ ,\]

for all \(y, \, y' \in V_{y_0}\). Applying Mean Value Theorem to \(\Phi(x) := f(x,y) - f(x,y')\), differentiable for \(x \in I\), there’s a \(\xi \in [x, x_1]\) so that

\[\Phi(x) - \Phi(x_1) = \Phi'(\xi) \, (x - x_1) = \left[ f_x'(\xi, y) - f_x'(x,y) \right] ( x - x_1 ) \ ,\]

and

\[\left| f(x,y) - f(x,y') \right| \le \left| f(x_1,y) - f(x_1,y') \right| + \left| \left[ f_x'(\xi, y) - f_x'(x,y) \right] ( x - x_1 ) \right|\]

By 1., for \(\forall \varepsilon > 0\), there’s a \(V_{y_0,\delta_1}\) (\(\subseteq V_{y_0,\delta\), so that the uniform convergence of \(f_x(\cdot, y)\) still holds) so that \(\left| f(x_1, y) - f(x_1, y') \right| < \frac{\varepsilon}{2}\). Using (8.1), the second term is bounded by \(\frac{\varepsilon}{2(|J|+1)}|J| < \frac{\varepsilon}{2}\), and thus for all \(x \in J\)

\[\left| f(x,y) - f(x,y') \right| < \varepsilon \ ,\]

i.e. \(f(x,y)\) converges uniformly for \(x \in J\) (to some \(g(x)\)) as \(y \rightarrow y_0\). todo Discuss how \(g(x)\) can be defined “interval-wise” through uniform convergence on all the \(J \subset I\).

Step 2. Differentiability of \(g\), through Moore-Osgood. As the derivative can be defined as the limit of the incremental ratio, the limit of a derivative and the derivative of a limit involve two limits on different variables. Thus, the possibility of interchanging limit and derivative operators can be proved trhough Moore-Osgood theorem.

Fix \(x \in I\). For \(t \ne 0\) so that \([x, x+t] \in I\) (or \([x+t,x] \in I\)), define the incremental ratios

\[Q(t,y) := \frac{f(x+t,y)-f(x,y)}{t} \quad , \quad \varphi(t) := \frac{g(x+t) - g(x)}{t} \ .\]

Let’s check the assumptions of Moore-Osgood theorem for the limits \(\lim_{t \rightarrow 0}\), \(\lim_{y \rightarrow y_0}\) applied to \(Q\)

• Assumption 1. of M-O theorem For every \(y \in V_{y_0}\), \(\lim_{t \rightarrow 0} Q(t,y)\) exists for all \(x \in J \subset I\), and it’s equal (by definition) to \(\lim_{t \rightarrow 0} Q(t,y) = f_{x}(x,y)\)

• Assumption 2. of M-O theorem (\(Q(t,y) \rightarrow \varphi(t)\) uniformly as \(y \rightarrow y_0\) for \(t \in T_0\) punctured). For every fixed \(t\), \(Q(t,y) \rightarrow \varphi(y)\) as \(f(\cdot,y) \rightarrow g(\cdot)\) as \(y \rightarrow y_0\) (and this convergence holds for \(x\), \(x+t\)). In order to proof that the convergence is uniform in \(t\), let’s apply MVT to \(f(x+t,y) - f(x+t,y')\)

  \[\left| Q(t,y) - Q(t,y') \right| = \left| \frac{f(x+t,y) - f(x,y) - f(x+t,y') + f(x,y')}{t} \right| = \left| f_x(\xi,y) - f_x(\xi,y') \right| \ ,\]

  for some \(\xi \in [x, x+t]\). By 2., for \(\forall \eta > 0\) \(\exists V_{y_0}\) (and Cauchy criterion) so that \(|f_x(\xi, y) - f_x(\xi,y')| < \eta\) for \(y, y' \in V_{y_0}\), independently of \(t\). Thus \(Q(t, \cdot)\) converges uniformly in \(t\) as \(y \rightarrow y_0\). Since \(Q(t,y) \rightarrow \varphi(t)\) pointwise for \(y \rightarrow y_0\), and \(Q(t,y)\) converges uniformly, thus \(Q(t,y) \rightarrow \varphi(t)\) uniformly in \(t\) for \(y \rightarrow y_0\).

As the two assumptions of Moore-Osgood theorem holds for \(Q(t,y)\), the theorem implies that

• \(\lim_{y \rightarrow y_0} f_x(x,y)\) exists

• \(\lim_{t \rightarrow 0} \varphi(t)\) exists

• and

  \[h(x) := \lim_{y \rightarrow y_0} f_x(x,y) = \lim_{t \rightarrow 0} \varphi(t) = \dfrac{d}{dx} g(x) = \dfrac{d}{dx} \left( \lim_{y \rightarrow y_0} f(x,y) \right) \ . \quad \blacksquare\]

Counter-example. \(f(x,y) = \frac{\sin(yx)}{\sqrt{y}}\), with \(x \in \mathbb{R}\) and \(y \rightarrow +\infty\).

The function \(f(x,y) \rightarrow 0\) uniformly as \(y \rightarrow +\infty\) (it converges pointwise to \(0\) and \(\sup |f(x,y) - 0| = \frac{1}{\sqrt{y}} \rightarrow 0\) as \(y \rightarrow + \infty\)).

The derivative \(\partial_x f(x,y) = \sqrt{y} \cos(yx)\) doesn’t converge to any function as \(y \rightarrow + \infty\), and thus it doesn’t converge uniformly neither. It meaningless even to try evaluating its derivative.

8.4. Extra

Uniqueness of the theorem. If \(\lim_{x \rightarrow x_0} f(x) = L_1\), and \(\lim_{x \rightarrow x_0} f(x) = L_2\), then \(L_1 = L_2\).