Enable contrast version

# Tutor profile: Youssef A.

Inactive
Youssef A.
MIT math tutor
Tutor Satisfaction Guarantee

## Questions

### Subject:Machine Learning

TutorMe
Question:

It is common to specify a multiple regression model when, in fact, interest centers in only one or a subset of the full set of variables- the remaining variables are often viewed as "controls". Suppose that a regression involves two sets of variables, $\textbf{X}_1$ and $\textbf{X}_2$. Prove the theorem: \textbf{Firsch-Waugh-Lovell} In the linear least squares regression of vector \textbf{y} on two subsets of variables $\textbf{X}_1$ and $\textbf{X}_2$, the subvector $\textbf{b}_2$ is the set of coefficients obtained when the residuals from a regression of \textbf{y} on $\textbf{X}_1$ alone are regressed on the set of residuals obtained when each column of $\textbf{X}_2$ is regressed on $\textbf{X}_1$.

Inactive
Youssef A.

To prove this theorem, start with the normal equations \begin{gather} \begin{bmatrix} \textbf{X}^T_1\textbf{X}_1 & \textbf{X}^T_1\textbf{X}_2 \\ \textbf{X}^T_2\textbf{X}_1 & \textbf{X}^T_2\textbf{X}_2 \end{bmatrix} \begin{bmatrix} \textbf{b}_1 \\ \textbf{b}_2 \end{bmatrix} = \begin{bmatrix} \textbf{X}^T_1\textbf{y} \\ \textbf{X}^T_2\textbf{y} \end{bmatrix} \end{gather} Insert the formula for $\textbf{b}_1$ into the second normal equation and collect terms to arrive at: \begin{equation*} \textbf{b}_2=(\textbf{X}_2^T\textbf{M}_1\textbf{X}_2)^{-1}(\textbf{X}_2^T\textbf{M}_1\textbf{y}) \end{equation*} By exploiting the fact that $\textbf{M}_1$ is symmetric and idempotent, we have \begin{equation*} \textbf{b}_2=((\textbf{M}_1\textbf{X}_2)^T\textbf{M}_1\textbf{X}_2)^{-1}((\textbf{M}_1\textbf{X}_2)^T\textbf{M}_1\textbf{y}) \end{equation*} This process is commonly called partialling out or netting out the effect of $\textbf{X}_1$.

### Subject:Applied Mathematics

TutorMe
Question:

Consider a continuous function $f: \mathbb{R}^n \rightarrow \mathbb{R}$. We are interested in the unconstrained optimization problem $\min_{\mathbf{x}\in \mathbb{R}^n}f(\mathbf{x})$ (a) Prove the Necessary condition" Let $f(\mathbf{x})$ be continuously differentiable. If $\mathbf{x}^* \in \mathbb{R}^n$ is a local minimum of $f(\mathbf{x})$, then $\nabla f(\mathbf{x}^*)=\mathbf{0}$ and $\nabla^2 f(\mathbf{x}^*)\geq \mathbf{0}$ (b) Prove the sufficient condition: Let $f(\mathbf{x})$ be twice continuously differentiable. If $\nabla f(\mathbf{x}^*)=\mathbf{0}$ and $\nabla^2 f(\mathbf{x})\geq \mathbf{0} \; \forall \mathbf{x} \in B(\mathbf{x}^*, \epsilon)$ then $\mathbf{x}^*$ is a local minimum.

Inactive
Youssef A.

(a) Since $\mathbf{x}^*$ is a local minimum, for $\forall \; \mathbf{d} \in \mathbb{R}^n, \lambda \geq 0$ sufficiently small, we have \begin{align*} f(\mathbf{x}^*) \leq f(\mathbf{x}^*+ \lambda \mathbf{d}) \end{align*} Pick $\lambda >0$ \begin{align*} 0 \leq \frac{f(\mathbf{x}^*+ \lambda \mathbf{d})-f(\mathbf{x}^*)}{\lambda} \end{align*} Take limits as $\lambda \rightarrow 0$ \begin{align*} 0 \leq \nabla f(\mathbf{x}^*)^T \mathbf{d} \; \forall \mathbf{d}\in \mathbb{R}^n \end{align*} Since \textbf{d} is arbitrary, we must have $\nabla f(\mathbf{x}^*)=0$. Since we have $\nabla f(\mathbf{x}^*)=0$, consider the Taylor series expansion \begin{align*} f(\mathbf{x}^*+ \lambda \mathbf{d})-f(\mathbf{x}^*) = \frac{1}{2}\lambda^2\mathbf{d}^T\nabla^2 f(\mathbf{x}^*)d + \lambda^2||\mathbf{d}||^2R(\mathbf{x}^*;\lambda\mathbf{d}) \end{align*} Hence \begin{align*} \frac{f(\mathbf{x}^*+ \lambda \mathbf{d})-f(\mathbf{x}^*)}{\lambda^2} = \frac{1}{2}\mathbf{d}^T\nabla^2 f(\mathbf{x}^*)\mathbf{d} + ||\mathbf{d}||^2R(\mathbf{x}^*;\lambda\mathbf{d}) \end{align*} If $\nabla^2 f(\mathbf{x}^*)< \mathbf{0}$, $\exists\bar{\mathbf{d}}:\bar{\mathbf{d}}^T\nabla^2 f(\mathbf{x}^*)\bar{\mathbf{d}} <0 \implies f(\mathbf{x}^*+ \lambda \mathbf{d})-f(\mathbf{x}^*) <0$ for $\lambda$ sufficiently small. (b) Consider a Taylor series expansion for all $\mathbf{x} \in B(\mathbf{x}^*, \epsilon)$. For some $\lambda \in [0,1]$ we have \begin{align*} f(\mathbf{x})=f(\mathbf{x}^*)^T(\mathbf{x-x^*})+\frac{1}{2}(\mathbf{x-x^*})^T\nabla^2 f(\mathbf{x}^*+\lambda (\mathbf{x-x^*}))(\mathbf{x-x^*}) \end{align*} Hence $f(\mathbf{x})\geq f(\mathbf{x}^*)$.

### Subject:Calculus

TutorMe
Question:

(a) Find the equation for the tangent line to $y = \ln{x}$ at the point $x = 1$. (b) Find the quadratic approximation $Q(x)$ to the function $f(x) = \ln{x}$ at $x=1$.

Inactive
Youssef A.

(a) The equation for the tangent line is $y = L(x)$ where $L(x)$ is the linear approximation of $y = \ln{x}$ at $x = 1$. Using the formula $L(x)=f(a)+f^{'}(a)(x-a)$, with $f(x) = \ln{x}$and $a=1$ and $f^{'}(x)=1/x$ we get $f(1) = ln(1) = 0$ and $f^{'}(1)=1$ $L(x)=0+1(x-1)=(x-1)$. (b) Using the formula $#Q(x)=f(a)+f^{'}(a)(x-1)+\frac{f^{''}(x-1)^2}{2} with$f(x)=\ln(x)$,. we get$f^{'}(x)=1/x$,$f^{''}(x)=-1/x^2$so$f^{''}=-1$and hence$Q(x)=(x-1)-\frac{(x-1)^2}{2}\$

## Contact tutor

Send a message explaining your
needs and Youssef will reply soon.
Contact Youssef