Simplify
\[\cos ^2 x + \cos^2 (x + y) - 2 \cos x \cos y \cos (x + y).\]
First, we can write
\begin{align*}
&\cos^2 x + \cos^2 (x + y) - 2 \cos x \cos y \cos (x + y) \\
&= \cos^2 x + \cos (x + y) (\cos (x + y) - 2 \cos x \cos y).
\end{align*}From the angle addition formula, $\cos (x + y) = \cos x \cos y - \sin x \sin y,$ so
\begin{align*}
&\cos^2 x + \cos (x + y) (\cos (x + y) - 2 \cos x \cos y) \\
&= \cos^2 x + \cos (x + y) (-\cos x \cos y - \sin x \sin y).
\end{align*}From the angle subtraction formula, $\cos (x - y) = \cos x \cos y + \sin x \sin y,$ so
\begin{align*}
&\cos^2 x + \cos (x + y) (-\cos x \cos y - \sin x \sin y) \\
&= \cos^2 x - \cos (x + y) \cos (x - y).
\end{align*}From the product-to-sum formula,
\begin{align*}
\cos^2 x - \cos (x + y) \cos (x - y) &= \cos^2 x - \frac{1}{2} (\cos 2x + \cos 2y) \\
&= \cos^2 x - \frac{1}{2} \cos 2x - \frac{1}{2} \cos 2y.
\end{align*}Finally, from the double-angle formula,
\begin{align*}
\cos^2 x - \frac{1}{2} \cos 2x - \frac{1}{2} \cos 2y &= \cos^2 x - \frac{1}{2} \cdot (2 \cos^2 x - 1) - \frac{1}{2} (2 \cos^2 y - 1) \\
&= 1 - \cos^2 y = \boxed{\sin^2 y}.
\end{align*}
First, we can write
\begin{align*}
&\cos^2 x + \cos^2 (x + y) - 2 \cos x \cos y \cos (x + y) \\
&= \cos^2 x + \cos (x + y) (\cos (x + y) - 2 \cos x \cos y).
\end{align*}From the angle addition formula, $\cos (x + y) = \cos x \cos y - \sin x \sin y,$ so
\begin{align*}
&\cos^2 x + \cos (x + y) (\cos (x + y) - 2 \cos x \cos y) \\
&= \cos^2 x + \cos (x + y) (-\cos x \cos y - \sin x \sin y).
\end{align*}From the angle subtraction formula, $\cos (x - y) = \cos x \cos y + \sin x \sin y,$ so
\begin{align*}
&\cos^2 x + \cos (x + y) (-\cos x \cos y - \sin x \sin y) \\
&= \cos^2 x - \cos (x + y) \cos (x - y).
\end{align*}From the product-to-sum formula,
\begin{align*}
\cos^2 x - \cos (x + y) \cos (x - y) &= \cos^2 x - \frac{1}{2} (\cos 2x + \cos 2y) \\
&= \cos^2 x - \frac{1}{2} \cos 2x - \frac{1}{2} \cos 2y.
\end{align*}Finally, from the double-angle formula,
\begin{align*}
\cos^2 x - \frac{1}{2} \cos 2x - \frac{1}{2} \cos 2y &= \cos^2 x - \frac{1}{2} \cdot (2 \cos^2 x - 1) - \frac{1}{2} (2 \cos^2 y - 1) \\
&= 1 - \cos^2 y = \boxed{\sin^2 y}.
\end{align*}
The final answer is $$\boxed{\sin^2 y}$$. I hope it is correct.