About the terminology “spectrum” in mathematics and physics Dieudonné writes:

It finally dawned upon them [the physicists in the 1920s] that their “observables” had properties which made them look like hermitian operators in Hilbert space, and that, by an extraordinary coincidence, the “spectrum” of Hilbert (a name which he had apparently chosen from a superficial analogy) was to be the central conception in the explanation of the “spectra” of atoms.¹⁾

We start with two examples of the most familiar Hamiltonians in QM:

Example. The free Hamiltonian $T = -\Delta$ is symmetric on the space of all infinitely-differentiable functions with compact support $\mathcal{C}^\infty_0(\Omega)$ and self-adjoint on either $H^2(\Omega)\cap H^1_0(\Omega)$ (zero boundary conditions), $H^2_p(\Omega)$ (periodic boundary conditions) or $H^2(\mathbb{R}^n)$ (no boundary). The spectrum of this operator on $H^2(\mathbb{R}^n)$ is the continuum $[0,\infty)$ but it is not possible to find any eigenstates in the Hilbert space $L^2(\mathbb{R}^n)$. One could also assign a bilinear form $\langle \varphi,-\Delta \psi \rangle = \langle \nabla \varphi,\nabla \psi \rangle$ defined on $H^1 \times H^1$ to the Hamiltonian which defines the so-called energetic extension of $T$ as the representing operator of this bilinear form.

Example. Now take the hydrogen Hamiltonian $H = -\Delta -1/|x|$ on $L^2(\mathbb{R^3})$ with its well known spectrum that is discrete for $E<0$ and continuous above. The domain for the Hamiltonian to be self-adjoint will be discussed later. We already have an infinite sequence of eigenstates assigned to the negative eigenvalues, so do we need the full spectrum to represent an arbitrary state? Yes! Take a little gaussian blob $\psi$ far away from the origin, then the potential energy is clearly $\approx 0$ while the kinetic energy can be made arbirarily large. So we definitely need parts of the Hilbert space with positive energy expectation value to represent this state.

(Note that a pure point spectrum, i.e. only discrete values, is assured for compact, self-adjoint operators by the Hilbert-Schmidt theorem.)

Diagonalizability follows from the spectral theorem, so what does the spectrum itself has to do with this process? The link between eigenvectors and the diagonalization of a matrix through a transformation into the eigenbasis is clear. But as we have seen, in the setting of infinite-dimensional vector spaces the eigenvectors do not necessarily span the whole space. So we can only try to approximate the missing section by a normalized but not necessarily converging sequence of vectors $\varphi_i$ that are almost eigenvectors of an operator $A$, i.e. \[ A\varphi_i = \lambda\varphi_i + \varepsilon_i, \quad \varepsilon_i \rightarrow 0, \] equivalent to \[ (A-\lambda\mathrm{id})\varphi_i = \varepsilon_i \] which formally leads to \[ \varphi_i = (A-\lambda\mathrm{id})^{-1}\varepsilon_i. \]

One discerns four scenarios:

1. $A-\lambda\mathrm{id}$ is bijective with bounded inverse, thus $\varphi_i \rightarrow 0$ and no approximation was found, so $\lambda$ is not counted part of the spectrum but of the resolvent set.
2. $A-\lambda\mathrm{id}$ is not injective, i.e. $\mathrm{ker}(A-\lambda\mathrm{id}) \neq \{0\}$, so there is a constant sequence $\varphi_i$ with $\varepsilon_i=0$ (no approximation needed) and we call $\lambda$ part of the point spectrum with a respective eigenspace.
3. $A-\lambda\mathrm{id}$ is injective but not surjective, yet $A-\lambda\mathrm{id}$ has a dense range which means we can define a densly defined but unbounded inverse $(A-\lambda\mathrm{id})^{-1}$ and calculate an approximate sequence $\varphi_i$ for zero sequences $\varepsilon_i$ coming from all possible directions. The same holds if $\lambda$ is slightly changed, so it is part of the continuous spectrum.
4. Finally the last case might hold with $A-\lambda\mathrm{id}$ having a non-dense range, so the $\varepsilon_i$ are limited to certain subspaces and $\lambda$ is called part of the residual spectrum. This is possible for bounded operators but not for self-adjoint ones.

Note. The spectral theorem and the Lebesgue decomposition theorem allow a different partitioning of the spectrum for normal operators into an absolutely continuous, singular continuous, and pure point part.

For a self-adjoint operator it holds that all $z \in \mathbb{C}$ with $\Im z \neq 0$ are in the resolvent set and the bounded operator $(A-z\mathrm{id})^{-1}$ is called a resolvent that establishes a very useful link to compex analysis. The norm of this operator obeys the following estimate:

Lemma. $z \in \mathbb{C}, \Im z \neq 0$, and $A$ self-adjoint, then $\|(A-z\mathrm{id})^{-1}\| \leq |\Im z|^{-1}$.

Proof. Take $\varphi \in D(A)$, the domain of the operator. Then \begin{align*} \|(A-z\mathrm{id})\varphi\|^2 &= |z|^2\|\varphi\|^2 + \|A\varphi\|^2 - 2(\Re z) \langle \varphi,A\varphi \rangle\\ &\geq |z|^2\|\varphi\|^2 + \|A\varphi\|^2 - 2|\Re z| \cdot |\langle \varphi,A\varphi \rangle|. \end{align*} By the Cauchy–Schwarz inequality $|\langle \varphi,A\varphi \rangle| \leq \|\varphi\| \cdot \|A\varphi\|$, so \begin{align*} \|(A-z\mathrm{id})\varphi\|^2 &\geq |z|^2\|\varphi\|^2 + \|A\varphi\|^2 - 2|\Re z| \cdot \|\varphi\| \cdot \|A\varphi\| \\ &= (\|A\varphi\| - |\Re z| \cdot \|\varphi\|)^2 + |\Im z|^2 \|\varphi\|^2 \geq |\Im z|^2 \|\varphi\|^2. \end{align*} It holds that an inverse operator $(A-z\mathrm{id})^{-1}$ exists with dense domain (proof omitted here), thus taking $(A-z\mathrm{id})\varphi = \psi$ we can argue for a bounded inverse that fulfills \[ |\Im z|^{-1} \|\psi\| \geq \|(A-z\mathrm{id})^{-1}\psi\|. \Box \]

» The abstract Cauchy problem in Banach space