Why Hilbert space?
Pragmatic answer. A Hilbert space is like an euclidean space with possibly infinite dimensions, i.e. it has a lot of structure, like that of a vector space, a scalar product, and completeness! Sometimes a Banach space (with just a norm and completeness) is enough.
Historical answer. Modern day QM started out as “Transformationstheorie” (Born–Jordan 1925, Dirac 1927) that unifies Heisenberg's matrix mechanics and Schrödinger's wave mechanics. The “Transformation” is actually the diagonalization of a matrix and refers to eigenvalue problems involving Heisenberg's infinite-dimensional matrices. Those matrices represent the usual PDOs of QM, but written as integral operators involving Dirac-deltas. \[ \partial_x^n \quad\longleftrightarrow\quad \delta^{(n)}(x-x') \]
This notation was ridiculed by von Neumann (1932) as non-mathematical (though useful) and he was able to build the equivalence of matrix and wave mechanics over Hilbert spaces. The starting point was the probability interpretation (Born 1926) that yields normalizability: \[ \int |\psi|^2 dx < \infty \quad\longleftrightarrow\quad \sum |x_n|^2 < \infty. \]
(Here the $x_n$ in the sum term can be the coordinates of the eigenstates of $H$ as an infinite matrix relative to some basis, represented as a sequence $\mathbb{N} \rightarrow \mathbb{C}$.)
The Riesz–Fischer theorem (1907) then told von Neumann, that the spaces of all such (normalized) objects are isomorphic as Hilbert spaces. \[ L^2(\Omega) \simeq \ell^2(\mathbb{N}) \] e.g., $L^2(\mathbb{R}/2\pi) \simeq \ell^2(\mathbb{Z})$ with Fourier series or with any other orthonormal basis $(e_i)_{i \in \mathbb{N}}$ of $L^2(\Omega)$: \[ \psi \quad\longleftrightarrow\quad (\langle \psi,e_i \rangle)_{i \in \mathbb{N}}. \]
The isomorphy holds on the level of state spaces but not on the level of configuration spaces, e.g. $\Omega=\mathbb{R}^3$ and $\mathbb{N}$ (the numbering of orbitals). This explains the focus of QM on the Hilbert space (state space) instead of configuration space.