Holonomy is curvature
$\newcommand{\Aut}{\mathrm{Aut}}$
The approach shown here is inspired by the writings of Eschenburg, Heintze, Jost, Karcher.
Let $E$ be a vector bundle over a smooth manifold $M$ and $\nabla$ a connection on $E$. The curvature of the connection is the section $\Omega$ of $\bigwedge^2T^*M\otimes \Aut(E)$ such that \begin{equation}\label{curvature} \Omega(X,Y)e = ([\nabla_X,\nabla_Y] - \nabla_{[X,Y]})e \in E_x, \end{equation} for any $x \in M$, $X, Y \in T_xM$, $e \in E_x$.
Given a smooth curve $c: [0,1] \rightarrow M$, the parallel transport of $e \in E_{c(0)}$ along $c$ is defined to be the section $f: [0,1] \rightarrow E$ such that the following hold for each $t \in [0,1]$: \begin{align*} f(t) &\in E_{c(t)}\\ f(0) &= e\\ \nabla_Tf(t) &= 0, \end{align*} where $T = \partial_t$. Denote $P_ce = f(1)$.
Let $c: [0,1] \rightarrow M$ be a $C^1$ null-homotopic curve based at $x$. There exists a $C^1$ map $C: [0,1]\times[0,1]\rightarrow M$ satisfying the following for each $0 \le s, t \le 1$: \begin{align*} C(0,t) &= x\\ C(1,t) &= c(t)\\ C(s,0) &= x\\ C(s,1) &= x. \end{align*}
Given $e_x \in E_x$, let $e: [0,1]\times [0,1] \rightarrow E$ be $C^2$ section of $C^*E$ satisfying the following for all $0 \le s, t \le 1$: \begin{align*} e(s,t) &\in E_{C(s,t)}\\ e(s,0) &= e_x\\ \nabla_Te(1,t) &= 0\\ \nabla_Se(s,t) &= 0, \end{align*} where $S = \partial_s$ and $T = \partial_t$. In particular, \begin{align*} e(s,1) &= P_ce_x. \end{align*}
Let $E^*$ be the dual vector bundle of $E$. Given $\varepsilon_x \in E^*_x$, let $\varepsilon: [0,1]\times[0,1] \rightarrow E^*$ satisfy the following for all $0 \le s, t \le 1$: \begin{align*} \varepsilon(s,t) &\in E^*_{C(s,t)}\\ \varepsilon(0,t) &= \varepsilon_x\\ \varepsilon(s,0) &= \varepsilon_x\\ \varepsilon(s,1) &= \varepsilon_x\\ \nabla_S\varepsilon(s,t) &= 0. \end{align*} It follows that \begin{align*} \nabla_T\varepsilon(0,t) &= 0. \end{align*}
An elegant presentation of the above can be found in section 3.1 of lecture notes by Werner Ballman. It can also be found on page 92 in Buser and Karcher's monograph, Gromov's almost flat manifolds. Thanks to @user127309 for posting these references on MathOverflow.
A corollary of this is the Ambrose-Singer theorem.