In his note A Note of Shimura’s Paper “Discontinuous Groups and Abelian Varieties” Mumford constructs a Hodge type Shimura variety (a family of polarised abelian fourfolds with level structures) that is not of PEL type. He makes the necessary group from a quaternion algebra over a totally real field which is engineered to admit a symplectic representation which is absolutely irreducible (so the resulting abelian varieties cannot admit extra endomorphisms).

More specifically, let $K/\mathbb{Q}$ be a cubic totally real field. We wish to carefully choose a quaternion algebra $D/K$ such that $H:= D^{N=1}$ admits a symplectic representation.

The trick is to note that corestriction on Brauer groups associates to $D/K$ a quaternion algebra $Cor(D)/\mathbb{Q}$, equipped with a natural map $Nm:D \rightarrow Cor(D)$. In particular, if we can arrange for $Cor(D)$ to be split, we will certainly get a natural (8-dimensional) representation of $D^*$. By then imposing some conditions at infinity we can produce a symplectic structure on this representation, realising the resulting Shimura variety as one of Hodge type.

These conditions can be realised: just take $D$ to be ramified at any even number of places above each place of $\mathbb{Q}$, and at two of the three places at infinity.

Then $H(\mathbb{R}) = SU(2) \times SU(2) \times SL_2(\mathbb{R})$, and unravelling the construction one sees that the map $Nm: H(\mathbb{R}) \rightarrow SL_8(\mathbb{R})$ factors $SU(2) \times SU(2) \times SL(2) \rightarrow SO(4) \times SL(2) \rightarrow SL(8)$ where the first map is the standard degree 2 isogeny $SU(2) \times SU(2) \rightarrow SO(4)$ and the second is given by the natural action on $\mathbb{R}^4 \otimes \mathbb{R}^2$, and visibly preserves the natural symplectic pairing coming from the product of orthogonal and symplectic pairings on these two spaces. Symplectic structures descend, giving the required map

$H \rightarrow Sp_8$  over $\mathbb{Q}$.

One extends this game to a Hodge type Shimura datum by taking

$G=Nm(D^*) \subset GL_{8,\mathbb{Q}}$, which is an extension of the image of $H$ by $\mathbb{G}_m$ landing in $GSp_8$.

One takes the Hodge structure $h$ to be the composite of $Nm$ with the map taking $a+ib \in \mathbb{S}$ to $1$ on the $GU(2)\times GU(2)$ factor of $D^*_\mathbb{R}$ and the standard 2-dimensional embedding into the $GL_2(\mathbb{R})$ factor. It is obvious this extends to give the Siegel Shimura datum on $GSp_8$, so we have a Shimura datum of Hodge type, as required.