Statistics of COSMOGRAIL: the COSmological MOnitoring of GRAvItational Lenses X. Modeling based on high-precision astrometry of a sample of 25 lensed quasars: consequences for ellipticity, shear, and astrometric anomalies