Arbuscular mycorrhizal (AM) fungi are key components of most agricultural ecosystems. Therefore, understanding the impact of agricultural practices on their community structure is essential to improve nutrient mobilization and reduce plant stress in the field. The effects of five different organic or mineral sources of phosphorus (P) for a maize–soybean rotation system on AM fungal diversity in roots and soil were assessed over a 3-year period. Total DNA was extracted from root and soil samples collected at three different plant growth stages. An 18S rRNA gene fragment was amplified and taxa were detected and identified using denaturing gradient gel electrophoresis followed by sequencing. AM fungal biomass was estimated by fatty acid methyl ester analysis. Soil P fertility parameters were also monitored and analyzed for possible changes related with fertilization or growth stages. Seven AM fungal ribotypes were detected. Fertilization significantly modified soil P flux, but had barely any effect on AM fungi community structure or biomass. There was no difference in the AM fungal community between plant growth stages. Specific ribotypes could not be significantly associated to P treatment. Ribotypes were associated with root or soil samples with variable detection frequencies between seasons. AM fungal biomass remained stable throughout the growing seasons. This study demonstrated that roots and soil host distinct AM fungal communities and that these are very temporally stable. The influence of contrasting forms of P fertilizers was not significant over 3 years of crop rotation.