Those State entities were well-understood, and pre-existed the Constitution for decades. They were required by the Articles of Confederation, and "inherited" into the new govt organization.
AoC
"; but every State shall always keep up a well-regulated and disciplined militia, sufficiently armed and accoutered,"
The militia were of the state (today there'd be 50), and they HAD TO BE maintained - well-armed and well trained (well-regulated militias were "necessary"

. And rightly so - OUR freedoms as outlined in the new Constitution depended on them, made from the body of the people, the militia had very vital roles to fill when called into federal service.
"We the People of the United States, in Order to form a more perfect Union, establish Justice, insure domestic Tranquility, provide for the common defense, promote the general Welfare, and secure the Blessings of Liberty to ourselves and our Posterity,
"Congress shall have power...
...To provide for calling forth the Militia to execute the Laws of the Union, suppress Insurrections and repel Invasions;"
...To provide for organizing, arming, and disciplining, the Militia, and for governing such Part of them as may be employed in the Service of the United States"
"The United States shall guarantee to every State in this Union a Republican Form of Government, and shall protect each of them against Invasion; and on Application of the Legislature, or of the Executive (when the Legislature cannot be convened), against domestic Violence."
These days just about all of us would be subject to 'exercise and service', not just select "gun bearers". "each and every free able-bodied white male citizen..." would of course be much expanded.
The arms we'd need to supply ourselves would be the very ones many want to ban (ARs and hi-cap mags, M-9 pistols etc).