Right now i can only upgrade one at a time and manually have to select another upgrade after it finishes. How do i make it so that i can have a max 5 upgrades queued up at a time?

Most probably have a fiddle around with the Requirements in the data editor.

Something like Melee Weapons Level 2 will require Melee Weapons Level 1 at least in progress.

Add queue(5) to your upgrade building's abilities.
It gives the building 5 slots to queue up units or upgrades.