Whoever has the administrator login information is able to modify account settings and uploads like panoramas indeed, added users to the account (invited by the administrator account) will only be able to modify uploads - Overall though, uploads are generally linked to the main account and can not be tied/restricted to individual users/main admin for example. Feel free to also check out our dedicated “Manage Uploads” KB article here for generally more info.
When it comes to any specific improvement wishes of yours you can also forward them to product management directly through our voting portal.