Yes, they absolutely do.
When Payments2Us push upgrades to your Salesforce Org, any new Fields added to Objects are not visible, even to System administrators. By having the Payments2Us Permission Set assigned, the appropriate level of permissions will ensure the System Administrator gets access to the newly added Fields and features.
Failure to have Permission Set assigned to the User the starts the Recurring Payment Process, Batch Process or Xero Sync Processor could mean these processes fail after a Push Upgrade is performed.
Please see Which Users Require Permission Sets for more details or How to setup Security for Internal Users for details on how to assign Permission Sets.