You can turn off the system music that plays on the PS Vita by going to the [Settings] App and selecting [Sound & Display] unchecking the [System Music] option. This will stop the music playing on the main PS Vita menu system, but not within other apps.