Delete mailbox for user without license - is not within a valid server write scope

After trying to remove a mailbox in Exchange online for a user that had a mailbox assigned on-prem, but then the license got revoked and the sync didn't go as it should you are left with a mailbox in Exchange Online for a user that doesn't have a license for it.

So this has to go, but when trying to delete the mailbox like so, you get the following error:

Remove-Mailbox usermailbox@domain.com
Remove-Mailbox: ExE71684|Microsoft.Exchange.Configuration.ObjectModel.ProvisioningValidationException|
The following error occurred during validation in agent 'Windows LiveId Agent': 'Unable to perform the save operation.
'usermailbox' is not within a valid server write scope.'

Since the user must be retained and only the mailbox has to be deleted we can delete just the mail user:

Connect-MsolService
Get-MsolUser -UserPrincipalName usermailbox@domain.com | Remove-MsolUser -Force

Now see what we did there:

Get-MsolUser -UserPrincipalName usermailbox@domain.com | fl
Get-MsolUser : User Not Found.  User: usermailbox@domain.com.
At line:1 char:1
+ Get-MsolUser -UserPrincipalName usermailbox@domain.com | fl
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OperationStopped: (:) [Get-MsolUser], MicrosoftOnlineException
    + FullyQualifiedErrorId : Microsoft.Online.Administration.Automation.UserNotFoundException,Microsoft.Online.Administration.Automation.GetUser

And it's gone.