You can safely delete these bills, as they haven’t been posted, so there are no concerns regarding financial or VAT positions.
So you can delete a rejected bill in the Tasks screen by:
Right-click on Rejected Bill in Tasks.
Select Edit Item.
Select Edit Bill.
Select Delete.