Can I delete a transaction from a budget so it is not taken into account?

Yes, that's possible. You can change the transaction category or completely exclude the transaction so it's no longer included in your budget analysis and budget.