How should my customer enter a payment reference if their bank allows only a few characters?

Check whether the account instructions require an exact reference. If they do, ask Support before shortening it. A bank's character limit is not permission to remove a required part of a receiving instruction.

If you are using your own invoice number as the reference, agree a short, unique form that you can link back to the full invoice. Keep a written record of that mapping and tell the customer the exact value to enter. Do not reuse the same short reference for unrelated payments.

If the bank strips characters automatically, ask the customer to share the final receipt so you know what was actually sent. Keep that receipt with the invoice.