[chore] Add tests for hex/utf8 output

[scottnemes]

Description

Add tests for hex/utf8 output

Checklist

• I added this contribution to the changelog.md file.
• I added my name to the AUTHORS file (or it's already there).
• To lint and format the code, I ran

  uv run ruff check && uv run ruff format && uv run mypy --install-types .

[rolandwalker]

Thank you!