Skip to content

FEAT: Converter Panel for GUI!#1471

Open
jbolor21 wants to merge 48 commits intomicrosoft:mainfrom
jbolor21:bjagdagdorj/frontend_converters
Open

FEAT: Converter Panel for GUI!#1471
jbolor21 wants to merge 48 commits intomicrosoft:mainfrom
jbolor21:bjagdagdorj/frontend_converters

Conversation

@jbolor21
Copy link
Copy Markdown
Contributor

@jbolor21 jbolor21 commented Mar 15, 2026

Description

Adding converter panels to the GUI interface! This PR only lets you run ONE converter per turn per type of media (ie you can do an image + text conversion in same turn but not two text conversions in one turn)

  • Created converter panel where you can select converter
  • Each converter shows a description of the converter & its parameters (note only simple parameters are editable with UI)
  • Visual badges for see which converters use an LLM - these LLMs will NOT automatically render a preview but will if you click "preview". The rest will auto show the preview!
  • Converter panel uses tabs to let you select which modality you want converter on and filters list based on your input type

Tests and Documentation

  • Manual testing with the UI
  • Frontend unit tests pass (npx jest) & added new e2e tests for new functionality
  • Added unit tests & ensured all unit tests pass

Images:
(new button is shown):
image

(new panel)
image

(panel dropdown)
image

(once you've selected converter)
image

(tabs for different modalities of converters - you can have 1 per mode per turn)
image

@jbolor21 jbolor21 marked this pull request as draft March 15, 2026 19:24
Bolor and others added 28 commits March 25, 2026 10:15
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@jbolor21 jbolor21 force-pushed the bjagdagdorj/frontend_converters branch from 446c643 to 69d038f Compare March 25, 2026 17:16
}

const ChatInputArea = forwardRef<ChatInputAreaHandle, ChatInputAreaProps>(function ChatInputArea({ onSend, disabled = false, activeTarget, singleTurnLimitReached = false, onNewConversation, operatorLocked = false, crossTargetLocked = false, onUseAsTemplate, attackOperator, noTargetSelected = false, onConfigureTarget }, ref) {
const ChatInputArea = forwardRef<ChatInputAreaHandle, ChatInputAreaProps>(function ChatInputArea({ onSend, disabled = false, activeTarget, singleTurnLimitReached = false, onNewConversation, operatorLocked = false, crossTargetLocked = false, onUseAsTemplate, attackOperator, noTargetSelected = false, onConfigureTarget, onToggleConverterPanel, isConverterPanelOpen = false, onInputChange, onAttachmentsChange, convertedValue, originalValue: _originalValue, onClearConversion, onConvertedValueChange, mediaConversions = [], onClearMediaConversion }, ref) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ik you didn't write this but do you know what this forward ref is doing ? I'm not super familiar with forwardref

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets parent component get direct reference to things inside ChatInputArea - here the parent passes ref to CHatInputArea and then the useImperativeHandle exposes the addAttachment and setText methods

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

couldn't we use a callback so the parentcomponent passes a callback and then the child component calls that callback in addAttachment & setText methods. I'm guessing the parent component needs the attachment & text information so that could be sent back in the callback. but maybe i'm missing something

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"A callback goes child → parent ("here's my new state"), but these cases need parent → child ("set your state to this"). To do it with callbacks you'd have to lift the state up — move input and state into 'ChatWindow' and pass them down as props. That would work, but it means:

  • ChatWindow(already large) takes on more state management & every keystroke in the textarea triggers a re-render of ChatWindow

The useImperativeHandle pattern is here because the parent needs to imperatively poke the child in response to discrete events (error, button click), not continuously sync state. It keeps the input state encapsulated in ChatInputArea

@jbolor21 jbolor21 requested a review from Copilot April 2, 2026 17:58
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new “Converter Panel” to the PyRIT GUI so users can discover prompt converters, configure basic parameters, preview outputs, and apply one converter per modality per turn, with supporting backend endpoints and test coverage.

Changes:

  • Backend: added a converter “catalog” API (types + supported IO + simple parameter schema) and enhanced preview to support non-text data types.
  • Frontend: implemented the ConverterPanel UI (tabs by modality, parameter form, preview + “use converted value”) and integrated it into ChatWindow/ChatInputArea send flow.
  • Tests: added/updated unit tests (backend + frontend) and introduced Playwright e2e coverage for converter workflows.

Reviewed changes

Copilot reviewed 22 out of 23 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
tests/unit/backend/test_converter_service.py Adds unit tests for the new converter catalog service method and updates preview-chain expectation.
tests/unit/backend/test_api_routes.py Adds route-level test for /api/converters/catalog.
pyrit/backend/services/converter_service.py Implements converter catalog generation (doc/params introspection), param coercion, and enhanced preview handling for media types.
pyrit/backend/services/attack_service.py Ensures atomic_attack_identifier stays consistent when per-message converters update the attack identifier.
pyrit/backend/routes/converters.py Adds a new GET /converters/catalog endpoint.
pyrit/backend/models/converters.py Introduces DTOs for catalog entries and parameter schemas.
frontend/src/types/index.ts Adds TS types for converter catalog and converter instances.
frontend/src/services/api.ts Adds convertersApi client functions (catalog/create/preview/etc.).
frontend/src/components/Config/TargetConfig.test.tsx Adds dialog cancel/close test coverage.
frontend/src/components/Config/CreateTargetDialog.test.tsx Makes target creation test input method consistent.
frontend/src/components/Chat/converterTypes.ts Adds shared mapping/types for piece-type ↔ data-type and conversion payloads.
frontend/src/components/Chat/ConverterPanel.tsx New converter panel UI: selection, param editing, preview, auto-preview behavior, and “use converted value”.
frontend/src/components/Chat/ConverterPanel.test.tsx Unit tests for converter panel UI behaviors (selection, params, preview, resizing, etc.).
frontend/src/components/Chat/ConverterPanel.styles.ts Styles for the converter panel per repo convention.
frontend/src/components/Chat/ChatWindow.tsx Integrates converter panel state + applies conversions into outgoing message requests.
frontend/src/components/Chat/ChatWindow.test.tsx Adds integration tests covering converter panel toggling and preview flow in chat.
frontend/src/components/Chat/ChatInputArea.tsx Adds converter toggle button, “converted” UI rows, and attachment→base64 propagation to converter panel.
frontend/src/components/Chat/ChatInputArea.test.tsx Expands unit tests for converter toggle, converted UI, and attachment conversion indicators.
frontend/src/components/Chat/ChatInputArea.styles.ts Layout updates for converted/original display and attachment rows.
frontend/package.json Adds react-async-hook and bumps frontend version.
frontend/package-lock.json Lockfile updates for new dependency/version bump.
frontend/e2e/converters.spec.ts New Playwright e2e suite covering converter panel flows.
frontend/e2e/chat.spec.ts Improves mocked message GET handling and skips a flaky/overbroad video role query test.
Files not reviewed (1)
  • frontend/package-lock.json: Language not supported

Comment on lines +611 to +619
<div key={param.name} className={styles.paramBlock}>
<span className={styles.paramLabel}>
<Text size={200} weight="semibold">{param.name}{param.required ? ' *' : ''}</Text>
{param.description && (
<Tooltip content={param.description} relationship="description">
<span className={styles.paramInfo}><InfoRegular fontSize={12} /></span>
</Tooltip>
)}
</span>
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this basically the same as InfoLabel ? https://storybooks.fluentui.dev/react/?path=/docs/components-infolabel--docs or what's the difference ?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah its similar but just a tooltip + icon pattern which "is meant to have short text and no interaction. If the content is longer and/or has interaction, then it must be an InfoButton." - the infobutton looks like you physically press the button if i'm looking correctly? this is the hover over tooltip if that makes sense!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants