Add per-question LLM/Standard mode toggle for async peer instruction (discussion)

[sethbern]

This PR gives instructors the ability to toggle async peer instruction on and off and choose which PI questions use an LLM. This is what I have so far I would love any feedback, especially with the assignment dashboard since there have been recent changes. @bnmnetp @barbarer

• Instructors can toggle async peer mode on/off from the assignment list

  • 

• Each peer question can be set to LLM or Standard mode individually (defaults to Standard)

  • LLM option only appears when async mode is enabled
  • Edit All lets you bulk switch all questions at once
  • 
  • 

• LLM option only shows up if there's a valid API key configured

  • 

• If the API key is broken students get a fallback to standard discussion instead of an error

  • this also works if somebody is in the middle of a conversation with the llm
  • 

• Deleting an API key flips everything back to Standard and re-adding restores previous settings

• Students see a small disclaimer when chatting with the AI peer

[bnmnetp]

I'm torn, I know @barbarer wants this, but very few people use async PI and to add more compexity to this page is worrisome. I understand it is extra clicks to enable, the async, but we cannot have everything on the front page.

I'm thinking about it....

[sethbern]

I think it might make more sense and reduce interface clutter to integrate the async option into the Visibility status dropdown rather than having a separate toggle.

Instead of a standalone async toggle, what if the Visibility dropdown could include an option like "Visible + Async PI (after X time)" that enables async peer instruction when the visibility conditions are met?

[bnmnetp]

What about adding a switch to the peer instruction dashboard, so that an instructor could make the asynchronous questions available at the end of the peer instruction session?

[bnmnetp]

Why would this be on a per question basis? Shouldn't this be on an assignment ?

[sethbern]

The per-question setting is specifically for AB testing in research studies, where we need to assign different conditions to different questions within the same assignment. For regular classroom use, instructors would just set it once, they have the ability to select the same option for all questions, similar to how selecting how items will be graded works.

[sethbern]

Hey @bnmnetp, I removed the async toggle from the assignment builder and moved it to the PI instructor dashboard so it now shows up as a "Release After-Class PI" button on the last question instead of Next Question. I also fixed a question counter bug I found where clicking Next Question on the last question would save an out-of-bounds index to the database, causing it to show "Question 3 of 2".

Let me know what you think

[bnmnetp]

Thanks for the updates.

I am still struggling to understand how each question would need an async toggle.

We have select questions which are made for AB testing, but maybe I don't understand what you are trying to accomplish, or if they don't work in this context.

What I don't want to do is needlessly complicate the interface for 99% of the instructors (which will cause great confusion to most of them) to support AB testing by the 1%. So maybe we should have a zoom where we can talk about all of this so that I can fully understand.

Tagging @barbarer as I think she should be part of the conversation.