Enable conversation transcript access for Copilot Session Insights

[Copilot]

Copilot Session Insights workflow was analyzing GitHub Actions infrastructure logs, which lack agent conversation content needed for behavioral pattern analysis.

Changes

shared/copilot-session-data-fetch.md

• Fetch conversation transcripts via gh agent-task view --log instead of workflow run logs
• Extract session IDs from branch names (copilot/issue-123 → 123)
• Save as {session_number}-conversation.txt with agent internal monologue, reasoning, and tool usage
• Fallback to GitHub Actions logs if gh agent-task unavailable (requires gh CLI v2.80.0+)

copilot-session-insights.md

• Guide analysis of conversation logs: reasoning patterns, tool effectiveness, error recovery, prompt understanding
• Add conversation log access patterns and behavioral insights to extract

Example

# Before: Downloaded GitHub Actions workflow logs (infrastructure only)
gh api "repos/$REPO/actions/runs/${run_id}/logs" > logs.zip

# After: Fetch agent conversation transcript with reasoning
gh agent-task view --repo $REPO 123 --log > logs/123-conversation.txt

Conversation logs contain:

• Agent's step-by-step reasoning and planning
• Tool calls and results
• Code changes and validation attempts
• Error handling and recovery strategies

Enables true behavioral analysis instead of infrastructure metrics.

Original prompt

---

This section details on the original issue you should resolve

<issue_title>[plan] Add Conversation Transcript Access for Copilot Session Insights</issue_title>
<issue_description>## Objective

Enable Copilot Session Insights workflow to access Copilot agent conversation transcripts for behavioral pattern analysis, or redesign workflow to work with available data.

Context

From Discussion github/gh-aw#14345, the Copilot Session Insights workflow §21779587315 reported missing data:

Missing: Copilot agent session conversation transcripts (user prompts, agent responses, tool calls)

Impact: Cannot perform:

• Behavioral pattern analysis
• Prompt quality assessment
• Success factor identification

Currently only GitHub Actions infrastructure logs are available, which lack the actual conversation content needed for insights.

Approach

1. Research available data sources for Copilot agent conversations:
   • GitHub API endpoints for agent session data
   • Copilot telemetry/analytics APIs
   • Alternative data collection methods
2. Evaluate feasibility of accessing conversation transcripts:
   • Required permissions/scopes
   • API availability and stability
   • Data format and completeness
3. If conversation data is accessible:
   • Implement data collection in workflow
   • Add necessary authentication/permissions
   • Parse and analyze conversation patterns
4. If conversation data is NOT accessible:
   • Redesign workflow to work with available data (GitHub Actions logs, workflow metrics)
   • Document limitations and adjust workflow goals
   • Consider alternative approaches for gaining insights

Files to Review

• .github/workflows/copilot-session-insights.md - Workflow definition
• Copilot API documentation for available data sources
• Related data collection utilities

Acceptance Criteria

• Research completed on available Copilot conversation data sources
• Feasibility assessment documented
• If data accessible: implement data collection and analysis
• If data not accessible: redesign workflow with realistic scope
• Test workflow completes successfully with chosen approach
• Update workflow documentation with data source details and limitations

AI generated by Plan Command for discussion #14345

• expires on Feb 9, 2026, 2:05 PM UTC

Comments on the Issue (you are @copilot in this section)

• Fixes [plan] Add Conversation Transcript Access for Copilot Session Insights #14352

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.