diff --git a/.github/prompts/endgame.prompt.md b/.github/prompts/endgame.prompt.md deleted file mode 100644 index f9d9a969..00000000 --- a/.github/prompts/endgame.prompt.md +++ /dev/null @@ -1,158 +0,0 @@ -# Endgame Verification Prompt - -## Instructions - -**⚠️ CRITICAL - YOU MUST FOLLOW THESE RULES:** -1. **Do NOT run `gh pr view` or `gh issue view` for individual tasks** - only run it once for the main endgame issue -2. **Do NOT research or analyze any task yourself** -3. **Do NOT create any verification files yourself** -4. **IMMEDIATELY call `runSubagent` for each task** after parsing the issue - no delays, no research - -Your workflow is ONLY: -1. Create initial todo: "Fetch endgame issue and parse tasks" -2. Fetch the endgame issue (ONE `gh issue view` call) -3. Parse to get task list, then ADD a todo item for each task found -4. Create output directory -5. Call `runSubagent` for each task - mark todo complete when subagent returns - -### Steps - -1. **Create initial todo list**: - Use `manage_todo_list` to create the first todo: - - Todo 1: "Fetch endgame issue" (mark as in-progress) - -2. **Ask the user** for the following information: - - The GitHub endgame issue link (e.g., `https://github.com/microsoft/copilot-for-eclipse/issues/XXXX`) - - The user's GitHub account name - -3. **Fetch the endgame issue** (this is the ONLY `gh issue view` you should run): - ```shell - gh issue view --repo microsoft/copilot-for-eclipse - ``` - Parse the issue body to find all tasks (checkboxes) assigned to the specified user. - Extract the task title and any linked PR/issue URL as plain text. - **STOP - do NOT fetch any of the linked PRs or issues.** - -4. **Update todo list with all tasks found**: - Use `manage_todo_list` to: - - Mark "Fetch endgame issue" as completed - - ADD a new todo for each task found (e.g., "Task 1: ", "Task 2: <title>", etc.) - - All new task todos should be "not-started" - -5. **Create the output directory**: - ```shell - mkdir -p .github/endgame/<issue_number> - ``` - -6. **For each task, mark todo as in-progress, then call `runSubagent`**: - - For each task: - 1. Update todo list - mark that task as "in-progress" - 2. Call `runSubagent` tool with: - - **description**: "Endgame: <short_task_title>" - - **prompt**: The template below filled in with only the info you extracted - 3. When subagent returns, mark that task's todo as "completed" - - **Subagent prompt template**: - - --- - ## Task Details - - Task Number: <N> - - Task Title: <task_title> - - Assignee: <username> - - Related Issue/PR: <link_if_available> (NOT YET FETCHED - you must fetch this) - - ## Your Mission - YOU (the subagent) must research this task AND create the verification file. - - ### Step 1: Research the Task - - If there is a linked PR/issue, fetch it using `gh pr view` or `gh issue view` - - Understand what feature/fix needs to be verified - - **If anything is ambiguous or unclear after reading the PR/issue, ask the user - for clarification before proceeding. Keep asking until you have enough - information to write concrete, accurate test steps.** - - ### Step 2: Create Verification File (YOU must create this file) - Create the file: `.github/endgame/<issue_number>/<N>_<task_slug>.md` - - Use **exactly** this format (modelled on the project test-plan style under - `com.microsoft.copilot.eclipse.swtbot.test/test-plans/`): - - ```markdown - # <Feature / Task Title> - - ## Overview - <1-3 sentences: what is being verified and why it matters.> - - Entry points: - - <Primary UI/menu path to reach the feature> - - Not exercised: - - <Known out-of-scope items, or omit this block if none> - - --- - - ## Prerequisites - - - Eclipse IDE with the GitHub Copilot for Eclipse plugin installed and activated. - - <Additional prerequisite specific to this feature> - - --- - - ## 1. <Scenario group name, e.g. "Happy-path verification"> - - ### TC-001: <Specific test case title> - - **Type:** `Happy Path` - **Priority:** `P0` - - #### Preconditions - - <Specific state required before starting these steps> - - #### Steps - 1. <Detailed, concrete step> - 2. <Next step> - 3. <Continue as needed> - - #### Expected Result - - <Observable outcome that proves the feature works> - - <Second outcome if needed> - - #### Key Screenshots - - [ ] **<Label>** -- <What to capture in this screenshot> - - #### Notes on failure modes - - <Common failure symptom> -- <Likely cause and where to look> - ``` - - Guidelines for filling in the template: - - **Overview**: 1-3 sentences max. Do not repeat what is already covered by - Prerequisites or the Steps. - - **Type** values: `Happy Path`, `Negative`, `Edge Case`, `Regression`. - - **Priority** values: `P0` (must-pass), `P1` (high), `P2` (medium). - - Add additional `### TC-NNN` blocks (3-digit zero-padded: TC-002, TC-003, …) if multiple distinct scenarios are needed. - - Omit `#### Notes on failure modes` if there are no obvious failure modes to call out. - - If you are still unsure about any step after researching, **ask the user** - before writing that step -- do not guess. - - ### Step 3: Return Summary - Return ONLY: - - File path created - - One-line summary of what needs to be verified - --- - -7. **After all subagents complete**, provide: - - Summary table of all generated verification files - - Total tasks processed - - Any tasks that could not be processed (with reasons) - ---- - -## Notes - -- **NEVER run `gh pr view` or `gh issue view` on task links** - only on the main endgame issue -- **NEVER analyze or research tasks yourself** - immediately delegate to subagents -- Each subagent runs independently -- Subagents should be concise - create the file and return a brief summary -- If a task is unclear after asking the user, note the outstanding question in the verification file -- Use slugified task titles for filenames (lowercase, hyphens, no special chars) diff --git a/.github/skills/endgame/SKILL.md b/.github/skills/endgame/SKILL.md new file mode 100644 index 00000000..2aec5de6 --- /dev/null +++ b/.github/skills/endgame/SKILL.md @@ -0,0 +1,173 @@ +--- +name: endgame +description: Orchestrate endgame verification for a GitHub milestone issue. Fetches the issue, parses assigned tasks, delegates each task to a subagent that researches the linked PR/issue and writes a test plan, and saves every plan to test-plans/ following the project's standard test-plan format. +--- + +# Endgame Verification Skill + +Use this skill to run an endgame verification pass for a GitHub milestone issue. +Each task in the issue is delegated to a subagent that researches the linked +PR/issue and writes a test plan under `test-plans/`. + +## Workflow + +**⚠️ CRITICAL — YOU MUST FOLLOW THESE RULES:** + +1. **Do NOT run `gh pr view` or `gh issue view` for individual tasks** — only + run it once for the main endgame issue. +2. **Do NOT research or analyse any task yourself.** +3. **Do NOT create any test-plan files yourself.** +4. **IMMEDIATELY call `runSubagent` for each task** after parsing the issue — + no delays, no research. + +--- + +### Step 1 — Ask the user for inputs + +Ask for: +- The GitHub endgame issue URL + (e.g. `https://github.com/microsoft/copilot-for-eclipse/issues/XXXX`) +- The user's GitHub account name + +--- + +### Step 2 — Fetch the endgame issue (ONE call only) + +```shell +gh issue view <issue_number> --repo microsoft/copilot-for-eclipse --json body --jq '.body' +``` + +Parse the body to find **all tasks (checkboxes) assigned to the specified +user**. Extract each task's title and any linked PR/issue URL as plain text. + +**STOP — do NOT fetch any of the linked PRs or issues.** + +--- + +### Step 3 — Process each task via subagent + +For each task, call `runSubagent` with: +- **description**: `"Endgame: <short_task_title>"` +- **prompt**: the template below, filled in with only the info you extracted. + +#### Subagent prompt template + +--- +## Task Details +- Task Number: \<N\> +- Task Title: \<task_title\> +- Assignee: \<username\> +- Related Issue/PR: \<link_if_available\> (NOT YET FETCHED — you must fetch this) + +## Your Mission + +YOU (the subagent) must research this task AND create the test-plan file. + +### Step 1: Research the task + +- Fetch the linked PR or issue with `gh pr view` / `gh issue view`. +- Understand what feature or fix needs to be verified. +- **If anything is ambiguous or unclear after reading the PR/issue, ask the + user for clarification before proceeding. Keep asking until you have enough + information to write concrete, accurate test steps.** + +### Step 2: Create the test-plan file + +Determine a short `<feature-slug>` (lowercase, hyphens, no special chars) +derived from the task title (e.g. `chat-history-restore`). + +Create the directory and file: + +``` +test-plans/<feature-slug>/<feature-slug>.md +``` + +Use **exactly** this format, matching the project's existing test plans +(see `test-plans/thinking-persistence/thinking-persistence.md` for a live +example): + +```markdown +# <Feature / Task Title> + +## Overview +<1–3 sentences: what is being verified and why it matters.> + +--- + +## Test Cases + +### TC-001: <Specific test case title> + +**Type:** `Happy Path` +**Priority:** `P0` + +#### Preconditions +- <Specific state required before starting these steps> + +#### Steps +1. <Detailed, concrete step> +2. <Next step> +3. <Continue as needed> + +#### Expected Result +- <Observable outcome that proves the feature works> + +#### 📸 Key Screenshots +- [ ] **<Label>** — <What to capture> + +--- + +### TC-002: <Next scenario if needed> +<!-- Repeat TC block for each distinct scenario. Use TC-003, TC-004, … --> + +--- + +## Screenshots Checklist +> Consolidated list of all key screenshot moments. + +- [ ] `TC-001` <Label> +- [ ] `TC-002` <Label> +``` + +Guidelines: +- **Overview**: 1–3 sentences max. Do not repeat what is already covered by + Preconditions or Steps. +- **Type** values: `Happy Path`, `Negative`, `Edge Case`, `Regression`. +- **Priority** values: `P0` (must-pass), `P1` (high), `P2` (medium). +- Use 3-digit zero-padded TC numbers: TC-001, TC-002, TC-003, … +- Omit `📸 Key Screenshots` within a TC block if there are no meaningful + screenshots to capture for that case. +- Keep the `## Screenshots Checklist` section at the end — list every + screenshot across all TC blocks. +- If you are still unsure about any step after researching, **ask the user** + before writing that step — do not guess. + +### Step 3: Return a brief summary + +Return ONLY: +- File path created (e.g. `test-plans/chat-history-restore/chat-history-restore.md`) +- One-line summary of what needs to be verified + +--- + +### Step 4 — Final summary + +After all subagents complete, provide: +- A table of all generated test-plan files +- Total tasks processed +- Any tasks that could not be processed (with reasons) + +--- + +## Notes + +- **NEVER run `gh pr view` or `gh issue view` on task links yourself** — only + on the main endgame issue. Subagents handle the individual links. +- **NEVER analyse or research tasks yourself** — immediately delegate to + subagents. +- Each subagent runs independently. +- Subagents should be concise — create the file and return a brief summary. +- If a task is still unclear after the user is asked, note the outstanding + question inside the test-plan file. +- Use slugified task titles for the feature slug and directory name + (lowercase, hyphens, no special characters). diff --git a/com.microsoft.copilot.eclipse.swtbot.test/probe-scripts/endgame-chat-open-001.json b/com.microsoft.copilot.eclipse.swtbot.test/probe-scripts/endgame-chat-open-001.json new file mode 100644 index 00000000..982e6ba2 --- /dev/null +++ b/com.microsoft.copilot.eclipse.swtbot.test/probe-scripts/endgame-chat-open-001.json @@ -0,0 +1,31 @@ +[ + { "action": "waitForIdle" }, + { "action": "screenshot", "id": "01-startup" }, + + { "action": "showView", "idRef": "com.microsoft.copilot.eclipse.ui.chat.ChatView" }, + { "action": "waitForIdle" }, + { "action": "screenshot", "id": "02-chat-open" }, + + { "action": "assertExists", + "locator": { "by": "viewId", "id": "com.microsoft.copilot.eclipse.ui.chat.ChatView" } }, + + { "action": "waitFor", + "locator": { "by": "styledText" }, + "timeoutSec": 30 }, + + { "action": "dumpUi", "id": "03-chat-widgets" }, + { "action": "screenshot", "id": "03-chat-ready" }, + + { "action": "waitForMethod", + "locator": { "by": "widgetId", "value": "model-picker" }, + "method": "getSelectedItemId", + "timeoutSec": 60 }, + + { "action": "screenshot", "id": "04-model-loaded" }, + { "action": "assertExists", + "locator": { "by": "widgetId", "value": "model-picker" } }, + + { "action": "screenshot", "id": "05-final" }, + { "action": "assertExists", + "locator": { "by": "styledText" } } +] diff --git a/test-plans/cls-session-persistence/cls-session-persistence.md b/test-plans/cls-session-persistence/cls-session-persistence.md new file mode 100644 index 00000000..454c5b8b --- /dev/null +++ b/test-plans/cls-session-persistence/cls-session-persistence.md @@ -0,0 +1,197 @@ +# CLS Session Persistence and Restoration for Conversation History + +## Overview +This feature integrates CLS (Copilot Language Server) server-side session persistence so that when a conversation is restored the CLS receives the original `conversationId` and `restoreToTurnId`, enabling full context continuation rather than relying solely on IDE-side turn history. It also configures `<user home>/.copilot/eclipse` as the transcript directory (matching IntelliJ's convention). + +> **Path note:** On macOS/Linux this directory is `~/.copilot/eclipse`; on Windows it is `%USERPROFILE%\.copilot\eclipse`. + +--- + +## Test Cases + +### TC-001: Conversation history displays correctly after session switch + +**Type:** `Happy Path` +**Priority:** `P0` + +#### Preconditions +- Copilot Chat is open +- At least one conversation with multiple turns exists in history + +#### Steps +1. Create a conversation with at least 3 user–Copilot turn pairs +2. Open chat history and switch to a different conversation +3. Open chat history again and select the original conversation + +#### Expected Result +- All user and Copilot turns are displayed in the correct order +- Each turn shows its full content (no missing or truncated messages) +- Turn pairing is correct: each user message is followed by its corresponding Copilot response + +#### 📸 Key Screenshots +- [ ] **Before switch** — Full conversation with all turns visible +- [ ] **After restore** — Same conversation restored with all turns in correct order + +--- + +### TC-002: Conversation history persists across Eclipse restarts + +**Type:** `Happy Path` +**Priority:** `P0` + +#### Preconditions +- Eclipse is open with Copilot Chat functional +- The `<user home>/.copilot/eclipse` directory does not exist (or is empty) before the test + +#### Steps +1. Open Copilot Chat and send at least 2 messages; wait for responses +2. Close Eclipse completely (File → Exit or equivalent) +3. Verify that the `<user home>/.copilot/eclipse` directory exists and contains transcript files +4. Reopen Eclipse and open Copilot Chat +5. Open chat history and select the conversation from before the restart + +#### Expected Result +- The conversation appears in chat history with all turns intact +- Continuing the conversation produces contextually aware responses (CLS uses transcript for context) + +#### 📸 Key Screenshots +- [ ] **Transcript directory** — `<user home>/.copilot/eclipse` folder visible with transcript files +- [ ] **After restart** — Conversation restored in chat history after Eclipse reopen + +--- + +### TC-003: New conversation starts without interference from restored state + +**Type:** `Regression` +**Priority:** `P0` + +#### Preconditions +- At least one existing conversation in chat history + +#### Steps +1. Open chat history and restore an existing conversation +2. Click "New Chat" to start a fresh conversation +3. Send a message unrelated to any prior conversation +4. Verify Copilot responds without referencing prior conversations + +#### Expected Result +- New conversation has a clean context; no cross-contamination from the restored conversation +- CLS receives a new `conversationId` (no `restoreToTurnId` for a brand-new conversation) + +--- + +### TC-004: Transcript directory created on first startup + +**Type:** `Happy Path` +**Priority:** `P1` + +#### Preconditions +- `<user home>/.copilot/eclipse` directory does not exist + +#### Steps +1. Start Eclipse with the Copilot plugin installed +2. Wait for the Copilot language server to finish initializing (status bar shows Copilot ready) +3. Check the file system for `<user home>/.copilot/eclipse` + +#### Expected Result +- `<user home>/.copilot/eclipse` directory is created automatically on startup +- The path mirrors IntelliJ's `<user home>/.copilot/jb` convention (different subdirectory, same parent) + +#### 📸 Key Screenshots +- [ ] **Directory exists** — File explorer showing `<user home>/.copilot/eclipse` after startup + +--- + +### TC-005: Partially completed conversation restores to the last completed turn + +**Type:** `Edge Case` +**Priority:** `P1` + +#### Preconditions +- Copilot Chat is open in Agent mode + +#### Steps +1. Send a message and wait for Copilot to begin responding +2. Click Cancel while the response is still streaming +3. Open chat history, switch to a different conversation +4. Open chat history again and select the original conversation +5. Send a new follow-up message + +#### Expected Result +- The restored conversation shows the partial/cancelled turn in the UI +- The `restoreToTurnId` sent to CLS corresponds to the last *fully completed* turn, not the cancelled one +- Copilot's next response is coherent with the conversation up to the last completed turn + +#### 📸 Key Screenshots +- [ ] **After cancel** — Conversation with cancelled/partial turn visible +- [ ] **After restore + new message** — New response coherent with completed context + +--- + +### TC-006: No 400 Bad Request after restoring a tool-call conversation + +**Type:** `Regression` +**Priority:** `P0` + +#### Preconditions +- Copilot Chat is open in Agent mode +- A workspace with at least one source file is open + +#### Steps +1. Send a message that triggers at least one tool call and gets a full assistant response (e.g. "list the files in this project and summarise what each one does") +2. Wait for the full multi-turn exchange to complete (tool calls executed, final assistant reply shown) +3. Restart Eclipse IDE +4. Open Copilot Chat and restore the previous conversation from history +5. Send a new follow-up message in the restored conversation + +#### Expected Result +- The follow-up message is sent without any 400 Bad Request error +- The assistant replies successfully on the first attempt (no error banner) + +#### 📸 Key Screenshots +- [ ] **Completed original turn** — Chat showing the tool-call exchange and final assistant reply before restart +- [ ] **After restore + reply** — Restored history with successful follow-up reply and no error + +--- + +### TC-007: Restored history shows no duplicated user messages + +**Type:** `Regression` +**Priority:** `P0` + +#### Preconditions +- Copilot Chat is open in Agent mode +- A workspace with at least one source file is open + +#### Steps +1. Send a message that triggers at least one tool call (e.g. "read the contents of README.md and explain it") +2. Wait for the full response including the assistant's explanation +3. Note the exact assistant response text +4. Restart Eclipse IDE +5. Open Copilot Chat and restore the conversation from history +6. Inspect the restored conversation turns + +#### Expected Result +- The assistant's previous response text does not appear as a user message in the restored conversation +- Each turn shows the correct role (user prompt → tool call → assistant reply), with no duplicated content + +#### 📸 Key Screenshots +- [ ] **Original conversation** — Chat showing the original tool-call turn and assistant reply +- [ ] **Restored conversation** — Restored history with correctly attributed messages and no duplicates + +--- + +## Screenshots Checklist +> Consolidated list of all key screenshot moments. + +- [ ] `TC-001` Full conversation before session switch +- [ ] `TC-001` Same conversation after restore, all turns in correct order +- [ ] `TC-002` `<user home>/.copilot/eclipse` directory with transcript files +- [ ] `TC-002` Conversation restored in history after Eclipse restart +- [ ] `TC-004` `<user home>/.copilot/eclipse` directory created on first startup +- [ ] `TC-005` Conversation with cancelled turn visible +- [ ] `TC-005` New response coherent with completed context after restore +- [ ] `TC-006` Completed tool-call turn before restart +- [ ] `TC-006` Restored history with successful follow-up and no 400 error +- [ ] `TC-007` Original conversation showing assistant reply +- [ ] `TC-007` Restored conversation with correctly attributed messages and no duplicates diff --git a/test-plans/skills-prompt-files/skills-prompt-files.md b/test-plans/skills-prompt-files/skills-prompt-files.md new file mode 100644 index 00000000..b88cb673 --- /dev/null +++ b/test-plans/skills-prompt-files/skills-prompt-files.md @@ -0,0 +1,234 @@ +# Support Skills and Prompt Files + +## Overview +This feature adds support for agent skills (`SKILL.md`) and custom prompt files (`.prompt.md`) in Agent Mode. Skills and prompts are discovered from workspace projects and user-global directories, exposed as slash commands in the chat input, and controlled by an **Enable Skills** toggle in `Window → Preferences → Copilot → Chat`. + +--- + +## Test Cases + +### TC-001: Enable Skills toggle appears in Chat preferences + +**Type:** `Happy Path` +**Priority:** `P0` + +#### Preconditions +- Eclipse is running with the Copilot plugin installed +- A valid GitHub Copilot subscription is active (client-preview feature flag must be on) + +#### Steps +1. Open **Window → Preferences → Copilot → Chat** +2. Observe the preference page content + +#### Expected Result +- An **Enable Skills** checkbox is visible +- A note below it reads "Controls whether agent skills can be used to enrich chat context." +- The checkbox is checked by default (enabled) + +#### 📸 Key Screenshots +- [ ] **Preferences page** — Chat preferences showing the Enable Skills toggle (checked) + +--- + +### TC-002: Workspace-scoped skill appears as slash command in Agent mode + +**Type:** `Happy Path` +**Priority:** `P0` + +#### Preconditions +- Eclipse is running with a project open in the workspace +- Copilot chat is open in **Agent** mode +- **Enable Skills** is checked in preferences + +#### Steps +1. Create the directory `.github/skills/my-skill/` inside any open project +2. Create `SKILL.md` inside that directory with the following content: + ```markdown + --- + name: My Skill + description: A test skill for verification + --- + This skill provides test context. + ``` +3. In the Copilot chat input (Agent mode), type `/` and observe the slash command popup + +#### Expected Result +- `my-skill` appears in the slash command list with description "A test skill for verification" +- The display name shown is `My Skill` (from `shortDescription`/name front matter) + +#### 📸 Key Screenshots +- [ ] **Slash command popup** — `/my-skill` visible in the Agent mode autocomplete list + +--- + +### TC-003: Prompt file appears as slash command in Agent mode + +**Type:** `Happy Path` +**Priority:** `P0` + +#### Preconditions +- Eclipse is running with a project open in the workspace +- Copilot chat is open in **Agent** mode +- **Enable Skills** is checked in preferences + +#### Steps +1. Create a `.github/` directory (or any supported location) inside the project +2. Create a file named `review.prompt.md` in the project with some markdown content +3. In the Copilot chat input (Agent mode), type `/` and observe the slash command popup + +#### Expected Result +- `review` (or the prompt file's ID) appears in the slash command list +- The template is listed alongside built-in slash commands + +#### 📸 Key Screenshots +- [ ] **Slash command popup** — Custom prompt file visible as slash command in Agent mode + +--- + +### TC-004: Skills do NOT appear in Ask mode + +**Type:** `Happy Path` +**Priority:** `P0` + +#### Preconditions +- A workspace-scoped `SKILL.md` exists (as created in TC-002) +- **Enable Skills** is checked in preferences + +#### Steps +1. Switch the Copilot chat to **Ask** mode +2. In the chat input, type `/` and observe the slash command popup + +#### Expected Result +- The skill (`my-skill`) does **not** appear in the Ask mode slash command list +- Only built-in `chat-panel`-scoped commands are shown + +#### 📸 Key Screenshots +- [ ] **Ask mode popup** — Skill absent from the slash command list in Ask mode + +--- + +### TC-005: Disabling Skills hides skills from slash command list + +**Type:** `Happy Path` +**Priority:** `P0` + +#### Preconditions +- A workspace-scoped `SKILL.md` exists and was visible in Agent mode (TC-002 completed) + +#### Steps +1. Open **Window → Preferences → Copilot → Chat** +2. Uncheck **Enable Skills** and click **Apply and Close** +3. In the Copilot chat (Agent mode), type `/` and observe the slash command popup + +#### Expected Result +- The skill is no longer listed in the slash command popup +- Built-in slash commands (e.g. `/fix`, `/explain`) still appear if scoped for Agent mode + +#### 📸 Key Screenshots +- [ ] **Preferences page** — Enable Skills unchecked +- [ ] **Slash command popup** — Skill absent after disabling + +--- + +### TC-006: Auto-refresh when SKILL.md is added or removed + +**Type:** `Happy Path` +**Priority:** `P1` + +#### Preconditions +- Eclipse is running with a project open +- Copilot chat is open in Agent mode +- **Enable Skills** is checked + +#### Steps +1. In the chat input, type `/` — note that the skill does not yet exist +2. In Eclipse's Project Explorer, create `.github/skills/refresh-skill/SKILL.md` with a description +3. Without restarting, return to the chat input and type `/` again +4. Then delete the `SKILL.md` file and type `/` once more + +#### Expected Result +- After adding: `refresh-skill` appears in the slash command list without restarting Eclipse +- After deleting: `refresh-skill` disappears from the slash command list without restarting Eclipse + +#### 📸 Key Screenshots +- [ ] **Before add** — Slash command popup without the skill +- [ ] **After add** — Slash command popup with the new skill +- [ ] **After delete** — Slash command popup with skill removed + +--- + +### TC-007: User-global skill discovery + +**Type:** `Happy Path` +**Priority:** `P1` + +#### Preconditions +- Copilot chat is open in Agent mode +- **Enable Skills** is checked + +#### Steps +1. Create `~/.copilot/skills/global-skill/SKILL.md` with a name and description in YAML front matter +2. Restart Eclipse (or wait for auto-refresh) +3. In the Copilot chat input (Agent mode), type `/` + +#### Expected Result +- `global-skill` appears in the slash command list, available regardless of which workspace is open + +#### 📸 Key Screenshots +- [ ] **Slash command popup** — User-global skill visible in Agent mode + +--- + +### TC-008: Skill prefix filtering in slash command autocomplete + +**Type:** `Happy Path` +**Priority:** `P1` + +#### Preconditions +- At least two skills are available in Agent mode (e.g., `my-skill` and `other-skill`) + +#### Steps +1. In the Copilot chat input (Agent mode), type `/my` +2. Observe the filtered slash command popup + +#### Expected Result +- Only commands matching "my" are shown (e.g., `my-skill` visible, `other-skill` absent) + +#### 📸 Key Screenshots +- [ ] **Filtered popup** — Autocomplete showing only matching skills for typed prefix + +--- + +### TC-009: Skills settings persist across Eclipse restarts + +**Type:** `Regression` +**Priority:** `P1` + +#### Preconditions +- **Enable Skills** preference is visible + +#### Steps +1. Open **Window → Preferences → Copilot → Chat** and uncheck **Enable Skills** +2. Click **Apply and Close** +3. Restart Eclipse +4. Open **Window → Preferences → Copilot → Chat** + +#### Expected Result +- **Enable Skills** remains unchecked after restart, confirming the preference is persisted + +--- + +## Screenshots Checklist +> Consolidated list of all key screenshot moments. + +- [ ] `TC-001` Chat preferences showing the Enable Skills toggle (checked) +- [ ] `TC-002` Slash command popup with `/my-skill` visible in Agent mode +- [ ] `TC-003` Custom prompt file visible as slash command in Agent mode +- [ ] `TC-004` Ask mode popup without skill in slash command list +- [ ] `TC-005` Enable Skills unchecked in preferences +- [ ] `TC-005` Slash command popup without skill after disabling +- [ ] `TC-006` Slash command popup before skill is added +- [ ] `TC-006` Slash command popup after skill is added +- [ ] `TC-006` Slash command popup after skill is deleted +- [ ] `TC-007` User-global skill visible in Agent mode slash command popup +- [ ] `TC-008` Filtered autocomplete showing only matching skills for typed prefix diff --git a/test-plans/subagent-restoration/subagent-restoration.md b/test-plans/subagent-restoration/subagent-restoration.md new file mode 100644 index 00000000..cbf0f70e --- /dev/null +++ b/test-plans/subagent-restoration/subagent-restoration.md @@ -0,0 +1,113 @@ +# Subagent Turns Appear as Separate Assistant Messages After Restoration + +## Overview +When a subagent executes during an agent-mode conversation, its turns were previously persisted as independent `CopilotTurnData` entries. On restoration from history, those turns rendered as standalone assistant messages instead of being nested inside the parent agent turn. + +--- + +## Test Cases + +### TC-001: Subagent turn restores nested inside parent agent turn + +**Type:** `Happy Path` +**Priority:** `P0` + +#### Preconditions +- Copilot chat is open in Agent mode +- A prompt that reliably triggers subagent execution is available (e.g. a task that invokes `run_subagent`) + +#### Steps +1. Send a message in Agent mode that causes a subagent to execute (e.g. "analyze all Java files in the workspace using a subagent") +2. Wait for the subagent to complete and the full response to finish streaming +3. Note the structure of the response: the subagent output should appear nested/indented inside the parent agent turn, **not** as a separate assistant message below it +4. Open chat history (the history panel), select a different conversation or click "New Chat" +5. Open chat history again and select the original conversation to restore it + +#### Expected Result +- The restored conversation shows the subagent output nested inside the parent agent turn, matching the structure visible before the session switch +- No additional standalone assistant message appears below the parent turn for the subagent execution + +#### 📸 Key Screenshots +- [ ] **Before switch** — Full conversation view showing subagent output nested inside parent turn +- [ ] **After restore** — Same conversation restored with subagent output still nested, no orphaned message + +--- + +### TC-002: Multiple subagent turns all restore nested correctly + +**Type:** `Happy Path` +**Priority:** `P0` + +#### Preconditions +- Copilot chat is open in Agent mode +- A prompt that triggers multiple sequential subagent invocations is available + +#### Steps +1. Send a message that causes two or more subagent invocations in a single parent turn +2. Wait for all subagents to complete +3. Verify each subagent's output is shown nested inside the parent turn (no orphaned messages) +4. Switch to a different conversation via chat history, then switch back to restore the original + +#### Expected Result +- All subagent turns restore nested inside the parent agent turn +- The number of visible assistant messages in the turn matches what was shown before the session switch +- No orphaned standalone assistant messages appear for any subagent + +#### 📸 Key Screenshots +- [ ] **Before switch** — Multiple subagent outputs nested inside one parent turn +- [ ] **After restore** — All subagent outputs still nested, none appear as separate messages + +--- + +### TC-003: Conversation with no subagent is unaffected + +**Type:** `Regression` +**Priority:** `P0` + +#### Preconditions +- Copilot chat is open in Agent mode + +#### Steps +1. Send a message that produces a normal (non-subagent) agent response +2. Wait for the response to complete +3. Switch to a different conversation via chat history, then switch back to restore the original + +#### Expected Result +- The restored conversation looks identical to before the session switch +- No duplicate or extra assistant messages appear + +--- + +### TC-004: Partially-streamed subagent turn (cancelled) restores correctly + +**Type:** `Edge Case` +**Priority:** `P1` + +#### Preconditions +- Copilot chat is open in Agent mode + +#### Steps +1. Send a message that triggers a subagent execution +2. While the subagent is still streaming (before it completes), click the Cancel button +3. Observe the state of the conversation — the parent turn and any partial subagent output +4. Switch to a different conversation via chat history, then switch back + +#### Expected Result +- The restored conversation reflects the cancelled state +- The partial subagent output (if any) is shown nested inside the parent turn, not as a separate assistant message + +#### 📸 Key Screenshots +- [ ] **After cancel** — Conversation showing cancelled subagent output nested in parent turn +- [ ] **After restore** — Same nested structure restored after session switch + +--- + +## Screenshots Checklist +> Consolidated list of all key screenshot moments. + +- [ ] `TC-001` Subagent nested in parent turn before switch +- [ ] `TC-001` Subagent nested in parent turn after restore +- [ ] `TC-002` Multiple subagents nested before switch +- [ ] `TC-002` Multiple subagents nested after restore +- [ ] `TC-004` Cancelled subagent nested after cancel +- [ ] `TC-004` Cancelled subagent nested after restore