MegaBites-AI/Windows-powershell

1---2applyTo: "**/*.Tests.ps1"3---4 5# Pester Test Status Meanings and Working Tests6 7## Purpose8 9This guide clarifies Pester test outcomes and what it means for a test to be "working" - which requires both **passing** AND **actually validating functionality**.10 11## Test Statuses in Pester12 13### Passed ✓14**Status Code**: `Passed` 15**Exit Result**: Test ran successfully, all assertions passed16 17**What it means**:18- Test executed without errors19- All `Should` statements evaluated to true20- Test setup and teardown completed without issues21- Test is **validating** the intended functionality22 23**What it does NOT mean**:24- The feature is working (assertions could be wrong)25- The test is meaningful (could be testing wrong thing)26- The test exercises all code paths27 28### Failed ✗29**Status Code**: `Failed`30**Exit Result**: Test ran but assertions failed31 32**What it means**:33- Test executed but an assertion returned false34- Expected value did not match actual value35- Test detected a problem with the functionality36 37**Examples**:38```39Expected $true but got $false40Expected 5 items but got 341Expected no error but got: Cannot find parameter42```43 44### Error ⚠45**Status Code**: `Error`46**Exit Result**: Test crashed with an exception47 48**What it means**:49- Test failed to complete50- An exception was thrown during test execution51- Could be in test setup, test body, or test cleanup52- Often indicates environmental issue, not code functional issue53 54**Examples**:55```56Cannot bind argument to parameter 'Path' because it is null57File not found: C:\expected\config.json58Access denied writing to registry59```60 61### Pending ⏳62**Status Code**: `Pending`63**Exit Result**: Test ran but never completed assertions64 65**What it means**:66- Test was explicitly marked as not ready to run67- `Set-ItResult -Pending` was called68- Used to indicate: known bugs, missing features, environmental issues69 70**When to use Pending**:71- Test for feature in development72- Test disabled due to known bug (issue #1234)73- Test disabled due to intermittent failures being fixed74- Platform-specific issues being resolved75 76**⚠️ WARNING**: Pending tests are NOT validating functionality. They hide problems.77 78### Skipped ⊘79**Status Code**: `Skipped`  80**Exit Result**: Test did not run (detected at start)81 82**What it means**:83- Test was intentionally not executed84- `-Skip` parameter or `It -Skip:$condition` was used85- Environment doesn't support this test86 87**When to use Skip**:88- Test not applicable to current platform (Windows-only test on Linux)89- Test requires feature that's not available (admin privileges)90- Test requires specific configuration not present91 92**Difference from Pending**:93- **Skip**: "This test shouldn't run here" (known upfront)94- **Pending**: "This test should eventually run but can't now"95 96### Ignored ✛97**Status Code**: `Ignored`98**Exit Result**: Test marked as not applicable99 100**What it means**:101- Test has `[Ignore("reason")]` attribute102- Test is permanently disabled in this location103- Not the same as Skipped (which is conditional)104 105**When to use Ignore**:106- Test for deprecated feature107- Test for bug that won't be fixed108- Test moved to different test file109 110---111 112## What Does "Working" Actually Mean?113 114A test is **working** when it meets BOTH criteria:115 116### 1. **Test Status is PASSED** ✓117```powershell118It "Test name" {119    # Test executes120    # All assertions pass121    # Returns Passed status122}123```124 125### 2. **Test Actually Validates Functionality**126```powershell127# ✓ GOOD: Tests actual functionality128It "Get-Item returns files from directory" -Tags @('Unit') {129    $testDir = New-Item -ItemType Directory -Force130    New-Item -Path $testDir -Name "file.txt" -ItemType File | Out-Null131    132    $result = Get-Item -Path "$testDir\file.txt"133    134    $result.Name | Should -Be "file.txt"135    $result | Should -Exist136    137    Remove-Item $testDir -Recurse -Force138}139 140# ✗ BAD: Returns Passed but doesn't validate functionality141It "Get-Item returns files from directory" -Tags @('Unit') {142    $result = Get-Item -Path somepath  # May not exist, may not actually test143    $result | Should -Not -BeNullOrEmpty  # Too vague144}145 146# ✗ BAD: Test marked Pending - validation is hidden147It "Get-Item returns files from directory" -Tags @('Unit') {148    Set-ItResult -Pending -Because "File system not working"149    return150    # No validation happens at all151}152```153 154---155 156## The Problem with Pending Tests157 158### Why Pending Tests Hide Problems159 160```powershell161# BAD: Test marked Pending - looks like "working" status but validation is skipped162It "Download help from web" {163    Set-ItResult -Pending -Because "Web connectivity issues"164    return165    166    # This code never runs:167    Update-Help -Module PackageManagement -Force -ErrorAction Stop168    Get-Help Get-Package | Should -Not -BeNullOrEmpty169}170```171 172**Result**:173- ✗ Feature is broken (Update-Help fails)174- ✓ Test shows "Pending" (looks acceptable)175- ✗ Problem is hidden and never fixed176 177### The Right Approach178 179**Option A: Fix the root cause**180```powershell181It "Download help from web" {182    # Use local assets that are guaranteed to work183    Update-Help -Module PackageManagement -SourcePath ./assets -Force -ErrorAction Stop184    185    Get-Help Get-Package | Should -Not -BeNullOrEmpty186}187```188 189**Option B: Gracefully skip when unavailable**190```powershell191It "Download help from web" -Skip:$(-not $hasInternet) {192    Update-Help -Module PackageManagement -Force -ErrorAction Stop193    Get-Help Get-Package | Should -Not -BeNullOrEmpty194}195```196 197**Option C: Add retry logic for intermittent issues**198```powershell199It "Download help from web" {200    $maxRetries = 3201    $attempt = 0202    203    while ($attempt -lt $maxRetries) {204        try {205            Update-Help -Module PackageManagement -Force -ErrorAction Stop206            break207        }208        catch {209            $attempt++210            if ($attempt -ge $maxRetries) { throw }211            Start-Sleep -Seconds 2212        }213    }214    215    Get-Help Get-Package | Should -Not -BeNullOrEmpty216}217```218 219---220 221## Test Status Summary Table222 223| Status | Passed? | Validates? | Counts as "Working"? | Use When |224|--------|---------|------------|----------------------|----------|225| **Passed** | ✓ | ✓ | **YES** | Feature is working and test proves it |226| **Failed** | ✗ | ✓ | NO | Feature is broken or test has wrong expectation |227| **Error** | ✗ | ✗ | NO | Test infrastructure broken, can't validate |228| **Pending** | - | ✗ | **NO** ⚠️ | Temporary - test should eventually pass |229| **Skipped** | - | ✗ | NO | Test not applicable to this environment |230| **Ignored** | - | ✗ | NO | Test permanently disabled |231 232---233 234## Recommended Patterns235 236### Pattern 1: Resilient Test with Fallback237```powershell238It "Feature works with web or local source" {239    $useLocal = $false240    241    try {242        Update-Help -Module Package -Force -ErrorAction Stop243    }244    catch {245        $useLocal = $true246        Update-Help -Module Package -SourcePath ./assets -Force -ErrorAction Stop247    }248    249    # Validate functionality regardless of source250    Get-Help Get-Package | Should -Not -BeNullOrEmpty251}252```253 254### Pattern 2: Conditional Skip with Clear Reason255```powershell256Describe "Update-Help from Web" -Skip $(-not (Test-InternetConnectivity)) {257    It "Downloads help successfully" {258        Update-Help -Module PackageManagement -Force -ErrorAction Stop259        Get-Help Get-Package | Should -Not -BeNullOrEmpty260    }261}262```263 264### Pattern 3: Separate Suites by Dependency265```powershell266Describe "Help Content Tests - Web" {267    # Tests that require internet - can be skipped if unavailable268    It "Downloads from web" { ... }269}270 271Describe "Help Content Tests - Local" {272    # Tests with local assets - should always pass273    It "Loads from local assets" {274        Update-Help -Module Package -SourcePath ./assets -Force275        Get-Help Get-Package | Should -Not -BeNullOrEmpty276    }277}278```279 280---281 282## Checklist: Is Your Test "Working"?283 284- [ ] Test status is **Passed** (not Pending, not Skipped, not Failed)285- [ ] Test actually **executes** the feature being tested286- [ ] Test has **specific assertions** (not just `Should -Not -BeNullOrEmpty`)287- [ ] Test includes **cleanup** (removes temp files, restores state)288- [ ] Test can run **multiple times** without side effects289- [ ] Test failure **indicates a real problem** (not flaky assertions)290- [ ] Test success **proves the feature works** (not just "didn't crash")291 292If any of these is false, your test may be passing but not "working" properly.293 294---295 296## See Also297 298- [Pester Documentation](https://pester.dev/)299- [Set-ItResult Documentation](https://pester.dev/docs/commands/Set-ItResult)300