win_psDscAdapter targeted module cache refreshes lead to broken cache state

[haodeon]

Prerequisites

• Write a descriptive title.
• Make sure you are able to repro it on the latest version
• Search the existing issues.

Summary

Invoke-DscCacheRefresh targeted module cache refreshes using $Module, has a number of bugs which can lead to incomplete or broken cache state. Once the cache enters this state there is no recovery, requiring either manual reset or triggering cache refresh and $Module not being passed for a full refresh.

These issues were discovered during efforts to understand how cache refresh worked while troubleshooting #745. It's possible the cache refresh was designed for a time when DSC v3 worked differently. With current behaviour, targeted refreshes add significant complexity, it could do with a review of whether it's still needed.

1. Cache refreshes only return $Module, fixed in Fix resourceCache to return proper modules when PS modules are installed during run #748. Refreshes would overwrite the cache with an incomplete cache.
2. Changes triggering PSModulePath refreshes are not included in the cache refresh, proposed fix fix: win_psDscAdapter targeted module cache refreshes #787
3. Stale resource entries will remain in the cache if they are not included in $Module.
4. Refreshes targeting only PSDesiredStateConfiguration overwrites cache with incomplete cache.
5. Refreshes containing PSDesiredStateConfiguration incorrectly handles PSDesiredStateConfiguration/SignatureValidation and PSDesiredStateConfiguration/File.
6. Refreshes containing duplicate module names result in duplicate resource cache entries.
7. Incompatible schema refresh retains ResourceCache entries with potentially incompatible schema.
8. If the cache is non-existent or empty, a partial/incomplete cache is built.

If the cache is cold (missing, empty, or schema-invalid), a full refresh would guarantee correctness. If PSModulePath changes also force full refreshes, only stale resource entries would result in a targeted refresh. It wouldn't be too far from removing it altogether.

Another approach would be to exploit how get, set and test are always called with $Module and provide incremental cache recovery. It's not perfect but aligns the most with the current code. An implementation is linked in the next section.

A redesign of the cache could ensure cold starts began with a full refresh and stale resource entries or stale paths are handled differently allowing an optimised targeted module refresh.

Steps to reproduce

Instead of providing repros for every case, this commit 27a3b6b, which tries to make targeted module refreshes safer, can be used to review this issue.

Expected behavior

For the cache state to be consistent with the module environment

Actual behavior

The cache is considered valid but the state can be:

- Incomplete/partially complete
- Contain duplicate resource entries
- Retain stale entries

Error details

2025-05-10T10:21:07.367034Z DEBUG dsc_lib::dscresources::command_resource: 902: PID 3456: Reading from Get-DscResource cache file C:\Users\Administrator\AppData\Local\dsc\WindowsPSAdapterCache.json
2025-05-10T10:21:07.411037Z DEBUG dsc_lib::dscresources::command_resource: 902: PID 3456: Checking cache for stale entries
2025-05-10T10:21:07.900558Z DEBUG dsc_lib::dscresources::command_resource: 902: PID 3456: Checking cache for stale PSModulePath
2025-05-10T10:21:07.990651Z DEBUG dsc_lib::dscresources::command_resource: 902: PID 3456: PSModulePath diff ''
2025-05-10T10:21:08.001003Z ERROR dsc_lib::dscresources::command_resource: 896: PID 3456: DSC resource 'xPSDesiredStateConfiguration/xWindowsFeature' module not found.
2025-05-10T10:21:08.048023Z DEBUG dsc_lib::dscresources::command_resource: 660: Process 'powershell' id 3456 exited with code 1
2025-05-10T10:21:08.049577Z ERROR dsc::subcommand: 106: Error: Command: Resource 'powershell' [exit code 1] manifest description: Error

Environment data

Name                           Value
----                           -----
PSVersion                      5.1.26100.3624
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.26100.3624
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1

Version

3.1.0-preview.5

Visuals

No response