Skip to content

Commit e4f2d39

Browse files
authored
fix(core): skip writing deps cache if already up-to-date (#34582)
## Current Behavior Frequent write cache calls during tasks hashing results in delays ## Expected Behavior Cache is validated but only written when changed ## AI Summary This pull request introduces an optimization to the project graph cache writing logic, reducing unnecessary disk writes when serving repeated requests with unchanged graphs. The main change is the addition of a mechanism to track the cache file's modification time and only write to disk if the file has been externally modified or not written yet by the current process. Optimizations to cache writing: * Added `writeCacheIfStale` function in `nx-deps-cache.ts` to prevent redundant cache writes by checking the cache file's modification time before writing. This function is now used in the daemon's graph recomputation logic, replacing the previous unconditional write. [[1]](diffhunk://#diff-82bd1a5a7b7320ffc3233470f191782c054bf69a696dc16001d2c4b1d0b04963R285-R312) [[2]](diffhunk://#diff-d5bf3c66e62cac1884a071bf07fd1991320a3e62b07bfc03af3b9557b714c892L17-R17) [[3]](diffhunk://#diff-d5bf3c66e62cac1884a071bf07fd1991320a3e62b07bfc03af3b9557b714c892L136-R149) * Introduced `lastWrittenCacheMtimeMs` variable to track the last successful write's modification time, updated after each cache write. [[1]](diffhunk://#diff-82bd1a5a7b7320ffc3233470f191782c054bf69a696dc16001d2c4b1d0b04963R202-R208) [[2]](diffhunk://#diff-82bd1a5a7b7320ffc3233470f191782c054bf69a696dc16001d2c4b1d0b04963R256-R261) Codebase updates: * Updated imports in `nx-deps-cache.ts` to include `statSync` for file modification time checks. ## Related Issue(s) <!-- Please link the issue being fixed so it gets closed when this is merged. --> Fixes #
1 parent 42f73a5 commit e4f2d39

File tree

2 files changed

+57
-6
lines changed

2 files changed

+57
-6
lines changed

packages/nx/src/daemon/server/project-graph-incremental-recomputation.ts

Lines changed: 15 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ import {
1515
nxProjectGraph,
1616
readFileMapCache,
1717
writeCache,
18+
writeCacheIfStale,
1819
} from '../../project-graph/nx-deps-cache';
1920
import {
2021
retrieveProjectConfigurations,
@@ -98,6 +99,7 @@ export async function getCachedSerializedProjectGraphPromise(): Promise<Serializ
9899
waitPeriod = 100;
99100
await resetInternalStateIfNxDepsMissing();
100101
const plugins = await getPlugins();
102+
const previousPromise = cachedSerializedProjectGraphPromise;
101103
if (collectedUpdatedFiles.size == 0 && collectedDeletedFiles.size == 0) {
102104
if (!cachedSerializedProjectGraphPromise) {
103105
cachedSerializedProjectGraphPromise =
@@ -117,6 +119,8 @@ export async function getCachedSerializedProjectGraphPromise(): Promise<Serializ
117119
cachedSerializedProjectGraphPromise =
118120
processFilesAndCreateAndSerializeProjectGraph(plugins);
119121
}
122+
const graphWasRecomputed =
123+
cachedSerializedProjectGraphPromise !== previousPromise;
120124
const result = await cachedSerializedProjectGraphPromise;
121125

122126
if (wasScheduled) {
@@ -133,16 +137,22 @@ export async function getCachedSerializedProjectGraphPromise(): Promise<Serializ
133137
: [result.error]
134138
: [];
135139

136-
// Always write the daemon's current graph to disk to ensure disk cache
137-
// stays in sync with the daemon's in-memory cache. This prevents issues
138-
// where a non-daemon process writes a stale/errored cache that never
139-
// gets overwritten by the daemon's valid graph.
140+
// Write the daemon's current graph to disk to ensure disk cache stays
141+
// in sync with the daemon's in-memory cache. This prevents issues where
142+
// a non-daemon process writes a stale/errored cache that never gets
143+
// overwritten by the daemon's valid graph.
144+
//
145+
// When the graph was just recomputed, always write so the new graph is
146+
// persisted. When serving the same graph from memory, use
147+
// writeCacheIfStale to skip the write unless an external process has
148+
// modified the file since this process last wrote it.
140149
if (
141150
result.projectGraph &&
142151
result.projectFileMapCache &&
143152
result.sourceMaps
144153
) {
145-
writeCache(
154+
const writeFn = graphWasRecomputed ? writeCache : writeCacheIfStale;
155+
writeFn(
146156
result.projectFileMapCache,
147157
result.projectGraph,
148158
result.sourceMaps,

packages/nx/src/project-graph/nx-deps-cache.ts

Lines changed: 42 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
import { existsSync, mkdirSync, renameSync } from 'node:fs';
1+
import { existsSync, mkdirSync, renameSync, statSync } from 'node:fs';
22
import { join } from 'path';
33
import { performance } from 'perf_hooks';
44
import { NxJsonConfiguration } from '../config/nx-json';
@@ -199,6 +199,13 @@ export function createProjectFileMapCache(
199199
return newValue;
200200
}
201201

202+
/**
203+
* Tracks the mtime of the project graph cache file after the last successful
204+
* writeCache() call. Used by writeCacheIfStale() to skip redundant writes
205+
* when no external process has modified the cache file since the last write.
206+
*/
207+
let lastWrittenCacheMtimeMs: number | undefined;
208+
202209
export function writeCache(
203210
cache: FileMapCache,
204211
projectGraph: ProjectGraph,
@@ -246,6 +253,12 @@ export function writeCache(
246253
);
247254
}
248255

256+
try {
257+
lastWrittenCacheMtimeMs = statSync(nxProjectGraph).mtimeMs;
258+
} catch {
259+
lastWrittenCacheMtimeMs = undefined;
260+
}
261+
249262
done = true;
250263
} catch (err: any) {
251264
if (err instanceof Error) {
@@ -269,6 +282,34 @@ export function writeCache(
269282
performance.measure('write cache', 'write cache:start', 'write cache:end');
270283
}
271284

285+
/**
286+
* Writes the cache only if the on-disk cache file has been modified since
287+
* this process last wrote it (i.e. an external process overwrote it), or
288+
* if this process has never written the cache.
289+
*
290+
* Use this instead of writeCache() on hot paths where the same graph may
291+
* be served multiple times without changing (e.g. the daemon responding
292+
* to repeated client requests).
293+
*/
294+
export function writeCacheIfStale(
295+
cache: FileMapCache,
296+
projectGraph: ProjectGraph,
297+
sourceMaps: ConfigurationSourceMaps,
298+
errors: ProjectGraphErrorTypes[]
299+
): void {
300+
if (lastWrittenCacheMtimeMs !== undefined) {
301+
try {
302+
const currentMtimeMs = statSync(nxProjectGraph).mtimeMs;
303+
if (currentMtimeMs === lastWrittenCacheMtimeMs) {
304+
return;
305+
}
306+
} catch {
307+
// File doesn't exist or can't be stat'd — proceed with write
308+
}
309+
}
310+
writeCache(cache, projectGraph, sourceMaps, errors);
311+
}
312+
272313
export function shouldRecomputeWholeGraph(
273314
cache: FileMapCache,
274315
packageJsonDeps: Record<string, string>,

0 commit comments

Comments
 (0)