feat(maven): add batch executor for multi-task Maven execution by FrozenPandaz · Pull Request #33228 · nrwl/nx

FrozenPandaz · 2025-10-23T22:42:40Z

Summary

Adds a batch executor for Nx Maven that enables parallel multi-task execution with significant performance improvements. The batch runner keeps Maven resident in memory, avoiding cold start overhead for each task.

Changes

1. Batch Runner JAR (`packages/maven/batch-runner`)

ResidentMavenExecutor: Uses Maven 4.x's ResidentMavenInvoker to keep Maven in memory
NxMaven: Custom Maven wrapper that caches project graphs and sessions across invocations
CachingResidentMavenInvoker: Preserves session state so artifacts from jar:jar are visible to install:install
BuildStateManager: Applies/records build states for cross-invocation caching
Maven 4.x dependencies are shaded into the JAR for standalone execution

2. TypeScript Executors (`packages/maven/src/executors/maven`)

maven.impl.ts: Single-task executor using mvnw/mvn
maven-batch.impl.ts: Batch executor that invokes the batch runner JAR
Automatic Maven version detection and executable resolution

3. Shared Utilities (`packages/maven/shared`)

BuildState, BuildStateApplier, BuildStateRecorder for cross-invocation state
MavenCommandResolver for detecting Maven executable
Reusable across batch-runner and maven-plugin modules

4. Maven Plugin Updates (`packages/maven/maven-plugin`)

Updated to use @nx/maven:maven executor (batch-aware)
Improved GitIgnoreClassifier for nested .gitignore handling
Cache config tweaks for compiler inputs

Performance

Scenario	Before	After
Cold start per task	100-500ms	N/A (one-time init)
Per-task execution	100-500ms	~1.3ms (cached)
Improvement	-	75-385x faster

Version Support

Maven 4.x: Full support with ResidentMavenExecutor (optimized)
Maven 3.x: Falls back to ProcessBasedMavenExecutor (subprocess)

Testing

E2E tests for Maven 4.0.0-rc-4, 4.0.0-rc-5
Unit tests for TypeScript executors
Tests for GitIgnoreClassifier

Current Behavior

Uses subprocess execution via mvnw/mvn for each task.

Expected Behavior

Batch execution keeps Maven resident, dramatically reducing per-task overhead.

Related Issue(s)

Part of Maven integration improvements.

netlify · 2025-10-23T22:42:46Z

✅ Deploy Preview for nx-docs ready!

Name	Link
🔨 Latest commit	`1adecf5`
🔍 Latest deploy log	https://app.netlify.com/projects/nx-docs/deploys/6943100bace69900089b43c1
😎 Deploy Preview	https://deploy-preview-33228--nx-docs.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

vercel · 2025-10-23T22:42:48Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Review	Updated (UTC)
nx-dev	Ready	Preview	Dec 17, 2025 8:26pm

nx-cloud · 2025-10-23T22:43:28Z

View your CI Pipeline Execution ↗ for commit 1adecf5

Command	Status	Duration	Result
`nx affected --targets=lint,test,test-kt,build,e...`	✅ Succeeded	13m 36s	View ↗
`nx run-many -t check-imports check-lock-files c...`	✅ Succeeded	2m 34s	View ↗
`nx-cloud record -- nx-cloud conformance:check`	✅ Succeeded	12s	View ↗
`nx-cloud record -- nx format:check`	✅ Succeeded	2s	View ↗
`nx-cloud record -- nx sync:check`	✅ Succeeded	<1s	View ↗

☁️ Nx Cloud last updated this comment at 2025-12-18 02:39:49 UTC

github-actions · 2025-11-27T17:53:01Z

Failed to publish a PR release of this pull request, triggered by @FrozenPandaz.
See the failed workflow run at: https://github.com/nrwl/nx/actions/runs/19744405386

github-actions · 2025-11-27T20:56:14Z

🐳 We have a release for that!

This PR has a release associated with it. You can try it out using this command:

npx create-nx-workspace@0.0.0-pr-33228-f4ef600 my-workspace

Or just copy this version and use it in your own command:

0.0.0-pr-33228-f4ef600

Release details	📑
Published version	0.0.0-pr-33228-f4ef600
Triggered by	@FrozenPandaz
Branch	mvn-batch-spike
Commit	f4ef600
Workflow run	19746844547

To request a new release for this pull request, mention someone from the Nx team or the @nrwl/nx-pipelines-reviewers.

github-actions · 2025-12-02T13:34:02Z

🐳 We have a release for that!

This PR has a release associated with it. You can try it out using this command:

npx create-nx-workspace@0.0.0-pr-33228-3422425 my-workspace

Or just copy this version and use it in your own command:

0.0.0-pr-33228-3422425

Release details	📑
Published version	0.0.0-pr-33228-3422425
Triggered by	@FrozenPandaz
Branch	mvn-batch-spike
Commit	3422425
Workflow run	19847576661

To request a new release for this pull request, mention someone from the Nx team or the @nrwl/nx-pipelines-reviewers.

github-actions · 2025-12-02T22:18:37Z

Failed to publish a PR release of this pull request, triggered by @FrozenPandaz.
See the failed workflow run at: https://github.com/nrwl/nx/actions/runs/19874833727

github-actions · 2025-12-03T16:37:07Z

🐳 We have a release for that!

This PR has a release associated with it. You can try it out using this command:

npx create-nx-workspace@0.0.0-pr-33228-ece8d5d my-workspace

Or just copy this version and use it in your own command:

0.0.0-pr-33228-ece8d5d

Release details	📑
Published version	0.0.0-pr-33228-ece8d5d
Triggered by	@FrozenPandaz
Branch	mvn-batch-spike
Commit	ece8d5d
Workflow run	19900243030

To request a new release for this pull request, mention someone from the Nx team or the @nrwl/nx-pipelines-reviewers.

lourw · 2025-12-10T07:58:29Z

packages/maven/batch-runner/src/main/kotlin/dev/nx/maven/runner/MavenInvokerRunner.kt

+    // Thread-safe queue of ready tasks and graph state
+    val taskQueue = LinkedBlockingQueue<String>(initialGraph.roots)
+    val graphRef = AtomicReference(initialGraph)
+    val successfulTasks = ConcurrentHashMap<String, Boolean>()


Why use three maps here instead of one, we could keep a reference to some state instead of three booleans across three maps?

lourw · 2025-12-10T07:59:28Z

packages/maven/src/executors/maven/maven-batch.impl.ts

+    javaArgs.push('--verbose');
+  }
+
+  if (process.env.NX_VERBOSE_LOGGING === 'true') {


nit: do we need two if statements here

lourw · 2025-12-10T08:13:45Z

packages/maven/batch-runner/src/main/kotlin/dev/nx/maven/runner/MavenInvokerRunner.kt

+      repeat(numWorkers) {
+        executor.submit {
+          while (true) {
+            val taskId = taskQueue.poll() ?: break


Could you have a case where the queue is empty, but more work just hasn't been added to it yet?

lourw · 2025-12-10T08:21:56Z

packages/maven/batch-runner/src/main/kotlin/dev/nx/maven/runner/MavenInvokerRunner.kt

+      arguments.add("-e")
+    }
+
+    arguments.add("-nsu")


Do we ever want to not have this?

lourw · 2025-12-10T08:35:45Z

packages/maven/shared/src/main/kotlin/dev/nx/maven/shared/BuildStateApplier.kt

+                        attachedCount++
+                    }
+                } else {
+                    log.warn("    Cannot attach artifact (no MavenProjectHelper): ${file.name}")


This kind of error is fine just leaving as warnings?

cursor

This PR is being reviewed by Cursor Bugbot

Details

You are on the Bugbot Free tier. On this plan, Bugbot will review limited PRs each billing cycle.

To receive Bugbot reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.

cursor · 2025-12-15T16:17:41Z

packages/maven/batch-runner/src/main/kotlin/dev/nx/maven/runner/MavenInvokerRunner.kt

+            if (taskId == null) {
+              log.debug("Worker $workerId exiting loop (poll returned null)")
+              break
+            }


Bug: Workers exit prematurely when task queue temporarily empties

The worker loop uses non-blocking poll() and exits immediately when the queue is empty. This causes a race condition: if there are fewer initial roots than workers, or if the queue empties while tasks are being processed, workers exit permanently and never process newly available tasks. For example, with 8 workers and 1 initial root, 7 workers exit immediately and all subsequent work is done sequentially by a single worker. This was flagged in the PR discussion ("Could you have a case where the queue is empty, but more work just hasn't been added to it yet?"). The workers need to wait or check the completion state before exiting rather than breaking out of the loop on first empty poll.

cursor · 2025-12-15T16:17:41Z

packages/maven/batch-runner/src/main/kotlin/dev/nx/maven/runner/BatchExecutionListener.kt

+    // Track current project across invocations to avoid duplicate "Building" headers
+    @Volatile
+    private var currentProjectId: String? = null
+  }


Bug: Static state causes race condition across concurrent executions

The currentProjectId is stored in a companion object (static) and shared across all BatchExecutionListener instances. Since NxMaven.execute() creates new listener instances per invocation, and multiple worker threads call execute() concurrently from MavenInvokerRunner, the static variable creates a race condition. The check-then-update in projectStarted (if (projectId != currentProjectId) { currentProjectId = projectId; ... }) is not atomic, leading to incorrect suppression or duplication of "Building" headers when projects execute in parallel.

Additional Locations (1)

packages/maven/batch-runner/src/main/kotlin/dev/nx/maven/runner/BatchExecutionListener.kt#L22-L29

cursor · 2025-12-15T16:17:41Z

packages/maven/batch-runner/src/main/kotlin/dev/nx/maven/runner/NxMaven.kt

+      readers.add(reactorReader)
+      request.workspaceReader?.let { readers.add(it) }
+      ideWorkspaceReader?.let { readers.add(it) }
+      cachedWorkspaceReader!!.setReaders(readers)


Bug: Concurrent modification of shared workspace reader without synchronization

The executeWithCachedGraph method is called concurrently from multiple worker threads in MavenInvokerRunner, but it modifies the shared cachedWorkspaceReader by calling setReaders() without synchronization. Each thread creates its own reactorReader via lookup and then overwrites the shared workspace reader's readers set. When threads race, one thread's readers configuration gets overwritten by another's, potentially causing incorrect artifact resolution during parallel Maven executions.

cursor · 2025-12-15T16:17:41Z

packages/maven/batch-runner/src/main/kotlin/dev/nx/maven/runner/NxMaven.kt

+        )
+    } catch (e: Exception) {
+      log.warn("   ⚠️  Failed to initialize ModelBuilderSession: ${e.message}", e)
+    }


Bug: Concurrent modification of shared InternalSession data causes race

The initializeModelBuilderSession method modifies the session data of the shared cachedInternalSession by calling getData().set(). Since cachedInternalSession is reused across all concurrent executions (set via session.session = cachedInternalSession), and executeWithCachedGraph is called without synchronization from multiple worker threads, concurrent invocations overwrite each other's ModelBuilderSession. This race condition could cause Maven plugins that rely on the ModelBuilderSession (like install:install) to fail intermittently with NPEs or use incorrect session data.

Additional Locations (1)

packages/maven/batch-runner/src/main/kotlin/dev/nx/maven/runner/NxMaven.kt#L243-L248

nx-cloud

Important

A new CI pipeline execution was requested that may update the conclusion below...

Nx Cloud has identified a possible root cause for your failed CI:

Our investigation confirms this test failure exists in master with identical patterns, indicating a pre-existing issue rather than a regression from the Maven batch executor changes. The PR exclusively modifies Maven-related code (batch-runner, maven-plugin, executors) with no changes to web component generation, SWC compiler configuration, or TypeScript decorator handling that could affect this test.

No code changes were suggested for this issue.

If the issue was transient, you can trigger a rerun:

View detailed reasoning on Nx Cloud ↗

_{🎓 Learn more about Self-Healing CI on nx.dev}

Initial implementation of Maven batch execution framework for efficient multi-task execution: - Add batch executor for parallel execution of multiple Maven tasks - Implement project selector to filter and target specific projects - Parse TaskGraph JSON into typed Kotlin data classes for type safety - Configure invoker with workspace pom file and repository settings - Upgrade batch executor to Maven 4.x EmbeddedMavenExecutor API - Add Nx Maven batch-runner module with CLI argument parsing - Create executor implementations for Maven task invocation - Add TypeScript/JavaScript executor schemas and implementations - Support configurable Maven execution with proper error handling

…o 4.0.0-rc-5 - Update all pom.xml files from version 0.0.10 to 0.0.11 - Upgrade Maven 4 dependencies in batch-runner from rc-4 to rc-5 - Update e2e test to use Maven 4.0.0-rc-5 - Add rc-5 paths to MavenHomeDiscovery for better version detection

lourw · 2025-12-17T21:04:42Z

packages/maven/batch-runner/src/main/kotlin/dev/nx/maven/runner/TeeOutputStream.kt

+ * OutputStream that writes to both a capture buffer and streams to stdout in real-time.
+ * (stdout is inherited by parent process, stderr is used for result JSON lines)
+ */
+class TeeOutputStream(


Is this actually used anywhere?

github-actions · 2025-12-24T00:12:14Z

This pull request has already been merged/closed. If you experience issues related to these changes, please open a new issue referencing this pull request.

vercel bot had a problem deploying to Preview October 23, 2025 22:50 Failure

This comment was marked as outdated.

Sign in to view

vercel bot had a problem deploying to Preview October 24, 2025 00:31 Failure

This comment was marked as outdated.

Sign in to view

vercel bot had a problem deploying to Preview October 24, 2025 20:25 Failure

FrozenPandaz force-pushed the mvn-batch-spike branch from 030d9e0 to 6b7b8c4 Compare October 24, 2025 22:47

vercel bot had a problem deploying to Preview October 24, 2025 22:57 Failure

This comment was marked as outdated.

Sign in to view

vercel bot deployed to Preview October 28, 2025 13:42 View deployment

This comment was marked as outdated.

Sign in to view

FrozenPandaz force-pushed the mvn-batch-spike branch from a5fdb17 to 0cf765b Compare October 31, 2025 13:51

vercel bot deployed to Preview October 31, 2025 13:56 View deployment

This comment was marked as outdated.

Sign in to view

vercel bot had a problem deploying to Preview October 31, 2025 22:37 Failure

FrozenPandaz force-pushed the mvn-batch-spike branch from 11b5c2e to 8491549 Compare November 3, 2025 18:42

vercel bot had a problem deploying to Preview November 3, 2025 18:42 Failure

vercel bot had a problem deploying to Preview November 3, 2025 23:02 Failure

vercel bot had a problem deploying to Preview November 3, 2025 23:04 Failure

vercel bot had a problem deploying to Preview November 3, 2025 23:05 Failure

vercel bot had a problem deploying to Preview November 3, 2025 23:06 Failure

vercel bot had a problem deploying to Preview November 3, 2025 23:09 Failure

vercel bot had a problem deploying to Preview November 3, 2025 23:10 Failure

vercel bot had a problem deploying to Preview November 3, 2025 23:11 Failure

vercel bot had a problem deploying to Preview November 3, 2025 23:12 Failure

vercel bot had a problem deploying to Preview November 3, 2025 23:13 Failure

vercel bot had a problem deploying to Preview November 3, 2025 23:42 Failure

vercel bot had a problem deploying to Preview November 4, 2025 05:44 Failure

vercel bot had a problem deploying to Preview November 27, 2025 17:34 Failure

vercel bot deployed to Preview November 27, 2025 18:15 View deployment

vercel bot deployed to Preview November 28, 2025 23:01 View deployment

This comment was marked as outdated.

Sign in to view

lourw reviewed Dec 10, 2025

View reviewed changes

This comment was marked as outdated.

Sign in to view

cursor bot reviewed Dec 15, 2025

View reviewed changes

nx-cloud bot reviewed Dec 16, 2025

View reviewed changes

FrozenPandaz and others added 5 commits December 17, 2025 11:55

chore(maven): address PR review feedback

63b05af

fix(maven): record and apply changes to pomFile

623e70b

chore(repo): format files

1adecf5

lourw reviewed Dec 17, 2025

View reviewed changes

lourw approved these changes Dec 17, 2025

View reviewed changes

Conversation

FrozenPandaz commented Oct 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

1. Batch Runner JAR (packages/maven/batch-runner)

2. TypeScript Executors (packages/maven/src/executors/maven)

3. Shared Utilities (packages/maven/shared)

4. Maven Plugin Updates (packages/maven/maven-plugin)

Performance

Version Support

Testing

Current Behavior

Expected Behavior

Related Issue(s)

Uh oh!

netlify bot commented Oct 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for nx-docs ready!

Uh oh!

vercel bot commented Oct 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nx-cloud bot commented Oct 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

github-actions bot commented Nov 27, 2025

Uh oh!

github-actions bot commented Nov 27, 2025

🐳 We have a release for that!

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

github-actions bot commented Dec 2, 2025

🐳 We have a release for that!

Uh oh!

This comment was marked as outdated.

Uh oh!

github-actions bot commented Dec 2, 2025

Uh oh!

This comment was marked as outdated.

Uh oh!

github-actions bot commented Dec 3, 2025

🐳 We have a release for that!

Uh oh!

lourw Dec 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lourw Dec 10, 2025

Choose a reason for hiding this comment

Uh oh!

lourw Dec 10, 2025

Choose a reason for hiding this comment

Uh oh!

lourw Dec 10, 2025

Choose a reason for hiding this comment

Uh oh!

lourw Dec 10, 2025

Choose a reason for hiding this comment

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

cursor bot left a comment

FrozenPandaz commented Oct 23, 2025 •

edited

Loading

1. Batch Runner JAR (`packages/maven/batch-runner`)

2. TypeScript Executors (`packages/maven/src/executors/maven`)

3. Shared Utilities (`packages/maven/shared`)

4. Maven Plugin Updates (`packages/maven/maven-plugin`)

netlify bot commented Oct 23, 2025 •

edited

Loading

vercel bot commented Oct 23, 2025 •

edited

Loading

nx-cloud bot commented Oct 23, 2025 •

edited

Loading

lourw Dec 10, 2025 •

edited

Loading

nx-cloud bot left a comment •

edited

Loading