Skip to content

feat: auto add MCP projects servers with auth API key mode#9891

Merged
edwinjosechittilappilly merged 77 commits intomainfrom
lf-auto-add-mcp-servers
Sep 30, 2025
Merged

feat: auto add MCP projects servers with auth API key mode#9891
edwinjosechittilappilly merged 77 commits intomainfrom
lf-auto-add-mcp-servers

Conversation

@edwinjosechittilappilly
Copy link
Collaborator

@edwinjosechittilappilly edwinjosechittilappilly commented Sep 17, 2025

This pull request introduces significant improvements to the MCP (Multi-Component Project) server configuration and API utilities in the backend. The main changes include refactoring and centralizing MCP utility functions, enhancing the auto-configuration process for starter projects, and improving runtime port handling for better compatibility. These updates result in cleaner code organization, easier maintenance, and more robust handling of project/server relationships.

MCP Server Configuration and Starter Projects:

  • Added auto_configure_starter_projects_mcp, get_project_sse_url, and get_url_by_os utility functions to langflow.api.utils.mcp, centralizing MCP logic and enabling automatic MCP server setup for starter projects per user at startup. [1] [2] [3] [4]
  • Implemented validate_mcp_server_for_project to robustly check for server/project conflicts and ensure correct server assignment, preventing accidental overwrites or mismatches.

API Utilities Refactoring:

  • Refactored langflow.api.utils to re-export all core utilities for backward compatibility and improved code organization, including explicit __all__ exports for IDE/documentation support.
  • Created a dedicated MCP utilities module (langflow.api.utils.mcp) with clear exports for MCP-related helper functions.

Runtime Port Handling:

  • Updated server startup code to store the runtime-detected port in settings, ensuring that subsequent operations (such as SSE URL generation) use the correct port even if the requested port is unavailable.

Codebase Cleanup and Consistency:

  • Removed duplicate implementations of get_project_sse_url and get_url_by_os from mcp_projects.py, replacing them with imports from the new MCP utilities module for consistency and maintainability. [1] [2]
  • Standardized use of the ALL_INTERFACES_HOST constant throughout the codebase for clarity and consistency. [1] [2]

Logging and Debugging Enhancements:

  • Improved debug logging in MCP server initialization and auto-configuration routines to aid troubleshooting and provide better visibility into startup processes. [1] [2]

These changes collectively improve the reliability, maintainability, and scalability of MCP server management and API utilities in the project.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Sep 17, 2025

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Walkthrough

Implements settings to record the actual server port, refactors MCP project utilities (host/port/SSE URL resolution and config path retrieval), adds project-scoped MCP handler methods, and auto-registers MCP servers upon project creation when enabled by settings.

Changes

Cohort / File(s) Summary
Settings additions
src/lfx/src/lfx/services/settings/base.py
Adds Settings.current_port (int|None) and Settings.add_projects_to_mcp_servers (bool).
Server startup port resolution
src/backend/base/langflow/__main__.py
After resolving the listening port, persists it to settings.current_port before proceeding.
MCP projects utilities and endpoints
src/backend/base/langflow/api/v1/mcp_projects.py
Introduces ALL_INTERFACES_HOST; updates is_local_ip usage; refactors get_project_sse_url to derive host/port from settings with current_port fallback and WSL handling; adds async get_config_path(client) for platform-specific config locations; extends ProjectMCPServer with project-scoped handlers: list tools, list prompts, list resources, read resource, and call tool.
Project creation auto MCP registration
src/backend/base/langflow/api/v1/projects.py
After project creation, when add_projects_to_mcp_servers is true: creates API key, encrypts auth settings to API-key mode, computes project SSE URL, builds mcp-proxy config, sanitizes/limits server name, and registers/updates the MCP server; errors are logged without blocking.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  actor User
  participant CLI as CLI/Entrypoint
  participant Server as Langflow Server
  participant Settings as SettingsService

  User->>CLI: Start server (port P requested)
  CLI->>Server: Resolve available port (may adjust P)
  Server-->>CLI: Final port = PF
  CLI->>Settings: settings.current_port = PF
  Note right of Settings: Records actual listening port
  CLI->>Server: Continue startup (protocol, DB, routes)
  Server-->>User: Server ready on host:PF
Loading
sequenceDiagram
  autonumber
  actor Client as API Client
  participant API as Projects API
  participant Settings as SettingsService
  participant Keys as API Key Service
  participant MCP as MCP Registry
  participant SSE as get_project_sse_url

  Client->>API: Create project
  API-->>Client: Project created
  API->>Settings: Check add_projects_to_mcp_servers
  alt Enabled
    API->>Keys: Create API key
    API->>API: Encrypt project auth settings (API key)
    API->>SSE: Compute project SSE URL (host/port via settings.current_port)
    API->>MCP: update_server(name, command, env/headers incl. API key & SSE URL)
    MCP-->>API: Registered/updated
  else Disabled
    Note over API: Skip MCP registration
  end
Loading
sequenceDiagram
  autonumber
  actor MCPClient as MCP Client
  participant ProjectSrv as ProjectMCPServer
  participant Core as Core Handlers

  MCPClient->>ProjectSrv: listTools/readResource/callTool (project-scoped)
  ProjectSrv->>Core: Delegate to existing handlers with project_id
  Core-->>ProjectSrv: Results
  ProjectSrv-->>MCPClient: Response
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

Suggested labels

enhancement, size:XL, lgtm

Suggested reviewers

  • lucaseduoli
  • ogabrielluiz
  • phact

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 75.00% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check ✅ Passed The title succinctly and accurately describes the main change: automatic addition of MCP project servers using API key authentication. It is concise, focused on the primary feature introduced in the changeset, and clear enough for teammates scanning history to understand the intent.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions github-actions bot added enhancement New feature or request and removed enhancement New feature or request labels Sep 17, 2025
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (3)
src/lfx/src/lfx/services/settings/base.py (1)

217-218: Make current_port ephemeral (avoid persisting to YAML).

current_port is runtime-only. Exclude it from model_dump to prevent leaking transient state into saved configs.

-    current_port: int | None = None
+    current_port: int | None = Field(default=None, exclude=True)
src/backend/base/langflow/api/v1/projects.py (1)

86-125: Auto‑register MCP servers: tweak server name length and consider backgrounding.

  • Off‑by‑one in truncation: subtract len('lf-') so the total length respects MAX_MCP_SERVER_NAME_LENGTH exactly.
  • Optional: run the registration in a background task to avoid adding latency to project creation (keep current try/except).
-                server_name = f"lf-{sanitize_mcp_name(new_project.name)[: (MAX_MCP_SERVER_NAME_LENGTH - 4)]}"
+                server_name = f"lf-{sanitize_mcp_name(new_project.name)[: (MAX_MCP_SERVER_NAME_LENGTH - len('lf-'))]}"
src/backend/base/langflow/api/v1/mcp_projects.py (1)

894-904: Prefer https in SSE URL when TLS is enabled.

get_project_sse_url always builds http://…, which breaks when running with cert/key. Detect TLS from settings and set the scheme.

-    base_url = f"http://{host}:{port}".rstrip("/")
+    protocol = "https" if getattr(settings_service.settings, "ssl_cert_file", None) and getattr(
+        settings_service.settings, "ssl_key_file", None
+    ) else "http"
+    base_url = f"{protocol}://{host}:{port}".rstrip("/")
📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 859f8da and 420dd6a.

📒 Files selected for processing (4)
  • src/backend/base/langflow/__main__.py (1 hunks)
  • src/backend/base/langflow/api/v1/mcp_projects.py (3 hunks)
  • src/backend/base/langflow/api/v1/projects.py (3 hunks)
  • src/lfx/src/lfx/services/settings/base.py (2 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
{src/backend/**/*.py,tests/**/*.py,Makefile}

📄 CodeRabbit inference engine (.cursor/rules/backend_development.mdc)

{src/backend/**/*.py,tests/**/*.py,Makefile}: Run make format_backend to format Python code before linting or committing changes
Run make lint to perform linting checks on backend Python code

Files:

  • src/backend/base/langflow/__main__.py
  • src/backend/base/langflow/api/v1/projects.py
  • src/backend/base/langflow/api/v1/mcp_projects.py
🧠 Learnings (2)
📓 Common learnings
Learnt from: deon-sanchez
PR: langflow-ai/langflow#9158
File: src/backend/base/langflow/api/v1/mcp_projects.py:404-404
Timestamp: 2025-07-23T21:19:22.567Z
Learning: In langflow MCP projects configuration, prefer using dynamically computed URLs (like the `sse_url` variable) over hardcoded localhost URLs to ensure compatibility across different deployment environments.
📚 Learning: 2025-07-23T21:19:22.567Z
Learnt from: deon-sanchez
PR: langflow-ai/langflow#9158
File: src/backend/base/langflow/api/v1/mcp_projects.py:404-404
Timestamp: 2025-07-23T21:19:22.567Z
Learning: In langflow MCP projects configuration, prefer using dynamically computed URLs (like the `sse_url` variable) over hardcoded localhost URLs to ensure compatibility across different deployment environments.

Applied to files:

  • src/backend/base/langflow/api/v1/mcp_projects.py
🧬 Code graph analysis (3)
src/backend/base/langflow/__main__.py (1)
src/backend/base/langflow/services/deps.py (1)
  • get_settings_service (111-124)
src/backend/base/langflow/api/v1/projects.py (7)
src/frontend/src/constants/constants.ts (1)
  • MAX_MCP_SERVER_NAME_LENGTH (562-562)
src/backend/base/langflow/api/v1/mcp_projects.py (1)
  • get_project_sse_url (890-930)
src/backend/base/langflow/api/v2/mcp.py (1)
  • update_server (201-245)
src/backend/base/langflow/services/auth/mcp_encryption.py (1)
  • encrypt_auth_settings (18-51)
src/backend/base/langflow/services/database/models/api_key/crud.py (1)
  • create_api_key (24-40)
src/backend/base/langflow/services/database/models/api_key/model.py (1)
  • ApiKeyCreate (39-47)
src/backend/base/langflow/services/deps.py (2)
  • get_settings_service (111-124)
  • get_storage_service (88-96)
src/backend/base/langflow/api/v1/mcp_projects.py (1)
src/backend/tests/unit/api/v2/test_mcp_servers_file.py (1)
  • settings_service (95-96)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (16)
  • GitHub Check: Lint Backend / Run Mypy (3.10)
  • GitHub Check: Lint Backend / Run Mypy (3.11)
  • GitHub Check: Lint Backend / Run Mypy (3.12)
  • GitHub Check: Lint Backend / Run Mypy (3.13)
  • GitHub Check: Run Frontend Tests / Determine Test Suites and Shard Distribution
  • GitHub Check: Run Backend Tests / Unit Tests - Python 3.10 - Group 5
  • GitHub Check: Run Backend Tests / Unit Tests - Python 3.10 - Group 2
  • GitHub Check: Run Backend Tests / Unit Tests - Python 3.10 - Group 4
  • GitHub Check: Run Backend Tests / Unit Tests - Python 3.10 - Group 1
  • GitHub Check: Run Backend Tests / Unit Tests - Python 3.10 - Group 3
  • GitHub Check: Test Starter Templates
  • GitHub Check: Run Backend Tests / Integration Tests - Python 3.10
  • GitHub Check: Ruff Style Check (3.13)
  • GitHub Check: Optimize new Python code in this PR
  • GitHub Check: Update Starter Projects
  • GitHub Check: Run Ruff Check and Format
🔇 Additional comments (4)
src/lfx/src/lfx/services/settings/base.py (1)

268-272: LGTM: feature flag is clear and env‑wired.

Consider adding this to the user‑visible settings docs/ENV list so operators know about LANGFLOW_ADD_PROJECTS_TO_MCP_SERVERS.

src/backend/base/langflow/api/v1/projects.py (1)

15-17: Imports for MCP auto‑registration look correct.

No issues spotted with symbol locations or dependency direction.

Also applies to: 24-24, 26-26, 30-33, 43-43

src/backend/base/langflow/api/v1/mcp_projects.py (2)

51-53: LGTM: centralized ALL_INTERFACES_HOST.

Good replacement for hardcoded "0.0.0.0".


454-456: LGTM: use ALL_INTERFACES_HOST in local IP check.

Keeps logic consistent and readable.

@codecov
Copy link

codecov bot commented Sep 17, 2025

Codecov Report

❌ Patch coverage is 61.09091% with 107 lines in your changes missing coverage. Please review.
✅ Project coverage is 24.14%. Comparing base (9c2628b) to head (a873c50).
⚠️ Report is 4 commits behind head on main.

Files with missing lines Patch % Lines
src/backend/base/langflow/api/v1/projects.py 33.33% 56 Missing ⚠️
...ackend/base/langflow/api/utils/mcp/config_utils.py 76.27% 42 Missing ⚠️
src/frontend/src/utils/stringManipulation.ts 0.00% 3 Missing ⚠️
src/backend/base/langflow/api/v1/mcp_projects.py 60.00% 2 Missing ⚠️
...sideBarFolderButtons/components/select-options.tsx 0.00% 2 Missing ⚠️
src/backend/base/langflow/main.py 50.00% 1 Missing ⚠️
.../pages/SettingsPage/pages/MCPServersPage/index.tsx 0.00% 1 Missing ⚠️

❌ Your project status has failed because the head coverage (47.04%) is below the target coverage (55.00%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #9891      +/-   ##
==========================================
+ Coverage   23.68%   24.14%   +0.45%     
==========================================
  Files        1090     1091       +1     
  Lines       39768    40014     +246     
  Branches     5542     5543       +1     
==========================================
+ Hits         9421     9660     +239     
- Misses      30176    30183       +7     
  Partials      171      171              
Flag Coverage Δ
backend 47.04% <62.45%> (+0.83%) ⬆️
frontend 10.11% <0.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/backend/base/langflow/__main__.py 55.36% <100.00%> (+0.09%) ⬆️
src/backend/base/langflow/api/utils/core.py 55.90% <ø> (ø)
.../controllers/API/queries/mcp/use-add-mcp-server.ts 0.00% <ø> (ø)
...controllers/API/queries/mcp/use-patch-flows-mcp.ts 0.00% <ø> (ø)
...ntrollers/API/queries/mcp/use-patch-install-mcp.ts 0.00% <ø> (ø)
...ontrollers/API/queries/mcp/use-patch-mcp-server.ts 0.00% <ø> (ø)
...rc/frontend/src/modals/addMcpServerModal/index.tsx 0.00% <ø> (ø)
...ages/ApiKeysPage/components/ApiKeyHeader/index.tsx 0.00% <ø> (ø)
...GeneralPage/components/GeneralPageHeader/index.tsx 0.00% <ø> (ø)
...s/SettingsPage/pages/GlobalVariablesPage/index.tsx 0.00% <ø> (ø)
... and 10 more

... and 6 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions github-actions bot added enhancement New feature or request and removed enhancement New feature or request labels Sep 17, 2025
@github-actions github-actions bot added enhancement New feature or request and removed enhancement New feature or request labels Sep 17, 2025
@github-actions github-actions bot added enhancement New feature or request and removed enhancement New feature or request labels Sep 17, 2025
@github-actions github-actions bot added enhancement New feature or request and removed enhancement New feature or request labels Sep 17, 2025
@jordanrfrazier jordanrfrazier added the DO NOT MERGE Don't Merge this PR label Sep 18, 2025
@jordanrfrazier
Copy link
Collaborator

Added Do Not Merge label until we determine intersection between this work and the pending release merge

@github-actions github-actions bot added enhancement New feature or request and removed enhancement New feature or request labels Sep 18, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Sep 29, 2025

Frontend Unit Test Coverage Report

Coverage Summary

Lines Statements Branches Functions
Coverage: 11%
10.96% (2889/26352) 4.57% (926/20233) 6.48% (370/5702)

Unit Test Results

Tests Skipped Failures Errors Time
1195 0 💤 0 ❌ 0 🔥 18.188s ⏱️

assert str(test_project.id) in result.conflict_message

# Cleanup - delete the server
await client.delete(f"/api/v2/mcp/servers/{server_name}", headers={"x-api-key": created_api_key.api_key})
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: could put the delete into a fixture so we don't forget it in any test

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we connect on more details, I would be happy to add but I have few queries.

# Note: Due to database constraints, mcp_enabled defaults to False instead of None
# The auto_configure function only acts when mcp_enabled is None, so in this case
# the flow configuration won't be updated. This test verifies the function runs
# without error even when flows are already configured (mcp_enabled=False).
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be True instead of None? Not sure I understand the None aspect otherwise.

…tings pages for testing purposes

🔧 (frontend): add convertTestName utility function for converting test names to lowercase with hyphens
🔧 (frontend): add cleanOldFolders utility function to remove old folders in testing environment
🔧 (frontend): add navigateSettingsPages utility function to navigate to specific settings pages in testing
…havior

✨ (use-patch-flows-mcp.ts): add retry: 0 option to improve mutation behavior
✨ (use-patch-install-mcp.ts): add retry: 0 option to improve mutation behavior
✨ (use-patch-mcp-server.ts): add retry: 0 option to improve mutation behavior
📝 (mcp-server-starter-projects.spec.ts): add test to prevent adding duplicate mcp servers from starter projects
…orTimeout to 5000ms for stability

⬆️ (starter-projects.spec.ts): adjust timeout value to 5000 * 3 for better performance and reliability
@codeflash-ai
Copy link
Contributor

codeflash-ai bot commented Sep 30, 2025

⚡️ Codeflash found optimizations for this PR

📄 12% (0.12x) speedup for is_local_ip in src/backend/base/langflow/api/v1/mcp_projects.py

⏱️ Runtime : 19.4 milliseconds 17.3 milliseconds (best of 82 runs)

A dependent PR with the suggested changes has been created. Please review:

If you approve, it will be merged into this PR (branch lf-auto-add-mcp-servers).

@sonarqubecloud
Copy link

Quality Gate Failed Quality Gate failed

Failed conditions
6 Security Hotspots

See analysis details on SonarQube Cloud

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request lgtm This PR has been approved by a maintainer needs-docs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants