feat: Add pluggable services architecture in lfx and comprehensive testing#10111
feat: Add pluggable services architecture in lfx and comprehensive testing#10111ogabrielluiz merged 20 commits intomainfrom
Conversation
…ager for pluggable service discovery - Added `register_service` decorator to allow services to self-register with the ServiceManager. - Enhanced `ServiceManager` to support multiple service discovery mechanisms, including decorator-based registration, config files, and entry points. - Implemented methods for direct service class registration and plugin discovery from various sources, improving flexibility and extensibility of service management.
- Introduced VariableService class to handle environment variables with in-memory caching. - Added methods for getting, setting, deleting, and listing variables. - Included logging for service initialization and variable operations. - Created an __init__.py file to expose VariableService in the package namespace.
…teardown - Updated LocalStorageService to inherit from both StorageService and Service for improved functionality. - Added a name attribute for service identification. - Implemented an async teardown method for future extensibility, even though no cleanup is currently needed. - Refactored the constructor to ensure proper initialization of both parent classes.
…l logging functionality - Added `BaseTelemetryService` as an abstract base class defining the interface for telemetry services. - Introduced `TelemetryService`, a lightweight implementation that logs telemetry events without sending data. - Created `__init__.py` to expose the telemetry service in the package namespace. - Ensured robust async methods for logging various telemetry events and handling exceptions.
- Added `BaseTracingService` as an abstract base class defining the interface for tracing services. - Implemented `TracingService`, a lightweight version that logs trace events without external integrations. - Included async methods for starting and ending traces, tracing components, and managing logs and outputs. - Enhanced documentation for clarity on method usage and parameters.
- Introduced a new test suite for validating the functionality of the @register_service decorator. - Implemented tests for various service types including LocalStorageService, TelemetryService, and TracingService. - Verified behavior for service registration with and without overrides, ensuring correct service management. - Included tests for custom service implementations and preservation of class functionality. - Enhanced overall test coverage for the service registration mechanism.
- Introduced a suite of unit tests covering edge cases for service registration, lifecycle management, and dependency resolution. - Implemented integration tests to validate service loading from configuration files and environment variables. - Enhanced test coverage for various service types including LocalStorageService, TelemetryService, and VariableService. - Verified behavior for service registration with and without overrides, ensuring correct service management. - Ensured robust handling of error conditions and edge cases in service creation and configuration parsing.
- Introduced comprehensive unit tests for LocalStorageService, TelemetryService, TracingService, and VariableService. - Implemented integration tests to validate the interaction between minimal services. - Ensured robust coverage for file operations, service readiness, and exception handling. - Enhanced documentation within tests for clarity on functionality and expected behavior.
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the WalkthroughIntroduces a pluggable services system with decorator, config, and entry-point discovery; updates ServiceManager for class registration, discovery orchestration, and DI; adds telemetry, tracing, and variable minimal services; adjusts local storage inheritance/lifecycle; exports registration API; adds example config and extensive unit tests; expands docs. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
actor User
participant App as Application
participant SM as ServiceManager
participant EP as Entry Points
participant CFG as Config Files
participant REG as Decorator Registry
participant FAC as Built-in Factories
User->>App: request service(ServiceType)
App->>SM: get(service_type)
alt First-time discovery
SM->>SM: discover_plugins()
SM->>EP: _discover_from_entry_points()
EP-->>SM: register classes
SM->>REG: load decorator registrations
REG-->>SM: register classes
SM->>CFG: _discover_from_config(lfx.toml/pyproject.toml)
CFG-->>SM: register classes from dotted paths
end
alt Class registered for type
SM->>SM: _create_service_from_class()
else Factory available
SM->>FAC: _create_service_from_factory()
else None
SM-->>App: NoServiceRegisteredError
end
SM-->>App: service instance (cached)
sequenceDiagram
autonumber
participant SM as ServiceManager
participant Impl as ServiceClass
note over SM: Dependency Injection on init
SM->>SM: inspect __init__ annotations
loop For each parameter
alt param is Service subclass type
SM->>SM: get(param_service_type)
SM-->>SM: inject dependency
else optional/has default
SM-->>SM: use default/None
else non-service required
SM-->>SM: raise TypeError
end
end
SM->>Impl: instantiate with resolved deps
Impl-->>SM: instance (ready or sets ready)
Estimated code review effort🎯 4 (Complex) | ⏱️ ~75 minutes Suggested labels
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (6 passed)
✨ Finishing touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…ection - Revised the documentation to highlight the advantages of the pluggable service system. - Replaced the migration guide with a detailed overview of features such as automatic discovery, lazy instantiation, dependency injection, and lifecycle management. - Clarified examples of service registration and improved overall documentation for better understanding.
|
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
src/lfx/src/lfx/services/manager.py (1)
225-255: Use the resolved factory instance instead of indexing the dict again
When we fall back to adefaultfactory, we callregister_factory(default)which stores it underdefault.service_class.name(a string such as"storage_service"). Immediately after, we still accessself.factories[service_name], whereservice_nameis aServiceType. If the enum value doesn’t hash-identically to that string (e.g. when using plainEnuminstead ofStrEnum), this will raise aKeyErroror return the wrong factory. We already have the correct factory selected in the localfactoryvariable—use it directly to avoid mismatched key lookups.- self.services[service_name] = self.factories[service_name].create(**dependent_services) + self.services[service_name] = factory.create(**dependent_services)
🧹 Nitpick comments (8)
src/lfx/src/lfx/services/variable/service.py (1)
18-83: Consider security implications of in-memory variable storage.Variables are stored in plain memory without encryption. If sensitive data (API keys, secrets) is stored via
set_variable, it remains in memory until teardown and could be exposed through memory dumps or debugging.For production use, consider:
- Adding a warning in the docstring about not storing secrets
- Or implementing memory encryption for sensitive values
- Or directing users to use environment variables for secrets (which are also plain text but at least OS-managed)
Example docstring addition:
def set_variable(self, name: str, value: str, **kwargs) -> None: # noqa: ARG002 - """Set a variable value (in-memory only). + """Set a variable value (in-memory only). + + Warning: Values are stored in plain memory. Do not use for secrets. + For sensitive data, use environment variables instead. Args: name: Variable namesrc/lfx/src/lfx/services/telemetry/base.py (1)
21-24: Consider removing@abstractmethodfrom__init__.Making
__init__abstract is unusual and may be overly restrictive. Subclasses must override__init__even if the base initialization logic would suffice. Typically, only behavior-defining methods (not initialization) are marked abstract.If all subclasses need custom initialization, keep it abstract. Otherwise, consider making
__init__concrete:- @abstractmethod def __init__(self): """Initialize the telemetry service.""" super().__init__()This allows subclasses to call
super().__init__()without being forced to override.src/lfx/src/lfx/services/registry.py (1)
36-52: Consider more specific exception handling.The broad
except Exceptionclause (line 47) logs a warning but continues execution, potentially hiding unexpected registration errors. While this prevents decorator application from breaking the code, it can make debugging difficult when registration silently fails.Consider catching more specific exceptions or providing a way to fail fast in development:
try: from lfx.services.manager import get_service_manager service_manager = get_service_manager() service_manager.register_service_class(service_type, service_class, override=override) logger.debug(f"Registered service via decorator: {service_type.value} -> {service_class.__name__}") except ValueError: # Re-raise ValueError (used for settings service protection) raise + except (ImportError, AttributeError) as exc: + # Expected errors during import/registration + logger.warning(f"Failed to register service {service_type.value} from decorator: {exc}") - except Exception as exc: # noqa: BLE001 + except Exception as exc: - logger.warning(f"Failed to register service {service_type.value} from decorator: {exc}") + # Unexpected error - log and optionally fail in development + logger.error(f"Unexpected error registering service {service_type.value}: {exc}", exc_info=True) + # Consider: raise in development mode return service_classAlternatively, add an environment variable to control whether to raise or warn on unexpected errors.
src/lfx/PLUGGABLE_SERVICES.md (1)
124-125: Resolve markdownlint MD036 warningmarkdownlint flagged the
**Important:**line as “emphasis used instead of a heading.” Please switch to a proper heading (for example,### Important) or rephrase to satisfy the lint rule and keep CI clean.src/lfx/tests/unit/services/test_edge_cases.py (1)
13-22: Deduplicate the repeatedclean_managerfixtureEach test class redefines the same
clean_managerfixture. Consider promoting it to a module-level (or conftest) fixture so it’s declared once and reused everywhere. That trims duplication and keeps future updates to the teardown logic in one location.src/lfx/tests/unit/services/test_integration.py (1)
15-24: Extract the sharedclean_managerfixtureMultiple classes in this module redeclare an identical
clean_managerfixture. Moving it to a single module-level (or shared) fixture avoids repetition and keeps the teardown logic consistent everywhere.src/lfx/tests/unit/services/test_minimal_services.py (2)
166-173: Consider using pytest's monkeypatch for environment variables.The manual try/finally blocks for environment variable cleanup work correctly, but pytest's
monkeypatchfixture provides cleaner and more robust test isolation with automatic cleanup.Example refactor using monkeypatch:
def test_get_from_environment(self, variables, monkeypatch): """Test getting variable from environment.""" monkeypatch.setenv("TEST_ENV_VAR", "env_value") value = variables.get_variable("TEST_ENV_VAR") assert value == "env_value" # Automatic cleanup by monkeypatch def test_in_memory_overrides_env(self, variables, monkeypatch): """Test that in-memory variables override environment.""" monkeypatch.setenv("TEST_VAR", "env_value") variables.set_variable("TEST_VAR", "memory_value") value = variables.get_variable("TEST_VAR") assert value == "memory_value" # Automatic cleanup by monkeypatchAlso applies to: 196-204
215-256: Integration tests provide good baseline coverage.The integration tests effectively verify that minimal services can be initialized, torn down, and work together. The basic coverage is appropriate for minimal service implementations.
For more thorough integration testing, consider verifying the actual interactions. For example, in
test_storage_with_tracing, you could verify that tracing logs were recorded:@pytest.mark.asyncio async def test_storage_with_tracing(self, tmp_path): """Test using storage with tracing.""" storage = LocalStorageService(data_dir=tmp_path) tracing = TracingService() # Track that logs were added log_count_before = len(tracing._logs) if hasattr(tracing, '_logs') else 0 tracing.add_log("storage_test", {"operation": "save", "flow_id": "123"}) await storage.save_file("flow_123", "test.txt", b"content") tracing.add_log("storage_test", {"operation": "saved", "flow_id": "123"}) # Verify both storage and tracing worked assert await storage.get_file("flow_123", "test.txt") == b"content" # Optionally verify logs were recorded (if tracing service exposes this)Note: This assumes the tracing service exposes a way to verify logs were recorded, which may not be the case for a minimal implementation.
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (19)
src/lfx/PLUGGABLE_SERVICES.md(1 hunks)src/lfx/README.md(1 hunks)src/lfx/lfx.toml.example(1 hunks)src/lfx/src/lfx/services/__init__.py(2 hunks)src/lfx/src/lfx/services/manager.py(6 hunks)src/lfx/src/lfx/services/registry.py(1 hunks)src/lfx/src/lfx/services/storage/local.py(1 hunks)src/lfx/src/lfx/services/telemetry/__init__.py(1 hunks)src/lfx/src/lfx/services/telemetry/base.py(1 hunks)src/lfx/src/lfx/services/telemetry/service.py(1 hunks)src/lfx/src/lfx/services/tracing/base.py(1 hunks)src/lfx/src/lfx/services/tracing/service.py(1 hunks)src/lfx/src/lfx/services/variable/__init__.py(1 hunks)src/lfx/src/lfx/services/variable/service.py(1 hunks)src/lfx/tests/unit/services/test_decorator_registration.py(1 hunks)src/lfx/tests/unit/services/test_edge_cases.py(1 hunks)src/lfx/tests/unit/services/test_integration.py(1 hunks)src/lfx/tests/unit/services/test_minimal_services.py(1 hunks)src/lfx/tests/unit/services/test_service_manager.py(1 hunks)
🧰 Additional context used
🪛 markdownlint-cli2 (0.18.1)
src/lfx/PLUGGABLE_SERVICES.md
158-158: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
210-210: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
220-220: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
227-227: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
🔇 Additional comments (9)
src/lfx/src/lfx/services/variable/__init__.py (1)
1-5: LGTM!The module correctly exports the VariableService as part of the public API, following standard Python packaging conventions.
src/lfx/src/lfx/services/telemetry/base.py (1)
26-86: LGTM!The abstract method definitions provide a clear and comprehensive interface for telemetry service implementations. Type hints, async declarations, and docstrings are all appropriate.
src/lfx/src/lfx/services/__init__.py (1)
12-12: LGTM!The addition of
register_serviceto the public API correctly exposes the decorator-based registration mechanism, enabling users to register pluggable services.Also applies to: 27-27
src/lfx/README.md (1)
29-71: LGTM!The documentation clearly explains the new pluggable services architecture and improved component access patterns. The before/after examples are helpful, and the reference to PLUGGABLE_SERVICES.md provides a clear path for users seeking detailed information.
src/lfx/src/lfx/services/telemetry/__init__.py (1)
1-5: LGTM!The module correctly exports the TelemetryService as part of the public API, following standard Python packaging conventions.
src/lfx/lfx.toml.example (1)
14-44: Confirm all service paths import correctly from the installed Langflow package. Ensure eachmodule.path:ClassNameinsrc/lfx/lfx.toml.examplemaps to an existing, importable class in your Langflow environment.src/lfx/tests/unit/services/test_minimal_services.py (3)
13-87: Comprehensive test coverage for LocalStorageService.The test suite provides solid coverage of LocalStorageService functionality, including CRUD operations, error handling, and lifecycle management. The use of
tmp_pathfixture ensures proper test isolation.
89-120: Test coverage appropriate for minimal TelemetryService.The tests appropriately verify that the no-op telemetry service methods complete without errors. For a minimal service implementation, this level of testing is acceptable.
122-145: Test coverage appropriate for minimal TracingService.The tests verify basic functionality of the minimal tracing service. The coverage is suitable for a minimal implementation.
8574e7a to
f0130bf
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. ❌ Your project check has failed because the head coverage (40.80%) is below the target coverage (60.00%). You can increase the head coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## main #10111 +/- ##
==========================================
- Coverage 34.24% 34.24% -0.01%
==========================================
Files 1409 1409
Lines 66929 66921 -8
Branches 9877 9877
==========================================
- Hits 22918 22915 -3
+ Misses 42810 42805 -5
Partials 1201 1201
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
|
@coderabbitai configuration |
|
Configuration used: Path: .coderabbit.yaml CodeRabbit configurationlanguage: en-US
tone_instructions: ''
early_access: true
enable_free_tier: true
inheritance: false
reviews:
profile: chill
request_changes_workflow: false
high_level_summary: true
high_level_summary_instructions: ''
high_level_summary_placeholder: '@coderabbitai summary'
high_level_summary_in_walkthrough: false
auto_title_placeholder: '@coderabbitai'
auto_title_instructions: Use conventional commits structure
review_status: true
commit_status: true
fail_commit_status: false
collapse_walkthrough: false
changed_files_summary: true
sequence_diagrams: true
estimate_code_review_effort: true
assess_linked_issues: true
related_issues: true
related_prs: true
suggested_labels: true
labeling_instructions: []
auto_apply_labels: false
suggested_reviewers: true
auto_assign_reviewers: false
in_progress_fortune: true
poem: false
enable_prompt_for_ai_agents: true
path_filters:
- '!src/backend/base/langflow/initial_setup/starter_projects'
path_instructions: []
abort_on_close: true
disable_cache: false
auto_review:
enabled: true
auto_incremental_review: false
ignore_title_keywords: []
labels: []
drafts: false
base_branches: []
ignore_usernames: []
finishing_touches:
docstrings:
enabled: true
unit_tests:
enabled: true
pre_merge_checks:
docstrings:
mode: warning
threshold: 80
title:
mode: warning
requirements: ''
description:
mode: warning
issue_assessment:
mode: warning
custom_checks:
- mode: error
name: Test Coverage for New Implementations
instructions: >
Check if the PR includes new or updated test files that correspond to
the code changes:
1. For new components or functionality, ensure corresponding tests are
included in the PR
2. For bug fixes, verify that regression tests are included
3. For new features, ensure both unit and integration tests are
present where appropriate
4. Check that test files follow the project's naming conventions
(test_*.py for backend, *.test.ts for frontend)
5. Verify that tests actually test the new functionality, not just
placeholder tests
- mode: warning
name: Test Quality and Coverage
instructions: >
Evaluate the quality and comprehensiveness of tests for new
implementations:
1. Tests should cover the main functionality being implemented
2. For async functions, ensure proper async testing patterns are used
(pytest for backend)
3. Check that tests are not just smoke tests but actually validate
behavior
4. Ensure tests follow the project's testing patterns (pytest for
backend, Playwright for frontend)
5. For API endpoints, verify both success and error response testing
- mode: warning
name: Test File Naming and Structure
instructions: >
Verify that test files follow the correct patterns and structure:
1. Backend tests: test_*.py with proper pytest structure
2. Frontend tests: *.test.ts or *.test.tsx using Playwright
3. Integration tests should be clearly marked and in appropriate
directories
4. Test files should have descriptive test function names that explain
what is being tested
5. Tests should be organized logically with proper setup and teardown
6. Consider including edge cases and error conditions for
comprehensive coverage
7. Verify tests cover both positive and negative scenarios where
appropriate
- mode: warning
name: Excessive Mock Usage Warning
instructions: >
Review test files for excessive use of mocks that may indicate poor
test design:
1. Check if tests have too many mock objects that obscure what's
actually being tested
2. Warn when mocks are used instead of testing real behavior and
interactions
3. Suggest using real objects or test doubles when mocks become
excessive
4. Ensure mocks are used appropriately for external dependencies, not
core logic
5. Recommend integration tests when unit tests become overly mocked
tools:
ast-grep:
rule_dirs: []
util_dirs: []
essential_rules: true
packages: []
shellcheck:
enabled: true
ruff:
enabled: true
markdownlint:
enabled: true
github-checks:
enabled: true
timeout_ms: 90000
languagetool:
enabled: true
enabled_rules: []
disabled_rules: []
enabled_categories: []
disabled_categories: []
enabled_only: false
level: default
biome:
enabled: true
hadolint:
enabled: true
swiftlint:
enabled: true
phpstan:
enabled: true
level: default
phpmd:
enabled: true
phpcs:
enabled: true
golangci-lint:
enabled: true
yamllint:
enabled: true
gitleaks:
enabled: true
checkov:
enabled: true
detekt:
enabled: true
eslint:
enabled: true
flake8:
enabled: true
fortitudeLint:
enabled: true
rubocop:
enabled: true
buf:
enabled: true
regal:
enabled: true
actionlint:
enabled: true
pmd:
enabled: true
clang:
enabled: true
cppcheck:
enabled: true
semgrep:
enabled: true
circleci:
enabled: true
clippy:
enabled: true
sqlfluff:
enabled: true
prismaLint:
enabled: true
pylint:
enabled: true
oxc:
enabled: true
shopifyThemeCheck:
enabled: true
luacheck:
enabled: true
brakeman:
enabled: true
dotenvLint:
enabled: true
htmlhint:
enabled: true
checkmake:
enabled: true
osvScanner:
enabled: true
chat:
art: true
auto_reply: true
integrations:
jira:
usage: auto
linear:
usage: auto
knowledge_base:
opt_out: false
web_search:
enabled: true
code_guidelines:
enabled: true
filePatterns: []
learnings:
scope: auto
issues:
scope: auto
jira:
usage: auto
project_keys: []
linear:
usage: auto
team_keys: []
pull_requests:
scope: auto
mcp:
usage: auto
disabled_servers: []
code_generation:
docstrings:
language: en-US
path_instructions: []
unit_tests:
path_instructions:
- path: src/backend/tests/**/*.py
instructions: We already have a client fixture and use pytest
- path: src/frontend/tests/**.ts
instructions: We use Playwright
issue_enrichment:
auto_enrich:
enabled: false
planning:
enabled: true
auto_planning:
enabled: true
labels: []
labeling:
labeling_instructions: []
auto_apply_labels: false
|
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
ca21ebd to
4b9a9a1
Compare
|
|
||
| # Create the service instance | ||
| try: | ||
| service_instance = service_class(**dependencies) |
There was a problem hiding this comment.
Seem like we are storing the partial state even before the validation is complete.
can do something like
| service_instance = service_class(**dependencies) | |
| service_instance = service_class(**dependencies) | |
| # Validate service has required attributes | |
| if not hasattr(service_instance, 'name'): | |
| raise ValueError(f"Service {service_class.__name__} missing 'name' attribute") | |
| ----- |
There was a problem hiding this comment.
- I need to circle back and evaluate this further before making any destabilizing changes
- Not sure if validating properties is sufficient or explicit validation should be done via introducing a
validatelifecycle method or deferring validation to theset_readymethod
HimavarshaVS
left a comment
There was a problem hiding this comment.
few comments to avoid race conditions . But other than that lgtm
jordanrfrazier
left a comment
There was a problem hiding this comment.
Looks good to me. Tested a brief script that plugged in a Custom Service. Didn't look at every line of code, but the idea and architecture look good to me and we can discover any small issues as we utilize this.
src/lfx/src/lfx/services/manager.py
Outdated
| msg = f"Circular dependency detected: {service_name.value} depends on itself" | ||
| raise RuntimeError(msg) | ||
| # Recursively create dependency if not exists | ||
| # Note: Thread safety is handled by the caller's keyed lock context |
There was a problem hiding this comment.
Does this potential race condition exist?
Thread A:
* get(serviceA) -> lock(serviceA) -> dependsOn(ServiceC) -> _create_service(serviceC) (no serviceC locking)
Thread B:
* get(serviceB) -> lock(serviceB) -> dependsOn(ServiceC) -> _create_service(serviceC) (no serviceC locking)
If so, we can do something like:
+ with self.keyed_lock.lock(dependency_type):
+ if dependency_type not in self.services:
+ self._create_service(dependency_type)
+ dependencies[param_name] = self.services[dependency_type]
src/lfx/src/lfx/services/manager.py
Outdated
| # Check for circular dependency (service depending on itself) | ||
| if dependency_type == service_name: | ||
| msg = f"Circular dependency detected: {service_name.value} depends on itself" | ||
| raise RuntimeError(msg) |
There was a problem hiding this comment.
This does only catch A -> A dependencies, I don't believe it will catch A -> B -> A. The error message does concur with that statement, but we may want to add logic to catch the latter scenarios.



Introduce a pluggable services architecture with detailed documentation and an example configuration file. Add unit and integration tests for various services, including LocalStorageService, TelemetryService, and VariableService, ensuring robust coverage and error handling. Implement service registration via decorators to enhance service discovery and management.
Summary by CodeRabbit
New Features
Documentation
Tests