feat: Allow processing full folders in the File Input by erichare · Pull Request #10208 · langflow-ai/langflow

erichare · 2025-10-09T20:39:59Z

Summary by CodeRabbit

New Features
- Optional folder selection in the file upload modal with a File/Folder toggle.
- Folder uploads filter hidden/system files and very large directories, with updated guidance texts.
- Extension filtering applied to folder contents; errors shown if no supported files found.
Improvements
- Safer uploads: directory paths are stripped from uploaded files.
- Custom input and a new example demonstrate folder selection.
Deprecations
- Directory component marked as legacy with a suggested replacement.

coderabbitai · 2025-10-09T20:40:29Z

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

🔍 Trigger a full review

Walkthrough

Adds optional folder selection across the frontend file upload flow. Introduces new props (allowFolderSelection, webkitdirectory), updates UI to toggle File/Folder modes, filters folder contents client-side, adjusts hook/queries to handle directory uploads, adds an example and customization usage, and marks a backend DirectoryComponent as legacy with a replacement.

Changes

Cohort / File(s)	Summary
Core Input Component `src/frontend/src/components/core/parameterRenderComponent/components/inputFileComponent/index.tsx`	Extends InputFileComponent props with allowFolderSelection and passes it to FileManagerModal. Minor refactor.
File Manager Modal and Drag UI `src/frontend/src/modals/fileManagerModal/index.tsx`, `src/frontend/src/modals/fileManagerModal/components/dragFilesComponent/index.tsx`	Adds allowFolderSelection prop; introduces folder mode toggle in DragFilesComponent with updated UI text and behavior; wires flag through modal.
Upload Hooks and Helpers `src/frontend/src/hooks/files/use-upload-file.ts`, `src/frontend/src/helpers/create-file-upload.ts`	Adds webkitdirectory support; filters/validates folder contents; client-side hidden/ignored files filtering; propagates folder mode through getFilesToUpload.
API Upload Handling `src/frontend/src/controllers/API/queries/file-management/use-post-upload-file.ts`	Normalizes uploaded File when webkitRelativePath is present to avoid path leakage; otherwise unchanged.
Customization & Example `src/frontend/src/customization/components/custom-input-file.tsx`, `src/frontend/src/components/examples/folder-selection-example.tsx`	Enables folder selection in customization component; adds example demonstrating folder selection usage and state handling.
Backend Metadata `src/lfx/src/lfx/components/data/directory.py`	Marks DirectoryComponent as legacy and points to replacement ["data.File"].

Sequence Diagram(s)

sequenceDiagram
  autonumber
  actor User
  participant InputFileComponent
  participant FileManagerModal
  participant DragFilesComponent
  participant useUploadFile
  participant createFileUpload
  participant usePostUploadFile as API: usePostUploadFile
  participant Server

  User->>InputFileComponent: Open file picker
  InputFileComponent->>FileManagerModal: open(allowFolderSelection)
  FileManagerModal->>DragFilesComponent: render(allowFolderSelection)

  alt Folder mode ON
    DragFilesComponent->>useUploadFile: upload({ types, multiple, webkitdirectory: true })
    useUploadFile->>createFileUpload: getFiles({ accept, multiple, webkitdirectory: true })
    createFileUpload-->>useUploadFile: File[] (filtered hidden/ignored)
    useUploadFile->>usePostUploadFile: mutate for each valid File
    usePostUploadFile->>usePostUploadFile: normalize File if webkitRelativePath
    usePostUploadFile->>Server: POST FormData(file)
    Server-->>usePostUploadFile: 200/response
  else File mode
    DragFilesComponent->>useUploadFile: upload({ types, multiple, webkitdirectory: false })
    useUploadFile->>createFileUpload: getFiles({ accept, multiple })
    createFileUpload-->>useUploadFile: File[]
    useUploadFile->>usePostUploadFile: mutate per File
    usePostUploadFile->>Server: POST FormData(file)
    Server-->>usePostUploadFile: 200/response
  end

  usePostUploadFile-->>DragFilesComponent: results
  DragFilesComponent-->>User: Selected/uploaded items

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

feat: add InputFile customization #9233 — Touches the same customization wrapper, likely coordinating behavior where allowFolderSelection is now passed through.

Suggested labels

lgtm

Suggested reviewers

lucaseduoli

Pre-merge checks and finishing touches

❌ Failed checks (1 error, 3 warnings)

Check name	Status	Explanation	Resolution
Test Coverage For New Implementations	❌ Error	The PR introduces significant new frontend functionality for folder selection and backend metadata changes, yet it does not add or modify any test files, nor are there existing tests updated to cover these behaviors. I searched the repository for updated or newly added frontend .test.ts files and backend test_.py files related to the touched components (InputFileComponent, FileManagerModal, createFileUpload, useUploadFile, etc.) and found no corresponding test coverage in this PR. Consequently, the new functionality lacks automated verification and fails the requirement that new features include appropriate tests.	Please add unit and, if applicable, integration tests that exercise the new folder-selection workflow (including filtering logic and upload handling) in accordance with the project’s test naming conventions, then resubmit for review.
Docstring Coverage	⚠️ Warning	Docstring coverage is 16.67% which is insufficient. The required threshold is 80.00%.	You can run `@coderabbitai generate docstrings` to improve docstring coverage.
Test Quality And Coverage	⚠️ Warning	No automated tests were added or updated to cover the new folder-selection flow, upload filtering, or filename normalization logic introduced across the frontend helpers, hooks, and modal components, so the primary functionality changes are unverified by the test suite and there is no evidence that existing tests exercise these code paths. Without targeted tests validating folder uploads, hidden-file filtering, extension enforcement, and the adjusted upload payload, the PR does not meet the stated test quality and coverage requirements.	Please add automated tests that follow the project’s testing conventions and thoroughly cover the new folder selection behavior, including success paths, filtering rules, and error handling, then rerun this check.
Test File Naming And Structure	⚠️ Warning	I searched the repository for Frontend or Backend tests covering the newly introduced folder-selection functionality and found no `.test.ts`, `.test.tsx`, or `test_*.py` files referencing the updated components or hooks, indicating that the changes lack accompanying tests. Because the PR modifies public-facing upload behavior without adding tests that meet the stated naming and structural requirements (no Playwright specs, pytest modules, or integration tests documenting positive/negative scenarios), the custom check fails.	Please add appropriately named tests (e.g., Playwright `.test.tsx` for the frontend components or pytest `test_*.py` for backend changes) that exercise folder-selection behavior, cover both successful and failure paths, and include any necessary setup and teardown so the test structure aligns with the repository’s conventions.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title Check	✅ Passed	The title clearly and concisely describes the primary enhancement of enabling full folder processing in the file input components, directly reflecting the main change introduced by the pull request to support folder selection across the UI and upload handling.
Excessive Mock Usage Warning	✅ Passed	Searched repository test files for mocking patterns and found none, indicating no reliance on mocks within the current changeset, so there is no evidence of excessive or inappropriate mock usage to warn about.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

✨ Finishing touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment
Commit unit tests in branch feat-directory-processing

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

coderabbitai

Actionable comments posted: 3

🧹 Nitpick comments (6)

src/frontend/src/modals/fileManagerModal/components/dragFilesComponent/index.tsx (1)
22-114: Reset folder mode when the flag is disabled

isFolderMode can stay true if the parent flips allowFolderSelection to false later, leaving the dropzone in folder mode while the toggle is hidden. Add an effect to clear the state when folder selection is disabled:
 const [isDragging, setIsDragging] = useState(false);
 const [isFolderMode, setIsFolderMode] = useState(false);
+useEffect(() => {
+  if (!allowFolderSelection && isFolderMode) {
+    setIsFolderMode(false);
+  }
+}, [allowFolderSelection, isFolderMode]);
This keeps the UI and behavior aligned if the prop changes dynamically.
src/frontend/src/helpers/create-file-upload.ts (4)
6-6: LGTM! Folder selection parameter added correctly.

The webkitdirectory parameter is properly typed and correctly sets the input element properties. The requirement that multiple must be true when using webkitdirectory is correctly enforced.

Minor suggestion: Consider adding a brief comment explaining that webkitdirectory is a non-standard but widely-supported browser API, which justifies the type assertion on line 22.

Also applies to: 20-24

77-122: Reduce redundancy in filtering logic.

The filtering logic has significant redundancy that reduces maintainability:

Lines 84-87 check for .mypy_cache specifically

Lines 90-101 check if any path part starts with . (which already catches .mypy_cache)

Lines 104-119 check for common hidden patterns including .git, .vscode, etc. (which are also caught by the dot-prefix check)

The check on lines 90-101 for parts starting with . will catch all the patterns in the hiddenPatterns array (lines 104-111) except __pycache__ and potentially .DS_Store if it's not a folder.

Consider consolidating to:
 files = files.filter((file) => {
   const path = file.webkitRelativePath;
   const pathParts = path.split("/");

-  // Multiple approaches to catch hidden files/folders:
-
-  // 1. Check if path contains .mypy_cache specifically
-  if (path.includes(".mypy_cache")) {
-    console.log("Filtering out .mypy_cache file:", path);
-    return false;
-  }
-
-  // 2. Check if any path part starts with '.'
+  // Check if any path part starts with '.' (catches .git, .vscode, .mypy_cache, etc.)
   const hasHiddenPath = pathParts.some((part) => {
-    const startsWithDot = part.startsWith(".") && part.length > 0;
-    if (startsWithDot) {
-      console.log(`Found hidden part "${part}" in path: ${path}`);
-    }
-    return startsWithDot;
+    return part.startsWith(".") && part.length > 0;
   });

   if (hasHiddenPath) {
-    console.log("Filtering out hidden path:", path);
     return false;
   }

-  // 3. Additional check for common hidden patterns
-  const hiddenPatterns = [
-    ".DS_Store",
-    ".git",
-    ".vscode",
-    ".idea",
-    "__pycache__",
-    ".pytest_cache",
-  ];
-  const hasHiddenPattern = hiddenPatterns.some((pattern) =>
-    path.includes(pattern),
-  );
-
-  if (hasHiddenPattern) {
-    console.log("Filtering out file with hidden pattern:", path);
+  // Check for __pycache__ separately since it doesn't start with '.'
+  if (path.includes("__pycache__")) {
     return false;
   }

   return true;
 });
60-75: Reduce console logging verbosity for production.

The current implementation logs extensively during folder filtering, which could clutter the console in production environments:

Lines 60-75: Pre-filtering diagnostics

Lines 85, 93, 99, 117: Per-file filtering logs

Lines 124-138: Post-filtering diagnostics

For a folder with hundreds of files and many hidden files, this could generate thousands of console messages.

Consider one of these approaches:

Remove detailed per-file logging and keep only summary logs:
-        // First, let's see what we're dealing with
-        console.log("Before filtering - total files:", originalCount);
-        console.log(
-          "Sample paths before filtering:",
-          files.slice(0, 10).map((f) => f.webkitRelativePath),
-        );
-
-        // Show a few .mypy_cache examples if they exist
-        const mypyCacheFiles = files
-          .filter((f) => f.webkitRelativePath.includes(".mypy_cache"))
-          .slice(0, 3);
-        if (mypyCacheFiles.length > 0) {
-          console.log(
-            "Found .mypy_cache files (sample):",
-            mypyCacheFiles.map((f) => f.webkitRelativePath),
-          );
-        }

         files = files.filter((file) => {
           const path = file.webkitRelativePath;
           const pathParts = path.split("/");

           if (path.includes(".mypy_cache")) {
-            console.log("Filtering out .mypy_cache file:", path);
             return false;
           }

           const hasHiddenPath = pathParts.some((part) => {
             const startsWithDot = part.startsWith(".") && part.length > 0;
-            if (startsWithDot) {
-              console.log(`Found hidden part "${part}" in path: ${path}`);
-            }
             return startsWithDot;
           });

           if (hasHiddenPath) {
-            console.log("Filtering out hidden path:", path);
             return false;
           }

           const hiddenPatterns = [
             ".DS_Store",
             ".git",
             ".vscode",
             ".idea",
             "__pycache__",
             ".pytest_cache",
           ];
           const hasHiddenPattern = hiddenPatterns.some((pattern) =>
             path.includes(pattern),
           );

           if (hasHiddenPattern) {
-            console.log("Filtering out file with hidden pattern:", path);
             return false;
           }

           return true;
         });

-        // Debug logging for folder selection
         console.log("Folder selection results:");
         console.log(
           `Total files found: ${originalCount}, after filtering hidden files/folders: ${files.length}`,
         );
-        if (files.length > 0) {
-          console.log(
-            "Remaining files (first 5):",
-            files.slice(0, 5).map((f) => ({
-              name: f.name,
-              webkitRelativePath: f.webkitRelativePath,
-              size: f.size,
-            })),
-          );
-        }
Use a debug flag to conditionally enable verbose logging only in development.

Also applies to: 85-85, 93-93, 99-99, 117-117, 124-138

49-56: Consider making the file limit configurable.

The hard-coded limit of 1000 files (line 49) may be too restrictive for legitimate large project folders that don't contain problematic hidden directories.

Consider either:

Making this limit a parameter in the function signature with a sensible default

Increasing the limit to a higher value (e.g., 5000)

Providing clearer guidance in the error message about which specific hidden directories were detected
src/frontend/src/hooks/files/use-upload-file.ts (1)
69-71: Move multiple file check outside the loop.

The check for multiple files (lines 69-71) is currently inside the for loop and uses filesToUpload.length. While this is safe because it's within the if (!webkitdirectory) block, it's inefficient because:

The check will be executed on every iteration of the loop

The error will be thrown on the first file, but after validateFileSize has already been called

Move this check before the loop:
+      // Check multiple file constraint for non-folder selection
+      if (!webkitdirectory && !multiple && filesToUpload.length !== 1) {
+        throw new Error("Multiple files are not allowed");
+      }
+
       for (const file of validFiles) {
         validateFileSize(file);
         // Check if file extension is allowed (for non-folder selection)
         if (!webkitdirectory) {
           const fileExtension = file.name.split(".").pop()?.toLowerCase();
           if (!fileExtension || (types && !types.includes(fileExtension))) {
             throw new Error(
               `File type ${fileExtension} not allowed. Allowed types: ${types?.join(", ")}`,
             );
           }
-          if (!multiple && filesToUpload.length !== 1) {
-            throw new Error("Multiple files are not allowed");
-          }
         }

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between d12bb50 and 2f58c54.

📒 Files selected for processing (9)

src/frontend/src/components/core/parameterRenderComponent/components/inputFileComponent/index.tsx (4 hunks)
src/frontend/src/components/examples/folder-selection-example.tsx (1 hunks)
src/frontend/src/controllers/API/queries/file-management/use-post-upload-file.ts (1 hunks)
src/frontend/src/customization/components/custom-input-file.tsx (1 hunks)
src/frontend/src/helpers/create-file-upload.ts (3 hunks)
src/frontend/src/hooks/files/use-upload-file.ts (3 hunks)
src/frontend/src/modals/fileManagerModal/components/dragFilesComponent/index.tsx (4 hunks)
src/frontend/src/modals/fileManagerModal/index.tsx (3 hunks)
src/lfx/src/lfx/components/data/directory.py (1 hunks)

🧰 Additional context used

📓 Path-based instructions (5)

src/frontend/src/**/*.{ts,tsx,js,jsx}