Use swift-huggingface to interact with Hub API by mattt · Pull Request #297 · huggingface/swift-transformers

mattt · 2025-12-05T12:15:20Z

This PR updates HubApi to use HubClient from the swift-huggingface package.

pcuenca

Looks good so far! Haven't had a chance to test it yet.

Do you think we should write a transition guide of some sort? Happy to help with that.

Package.swift

…behavior

Co-authored-by: Pedro Cuenca <pedro@huggingface.co>

Copilot

Pull request overview

This PR migrates the Hub module from a custom Downloader implementation to using HubClient from the swift-huggingface package. This simplifies the codebase by delegating download functionality to a dedicated, well-maintained library while maintaining backward compatibility with the existing HubApi interface.

Changes:

Added swift-huggingface dependency (v0.7.0+) and integrated HubClient for file downloads
Removed custom Downloader class and DownloaderTests (533 and 190 lines respectively)
Updated test model references to use canonical namespaced repository IDs (e.g., "google-t5/t5-base" instead of "t5-base")

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
Package.swift	Adds swift-huggingface dependency and links HuggingFace product to Hub target
Sources/Hub/Hub.swift	Adds type conversion extensions for mapping between Hub and HuggingFace types
Sources/Hub/HubApi.swift	Replaces custom download logic with HubClient, exposes client property, updates imports
Sources/Hub/Downloader.swift	Removed - custom download implementation replaced by HubClient
Tests/HubTests/DownloaderTests.swift	Removed - test coverage now provided by HubApiTests with HubClient
Tests/HubTests/HubApiTests.swift	Updates resume test to work with HubClient's cache structure, reduces validation scope
Tests/HubTests/HubTests.swift	Updates model references to canonical namespaced IDs
Tests/TokenizersTests/TokenizerTests.swift	Updates model references to canonical namespaced IDs

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Sources/Hub/HubApi.swift

Tests/HubTests/HubApiTests.swift

Sources/Hub/HubApi.swift

Copilot

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Sources/Hub/Hub.swift

Sources/Hub/HubApi.swift

… build failures on Linux

Copilot

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 5 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Sources/Hub/HubApi.swift

Copilot · 2026-02-19T13:41:08Z

Sources/Hub/HubApi.swift

+        // Keep HubClient progress isolated from parent-linked fileProgress to avoid
+        // totalUnitCount mutations affecting snapshot's aggregate progress tree.
+        let downloadProgress = Progress(totalUnitCount: 1)
+        let downloadStartTime = Date()

-                // etag is a sha256
-                // => means it's an LFS file (large)
-                // => let's compute local hash and compare
-                // => if match, update metadata and return file
-                if hub.isValidHash(hash: remoteEtag, pattern: hub.sha256Pattern) {
-                    let fileHash = try hub.computeFileHash(file: destination)
-                    if fileHash == remoteEtag {
-                        try hub.writeDownloadMetadata(commitHash: remoteCommitHash, etag: remoteEtag, metadataPath: metadataDestination)
-                        return destination
-                    }
-                }
-            }
+        // If the file exists locally but metadata check failed, force a re-download
+        // to skip HubClient's cache (which may have the old/wrong version)
+        let forceDownload = downloaded



When downloading via HubClient (the non-background path), this function never calls progressHandler until the download finishes, and it never maps downloadProgress.fractionCompleted into fileProgress. This is a behavior regression versus the previous per-byte progress updates and will break callers/tests that expect intermediate progress (e.g. cancellation after the first update). Consider observing downloadProgress and forwarding updates into fileProgress + progressHandler, or using a Progress child relationship that doesn’t mutate the parent’s totalUnitCount.

Sources/Hub/HubApi.swift

Tests/HubTests/HubApiTests.swift

…ad test

Copilot

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 4 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Sources/Hub/HubApi.swift

Tests/HubTests/HubApiTests.swift

Tests/ModelsTests/WeightsTests.swift

Copilot

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 8 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-02-19T17:33:02Z

Sources/Models/LanguageModel.swift

+// Linux Foundation may not provide String(localized:comment:), so keep call sites portable.
+private extension String {
+    init(localized key: String, comment: String? = nil) {
+        self = key
+    }


The Linux String(localized:comment:) shim is duplicated across multiple files. Consider centralizing it into a single #if !canImport(Darwin) helper file (module-wide/internal) and removing per-file copies so updates don't have to be replicated everywhere.

Sources/Models/Weights.swift

Sources/Tokenizers/Tokenizer.swift

Copilot · 2026-02-19T17:33:03Z

Sources/Hub/HubApi.swift

+        // Linux: let URLSession handle redirects with default behavior.
+        (_, response) = try await URLSession.shared.data(for: request)
+        #else
+        // Apple platforms: use shared session with custom relative-redirect handling.


In the FoundationNetworking (Linux) branch, httpHead uses URLSession.shared default redirect handling. This bypasses the custom redirect policy described above (relative-only redirects) and may allow absolute redirects while still attaching the Authorization header. Consider using the same RedirectDelegate-backed session (or otherwise disabling/validating redirects) on Linux as well, or update the doc comment to accurately describe the platform-specific behavior.

Suggested change

// Linux: let URLSession handle redirects with default behavior.

(_, response) = try await URLSession.shared.data(for: request)

#else

// Apple platforms: use shared session with custom relative-redirect handling.

// Linux: use URLSession.shared with default redirect handling. This may follow

// absolute redirects while still sending the Authorization header.

(_, response) = try await URLSession.shared.data(for: request)

#else

// Apple platforms: use a RedirectDelegate-backed session that enforces the

// custom redirect policy (for example, allowing only relative redirects).

Copilot · 2026-02-19T17:33:04Z

Tests/HubTests/HubApiTests.swift

        let hubApi = HubApi(downloadBase: downloadDestination)
        var lastProgress: Progress? = nil
-        var downloadedTo = FileManager.default.homeDirectoryForCurrentUser
-            .appendingPathComponent("Library/Caches/huggingface-tests/models/coreml-projects/Llama-2-7b-chat-coreml")
-
-        let metadataDestination = downloadedTo.appending(component: ".cache/huggingface/download")

+        // Get the etag from the file metadata
        let url = URL(string: "https://huggingface.co/coreml-projects/Llama-2-7b-chat-coreml/resolve/main/config.json")!
        let etag = try await Hub.getFileMetadata(fileURL: url).etag!

-        try FileManager.default.createDirectory(at: metadataDestination, withIntermediateDirectories: true, attributes: nil)
-        try "X".write(to: metadataDestination.appendingPathComponent("config.json.\(etag).incomplete"), atomically: true, encoding: .utf8)
-        downloadedTo = try await hubApi.snapshot(from: repo, matching: "config.json") { progress in
+        // This simulates a previous interrupted download.
+        // Use a temporary HF_HOME so we don't write into the real user home directory.
+        let tempHFHome = FileManager.default.temporaryDirectory.appendingPathComponent(UUID().uuidString)
+        try FileManager.default.createDirectory(at: tempHFHome, withIntermediateDirectories: true, attributes: nil)
+        let previousHFHome = ProcessInfo.processInfo.environment["HF_HOME"]
+        setenv("HF_HOME", tempHFHome.path, 1)


This test mutates the process-wide HF_HOME environment variable to control HubClient caching. That can leak across concurrently running tests and can also be order-dependent (the HubApi / HubClient is instantiated before HF_HOME is set). Consider setting HF_HOME before constructing HubApi, and/or adding a test-only way to inject a cache root into HubApi/HubClient so tests don't rely on global environment mutation.

Copilot · 2026-02-19T17:33:04Z

Sources/Hub/HubApi.swift

+        let state = BackgroundDownloadState()
+        let delegate = BackgroundDownloadDelegate(
+            state: state,
+            expectedSize: expectedSize,
+            fileProgress: fileProgress,
+            parentProgress: parentProgress,
+            progressHandler: progressHandler


progressHandler is called with the file marked as complete before writeDownloadMetadata(...) runs. If writing metadata fails, callers can observe 100% progress followed by an error, and offline mode metadata may be missing even though the UI saw completion. Consider writing metadata first (or only reporting completion after metadata write succeeds) so progress reflects the final committed state.

Suggested change

let state = BackgroundDownloadState()

let delegate = BackgroundDownloadDelegate(

state: state,

expectedSize: expectedSize,

fileProgress: fileProgress,

parentProgress: parentProgress,

progressHandler: progressHandler

// Write metadata for offline mode support before reporting completion

try writeDownloadMetadata(commitHash: remoteCommitHash, etag: remoteEtag, metadataPath: metadataDestination)

// When a file is re-downloaded, ensure the local file timestamp reflects this write.

try? FileManager.default.setAttributes([.modificationDate: Date()], ofItemAtPath: destination.path)

fileProgress.completedUnitCount = fileProgress.totalUnitCount

progressHandler(parentProgress)

Sources/Hub/Hub.swift

mattt · 2026-02-22T11:26:28Z

Attempting to rebase and reconcile broken CI for Linux created too much noise for this to be legible. Closing in favor of #315.

mattt mentioned this pull request Dec 5, 2025

Relax platform requirements to support macOS 13 huggingface/swift-huggingface#7

Merged

This comment was marked as resolved.

Sign in to view

pcuenca reviewed Dec 6, 2025

View reviewed changes

Package.swift Outdated Show resolved Hide resolved

This was referenced Feb 6, 2026

Optimizations for significantly faster downloads and cache hits #302

Closed

Support compiling swift-transformers on Linux #308

Merged

mattt mentioned this pull request Feb 12, 2026

Corrupted cached tokenizer.json can be reused without integrity verification #313

Closed

mattt and others added 9 commits February 19, 2026 02:50

Add dependency on swift-huggingface package

da5e186

Replace Downloader with HubClient

09d98eb

Fix issues with child Progress objects

e9439b5

Force download for corrupted files

a9149de

Store throughput in custom Progress userInfo key

e321974

Update tests to use fully-qualified model names

a455baf

Add explicit hf namespace on legacy unqualified model names

c2eb516

Update test expectations to allow for slight difference in HubClient …

f535359

…behavior

Update Package.swift

5ac8511

Co-authored-by: Pedro Cuenca <pedro@huggingface.co>

mattt force-pushed the mattt/hubclient branch from 08cc81c to 5ac8511 Compare February 19, 2026 11:06

Update usage of HubClient.downloadFile

6e6aaa8

mattt marked this pull request as ready for review February 19, 2026 11:08

mattt requested a review from Copilot February 19, 2026 11:11

Copilot started reviewing on behalf of mattt February 19, 2026 11:11 View session

Copilot AI reviewed Feb 19, 2026

View reviewed changes

mattt added 2 commits February 19, 2026 03:25

Add missing conditionalized Network import

ea7be95

Incorporate feedback from review

5cc8249

mattt requested a review from Copilot February 19, 2026 11:50

Copilot started reviewing on behalf of mattt February 19, 2026 11:51 View session

Copilot AI reviewed Feb 19, 2026

View reviewed changes

Sources/Hub/Hub.swift Show resolved Hide resolved

Sources/Hub/HubApi.swift Outdated Show resolved Hide resolved

Sources/Hub/HubApi.swift Outdated Show resolved Hide resolved

Implement background downloads

30fe942

mattt force-pushed the mattt/hubclient branch from dc2a7f0 to 30fe942 Compare February 19, 2026 12:41

mattt added 2 commits February 19, 2026 04:51

Add missing FoundationNetwork import

f7f9be7

Fix usage of ProgressUserInfoKey.throughputKey

7e92aab

Add optional comment parameter to localized string initializer to fix…

fe706ea

… build failures on Linux

mattt requested a review from Copilot February 19, 2026 13:33

Copilot started reviewing on behalf of mattt February 19, 2026 13:33 View session

Copilot AI reviewed Feb 19, 2026

View reviewed changes

mattt added 8 commits February 19, 2026 05:52

Add conditionalized localized string initializers to any file using them

3f5263b

Conditionalize WeightsTests on CoreML

2fd6618

Clean up partially downloaded files in tests

9444b25

Strengthen expectation around progress fulfillment in snapshot downlo…

c16d674

…ad test

Move ProgressUserInfoKey extension out of conditional compilation

3b6cd7d

Use consistent background download identifier

adaec1b

Update test on Linux to not throw unavailable error

b8f63c7

Fix Linux crash caused by setting Location header into request

8000bd9

mattt requested a review from Copilot February 19, 2026 17:03

Copilot started reviewing on behalf of mattt February 19, 2026 17:03 View session

More attempts to fix Linux crash

f15520b

Copilot AI reviewed Feb 19, 2026

View reviewed changes

Sources/Hub/HubApi.swift Show resolved Hide resolved

Sources/Hub/HubApi.swift Show resolved Hide resolved

Tests/HubTests/HubApiTests.swift Outdated Show resolved Hide resolved

Tests/ModelsTests/WeightsTests.swift Show resolved Hide resolved

mattt added 2 commits February 19, 2026 09:17

Incorporate feedback from review

9700658

Opt out Linux from redirect handling to avoid crashing

1e77a2c

mattt requested a review from Copilot February 19, 2026 17:21

Copilot started reviewing on behalf of mattt February 19, 2026 17:21 View session

Opt out Linux from progress reporting

1526928

Copilot AI reviewed Feb 19, 2026

View reviewed changes

Skip testRealDownloadInterruptionAndResumption on Linux

c280582

This was referenced Feb 19, 2026

Replace HubApi downloader with swift-huggingface HubClient #315

Merged

Fix CI for Linux #316

Merged

mattt closed this Feb 22, 2026

Conversation

mattt commented Dec 5, 2025

Uh oh!

This comment was marked as resolved.

pcuenca left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mattt commented Feb 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants