[skyrl-train][refactor] Inference Server Refactor -- RemoteInferenceClient 2/N

[kouroshHakha]

Summary: Adds RemoteInferenceClient, a lightweight, fully serializable HTTP client that wraps inference server APIs. This client replaces the old InferenceEngineInterface for HTTP-based inference and can work with any HTTP-compatible inference backend (vLLM, sglang-router, Ray Serve LLM, etc.).

Architecture:

• Router (InferenceRouter): Data plane only - routes requests to ONE backend (session-aware or round-robin)
• Client (RemoteInferenceClient): Fully responsible for control plane fan-out to all backends

This separation allows using external routers (vllm-router, sglang-router) that only handle data plane.

Key Features:

• Serializable: Can be pickled and passed between Ray actors/processes
• Two URL types: proxy_url for data plane (router), server_urls for control plane (fan-out to backends)
• Data plane: generate(), chat_completion(), completion(), tokenize(), detokenize()
• Control plane (fan-out): pause(), resume(), sleep(), wake_up(), reset_prefix_cache()
• Weight sync (fan-out): init_weight_transfer(), update_weights(), finalize_weight_update()
• PauseMode enum: Forward-compatible with vLLM RFC #32103 pause modes
• Built-in retry on abort: Handles stop_reason="abort" during weight sync

Comparison vs InferenceEngineInterface + InferenceEngineClient:

• Serializable - Just URLs, no Ray actors/tokenizers/thread events
• No local tokenizer - Uses /tokenize endpoint instead
• Server-side routing - Router handles session routing via X-Session-ID header
• Simplified parallelism - Single get_world_size() vs separate tp_size(), pp_size(), dp_size()
• No ABC hierarchy - Simple dataclass with async methods
• Backend-agnostic - Works with any HTTP server (vLLM, sglang, Ray Serve LLM)

Files Added:

• skyrl_train/inference_servers/remote_inference_client.py - The client implementation
• tests/cpu/inference_servers/test_remote_inference_client.py - Unit tests

[gemini-code-assist]

Code Review

This pull request refactors the inference engine by replacing the InferenceEngineInterface with a new RemoteInferenceClient for HTTP-based inference, introducing new modules for common utilities, protocols, server groups, and a robust router. While the changes are well-structured and include comprehensive unit and GPU CI tests, it introduces significant security risks. The most critical issue is the use of pickle.loads in the vLLM worker extension, which provides a direct path to Remote Code Execution (RCE). Additionally, the lack of authentication on sensitive control plane and weight synchronization endpoints in both the router and the server actor exposes the cluster to unauthorized control and potential weight hijacking. These security concerns must be addressed before deployment in untrusted network environments.

[vercel]

@kouroshHakha is attempting to deploy a commit to the Tyler's projects Team on Vercel.

A member of the Team first needs to authorize it.

[CharlieFRuan]

Left several comments, thank you!

[CharlieFRuan]

Thank you!