Use DatalandMCP with d-fine Azure LLM

[dleemueller]

To have a d-fine internal chat functionality that use the DatalandMCP server, we now have to try to connect the server to a client that uses the LLM provided to d-fine via Azure.

Azure resources: Resources
Models: Modelle und Endpunkte - Azure OpenAI

[dleemueller]

Azure Resource

In the dataland resource group we now have an OpenAI resource specifically for DatalandMCP.

GPT-4.1 was deployed as a model that now has to be connected to a client.

[dleemueller]

Open WebUI

Open WebUI allows user to run local LLMs but connects to cloud based OpenAIs.

Installation

To install openwebui, create a virtual environment, activate it and run the command pip install open-webui.
Note, that it is only compatible with Python 3.11 & 3.12.

Launch

To launch the app run the command open-webui serve within the venv.
After successful launch we can now open the UI via http://localhost:8080

Optional (Recommended):
Open WebUI support Xet storages, which avoids downloading files and models at every launch of the application.
To use this functionality download the hf_xet package via pip install hf_xet

Azure OpenAI

Open WebUI supports Azure OpenAI to connect the cloud based LLM.
For that one has to go to Profile -> Admin Panel -> Settings -> Connections -> +:

In the above screenshot [resource_name], [deployment_name] and [api_key] has to be replaced by the correct names defined in Azure.
If everything is correct you should now be able to see the deployed model in the list of available models:

Connecting MCP Servers with mcpo

MCPO exposes any MCP Tool as an OpenAPI-compatible HTTP server. This is needed in our case,
Note: Resources (Templates) and Prompts are yet not implemented in mcpo but there is are open Pull Requests regarding Resources: MCPO - Pull Requests

Installation and Launch

MCPO requires Python3.8+ and can be installed via pip install mcpo
After successful installation you can set up a config file in the same format as for Claude Desktop or Cursor:

{
    "mcpServers": {
        "DatalandMCP": {
            "type": "stdio",
            "command": "path_to_dataland_venv\\Scirpts\\python.exe",
            "args": ["path_to_server\\server.py", "stdio"],
            "env": {
              "DATALAND_MCP_ROOT_DIR": "",
              "DATALAND_QARG_ROOT_DIR": "",
              "DATALAND_API_KEY": ""
          }
        }
    },
    "port": 8000
}

Within Powershell you can launch the server via mcpo --config mcp.json --port 8000 --host localhost.

Now the server should be running. Its documentation (Swagger UI) should now be accessible via http://localhost:8000/DatalandMCP/docs.

Connecting to Open WebUI

Within Open WebUI you can now add the running server via Profile -> Settings -> Tools -> +:

Now the tools of the MCP Server are available in the chat via a toolbox under the input field:

Note: MCPO is strict regarding the data models of the return values and expects a somehow perfect schema. Returning non pydantic models can result in errors. Removing type hints gets the tools working.

Online hosting

Open WebUI can be hosted online via the correct IP adresses. We could run the app on a deployed VM on Azure and host it there. However, this will require many question regarding security, user management, etc.

[dleemueller]

Token Limit

The Token Limit with Azure OpenAI can be set to 1M Tokens per Minute which is enough to give around 80 SFDR reports as context.

[dleemueller]

Azure OpenAI resource has been added to MCP Host Config.