Multi-Provider LLM Client

The gaik.software_components.llm package lets text components use OpenAI, Azure OpenAI, Anthropic Claude, Google Gemini, CSC Aitta, or another OpenAI-compatible server. Choose a configuration and pass it to DataExtractor, SchemaGenerator, DocumentClassifier, TranscriptEnhancer, AnswerGenerator, Embedder, PostgresAgent, or TabularAgent. Each component keeps the same public interface; the chosen model must support the capabilities it uses.

create_llm_client() exposes a common ProviderClient interface for direct calls. OpenAI, Azure, Aitta, and compatible servers share the OpenAI SDK adapter; Anthropic, Google and Vertex use native SDKs. LiteLLM is an optional backend for additional provider integrations. Existing get_openai_config() configurations still use the legacy OpenAI/Azure component paths.

Installation

OpenAI, Azure, Aitta, and other OpenAI-compatible servers use the core openai dependency. Install the extras needed by your component and, for native Google or Anthropic, its SDK:

pip install 'gaik[extract]'
pip install 'gaik[llm-google]'      # google-genai + google-auth
pip install 'gaik[llm-anthropic]'   # anthropic SDK
pip install 'gaik[llm-all]'         # native SDKs and LiteLLM
pip install 'gaik[llm-litellm]'     # optional LiteLLM backend

Same component, different provider

For example, the following extraction uses the same schema and method for Azure, Google, or Aitta. Configure credentials for the selected provider, then change only provider:

from pydantic import BaseModel, ConfigDict

from gaik.software_components.extractor import DataExtractor
from gaik.software_components.extractor.schema import ExtractionRequirements, FieldSpec
from gaik.software_components.llm import get_llm_config


class Project(BaseModel):
    model_config = ConfigDict(extra="forbid")
    title: str


requirements = ExtractionRequirements(
    use_case_name="project",
    fields=[FieldSpec(field_name="title", field_type="str", description="Project title")],
)

provider = "aitta"  # "azure", "google", "openai", or "anthropic"
extractor = DataExtractor(config=get_llm_config(provider))
results = extractor.extract(
    extraction_model=Project,
    requirements=requirements,
    user_requirements="Extract the project title.",
    documents=["Project: Advanced AI Research Initiative."],
)

Extraction and classification require structured output support from the selected model and endpoint. OpenAI-compatible chat alone does not guarantee JSON-schema support. A complete multi-provider example is at example_multi_provider_extractor.py.

Configuration

get_llm_config(provider, **overrides) reads the provider's environment variables and returns a dictionary. Keyword overrides such as api_key, model, base_url, or embedding_model are applied before required fields are validated, so configuration can come from your own secret manager instead of environment variables.

When resolving an existing config, the priority is:

1. Explicit provider argument to resolve_provider()
2. config["provider"]
3. Legacy config["use_azure"] (True → Azure, False → OpenAI)
4. LLM_PROVIDER environment variable
5. Azure by default

get_llm_config() without an argument uses LLM_PROVIDER, then Azure. An existing legacy config with use_azure keeps its provider even if LLM_PROVIDER is set elsewhere.

A legacy dict with neither provider nor use_azure, such as {"api_key": key, "model": "gpt-5.4"}, keeps its earlier meaning in create_openai_client() and in components: it selects standard OpenAI whatever LLM_PROVIDER says (VisionParser keeps its own legacy Azure default). create_llm_client(), MultimodalParser(api_config=...) and LLMJudge(config=...) apply the order above instead, so the same dict selects LLM_PROVIDER or Azure. Add provider or use_azure to make the choice explicit.

The Google SDK adapter supports the direct Gemini API and Vertex AI. Vertex accepts application-default credentials (including a GOOGLE_APPLICATION_CREDENTIALS file), a credentials object, or service-account data via service_account_json (GOOGLE_SERVICE_ACCOUNT_JSON, inline JSON or a file path). Choose a region where your model is served.

LiteLLM is opt-in: get_llm_config("litellm", model="azure/gpt-6-luna", api_key=key, base_url=endpoint, api_version=version). It uses the same component interface and passes credentials per request. LITELLM_API_KEY, LITELLM_BASE_URL, LITELLM_API_VERSION, and LITELLM_EMBEDDING_MODEL are optional environment defaults. Select a provider-prefixed embedding model separately when needed.

CSC Aitta

Keep the token in AITTA_API_KEY in your environment or an untracked .env file. The Aitta config sets base_url to https://aitta-api.csc.fi/openai/v1 and defaults to a 600-second timeout to accommodate model startup:

from gaik.software_components.llm import create_llm_client, get_llm_config

config = get_llm_config("aitta", timeout=600.0, max_retries=1)
client = create_llm_client(config)
response = client.chat([
    {"role": "user", "content": "Explain knowledge management in one sentence."}
])
print(response.text)

Aitta provides chat-completion and embedding endpoints. Select a model using the authenticated model catalogue, and inspect its capabilities. Embeddings require an embedding model: set AITTA_EMBEDDING_MODEL or pass embedding_model explicitly. The chat model is not an embedding fallback. Embedding requests timed out in the integration checks, so live embedding availability is not confirmed; select another embedding provider if needed. See the official Aitta documentation.

Aitta is a shared research inference service. A first request may wait for a model to load or for computing capacity; a client timeout alone does not establish an authentication failure. Check the service status and active workers, and prefer an already running model when capacity is limited. Availability can change during LUMI maintenance. See Aitta's capacity and availability guidance.

In the September 2026 integration checks, google/gemma-4-31b-it completed chat, streaming, structured output, schema generation, extraction, image parsing and PDF vision extraction. LumiOpen/Llama-Poro-2-70B-Instruct handled chat and a small schema but failed the more complex schema-generation check with invalid JSON. Use the tested Gemma model for extraction, and validate your own schema before relying on another model. These checks establish compatibility for those requests, not a general quality ranking.

From a repository checkout, run the live connectivity example with credentials already configured:

uv run python implementation_layer/examples/software_components/llm/example_aitta_smoke.py

Other OpenAI-compatible servers

Use openai_compatible to keep the server's endpoint and model explicit:

config = get_llm_config(
    "openai_compatible",
    api_key=server_api_key,
    base_url="https://your-inference-server.example/v1",
    model="your-served-model",
    timeout=120.0,
    max_retries=1,
)
client = create_llm_client(config)

For embeddings, also set embedding_model or EMBEDDING_MODEL to a model served by that endpoint. Protocol compatibility does not add features the server or model lacks.

ProviderClient interface

Component support

Every “Yes” also requires a model or Azure deployment that supports the operation. With the optional LiteLLM backend, support follows the model it routes to. For Google embeddings, the default is gemini-embedding-001; match your vector store dimensions to the returned vectors. Use the same embedding model for documents and queries.

DocumentClassifier uses local parsing for PDF/DOCX text by default; classifying images or explicitly selecting its vision parser also requires vision support. VisionParser accepts all shared provider configs, and the endpoint must serve a vision model before sending images. MultimodalParser(api_config=config), VisionExtractor(api_config=config), FormUnderstander(config=config), and LLMJudge(config=config) also use the shared interface. Their legacy provider arguments remain available.

Shared vision messages contain text plus base64 image data URLs in OpenAI-shaped image_url parts. Native adapters translate these into provider-specific image payloads. This avoids a provider adapter fetching arbitrary remote image URLs; callers should supply image bytes through the vision components or construct base64 data URLs.

Choose providers per pipeline stage

Audio, parsing, embeddings and answering do not have to use the same provider:

from gaik.software_modules.audio_to_structured_data import AudioToStructuredData

audio = get_llm_config("azure")
text = get_llm_config("aitta")
pipeline = AudioToStructuredData(
    api_config=audio,
    transcription_config=audio,
    extraction_config=text,
)

DocumentsToStructuredData accepts parser_config and extraction_config. RAGWorkflow accepts parser_config, embedding_config, and answer_config. Omitted stage configs fall back to api_config. Indexing and queries must keep the same embedding model and vector dimensions.

Report generation accepts runtime configs under writer_options["api_config"] and review_options["api_config"]. save_report_config() rejects credential-bearing configs: save the nonsecret report settings and supply keys at runtime.

Audio

The audio components use the toolkit's OpenAI/Azure audio clients. Aitta and native Anthropic/Google configurations are rejected. Pointing a chat client at Gemini's OpenAI-compatible endpoint does not enable OpenAI transcription or speech endpoints. A workflow that uses Gemini or Aitta for text and OpenAI/Azure for audio needs separate configs and credentials for those services.

Helpers for component authors

To migrate a text component, use build_compat_client(config) in its constructor. Where it calls the model, use isinstance(self.client, ProviderClient) to select chat(), chat_parsed(), chat_stream(), or embed(), while retaining the existing raw SDK path for legacy OpenAI/Azure callers. Test both paths and reject unsupported capabilities explicitly.