Skip to main content

graniteLLMClient

Workspace API

Workspace API / services/granite/graniteLLMClient

services/granite/graniteLLMClient

Classes​

LLMTimeoutError​

Defined in: services/granite/graniteLLMClient.ts:107

Custom error for LLM request timeouts

Extends​

  • Error

Constructors​

Constructor​

new LLMTimeoutError(message, timeoutMs): LLMTimeoutError

Defined in: services/granite/graniteLLMClient.ts:108

Parameters​
message​

string

timeoutMs​

number

Returns​

LLMTimeoutError

Overrides​

Error.constructor

Properties​

timeoutMs​

readonly timeoutMs: number

Defined in: services/granite/graniteLLMClient.ts:110

LLMInputTooLargeError​

Defined in: services/granite/graniteLLMClient.ts:120

Custom error for LLM input exceeding limits

Extends​

  • Error

Constructors​

Constructor​

new LLMInputTooLargeError(message, inputTokens, maxContextLength): LLMInputTooLargeError

Defined in: services/granite/graniteLLMClient.ts:121

Parameters​
message​

string

inputTokens​

number

maxContextLength​

number

Returns​

LLMInputTooLargeError

Overrides​

Error.constructor

Properties​

inputTokens​

readonly inputTokens: number

Defined in: services/granite/graniteLLMClient.ts:123

maxContextLength​

readonly maxContextLength: number

Defined in: services/granite/graniteLLMClient.ts:124

GraniteLLMClient​

Defined in: services/granite/graniteLLMClient.ts:191

Client for Granite LLM via vLLM OpenAI-compatible API

Constructors​

Constructor​

new GraniteLLMClient(): GraniteLLMClient

Defined in: services/granite/graniteLLMClient.ts:200

Returns​

GraniteLLMClient

Methods​

generateText()​

generateText(prompt, options): Promise<string>

Defined in: services/granite/graniteLLMClient.ts:382

Generate text using Granite LLM (non-streaming)

Parameters​
prompt​

string

options​

GraniteLLMGenerateOptions = {}

Returns​

Promise<string>

generateTextStream()​

generateTextStream(prompt, options): AsyncGenerator<string, void, unknown>

Defined in: services/granite/graniteLLMClient.ts:542

Generate text with streaming Note: Retries only happen before stream starts, not mid-stream

Parameters​
prompt​

string

options​

GraniteLLMGenerateOptions = {}

Returns​

AsyncGenerator<string, void, unknown>

Interfaces​

GraniteLLMGenerateOptions​

Defined in: services/granite/graniteLLMClient.ts:155

Properties​

maxTokens?​

optional maxTokens: number

Defined in: services/granite/graniteLLMClient.ts:156

temperature?​

optional temperature: number

Defined in: services/granite/graniteLLMClient.ts:157

topP?​

optional topP: number

Defined in: services/granite/graniteLLMClient.ts:158

stream?​

optional stream: boolean

Defined in: services/granite/graniteLLMClient.ts:159

systemInstruction?​

optional systemInstruction: string

Defined in: services/granite/graniteLLMClient.ts:160

timeoutMs?​

optional timeoutMs: number

Defined in: services/granite/graniteLLMClient.ts:161

preCalculatedInputTokens?​

optional preCalculatedInputTokens: number

Defined in: services/granite/graniteLLMClient.ts:167

Pre-calculated input token count (optional) If provided, the client will use this value instead of re-estimating. Useful when the caller has already calculated tokens (e.g., RAG service).

Variables​

GRANITE_LLM_MODEL_ID​

const GRANITE_LLM_MODEL_ID: string = GRANITE_CONFIG.model

Defined in: services/granite/graniteLLMClient.ts:135

GRANITE_LLM_ENDPOINT​

const GRANITE_LLM_ENDPOINT: string = GRANITE_CONFIG.endpoint

Defined in: services/granite/graniteLLMClient.ts:136

Functions​

getGraniteLLMClient()​

getGraniteLLMClient(): GraniteLLMClient

Defined in: services/granite/graniteLLMClient.ts:799

Returns​

GraniteLLMClient