​
Overview

A MessageType consists of three main components:

​
role
RoleEnumType

The role of the message sender.

Constraints: Must be one of the defined roles.

​
content
Array<ContentType>

Array of content items.

Constraints: Must contain at least one content item.

​
Role Types

The role field uses a union of predefined roles that determine the sender of the message.

​
RoleEnumType

​
"system"
string

Messages from the system.

Usage: Used for system prompts and instructions.

​
"user"
string

Messages from the user.

Usage: User input and queries.

​
"assistant"
string

Messages from the assistant.

Usage: LLM responses and outputs.

​
"tool"
string

Messages from a tool.

Usage: Tool execution results.

// Examples of different roles
const systemMessage: MessageType = {
  role: "system",
  content: [{ modality: "text", value: "You are a helpful assistant."}]
};

const userMessage: MessageType = {
  role: "user",
  content: [{ modality: "text", value: "Hello, how are you?"}]
};

const assistantMessage: MessageType = {
  role: "assistant",
  content: [{ modality: "text", value: "I'm doing well, thank you!"}]
};

​
Content Types

The content field contains an array of ContentType items, which is a union based on the modality field. Each content type supports different data formats and use cases.

​
ContentType Discriminated Union

The ContentType is discriminated by the modality field:

​
"text"
TextContentType

Plain text content.

​
"image"
ImageContentType

Image content with various formats.

​
"tool-call"
ToolCallContentType

Function/tool call requests.

​
"tool-response"
ToolResponseContentType

Tool execution responses.

​
"reasoning"
ReasoningContentType

LLM reasoning and thinking content.

​
Text Content

​
TextContentType

Represents plain text content in messages.

​
modality
"text"

Content type discriminator.

Constraints: Must be "text".

​
value
string

The text content.

Constraints: Any valid string.

const textContent: TextContentType = {
  modality: "text",
  value: "Hello, how can I help you today?"
};

​
Image Content

​
ImageContentType

Represents image content with support for both base64-encoded images and URL references.

​
modality
"image"

Content type discriminator.

Constraints: Must be "image".

​
detail
ImageContentDetailsLiteralType

Processing detail level.

Constraints: Must be "low", "medium", "high", or "auto"

​
value
ImageContentValueType

Image data.

Constraints: Must be either Base64 or URL type

​
ImageContentDetailsLiteralType

Controls how the image should be processed by the LLM:

​
"low"
string

Low detail processing (faster, less accurate).

​
"medium"
string

Medium detail processing.

​
"high"
string

High detail processing (slower, more accurate).

​
"auto"
string

Automatic detail selection.

​
ImageContentValueType Discriminated Union

The value field in ImageContentType is itself a discriminated union based on the type field:

​
Base64ImageContentValueType

For inline base64-encoded images:

​
type
"base64"

Value type discriminator.

Constraints: Must be "base64"

​
base64
string

Base64-encoded image data.

Constraints: Valid base64 string.

​
mediaType
MediaTypeLiteral

Image format.

Constraints: Must be "png", "jpeg", "webp", or "gif"

​
UrlImageContentValueType

For images referenced by URL:

​
type
"url"

Value type discriminator.

Constraints: Must be "url"

​
url
string

URL pointing to the image.

Constraints: Valid URL string

// Base64 image example
const base64ImageContent: ImageContentType = {
  modality: "image",
  detail: "high",
  value: {
    type: "base64",
    base64: "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8z8BQDwAEhQGAhKmMIQAAAABJRU5ErkJggg==",
    mediaType: "png"
  }
};

// URL image example
const urlImageContent: ImageContentType = {
  modality: "image",
  detail: "medium",
  value: {
    type: "url",
    url: "https://example.com/image.jpg"
  }
};

​
Tool Call Content

​
ToolCallContentType

Represents a function or tool call request from the LLM.

​
modality
"tool-call"

Content type discriminator.

Constraints: Must be "tool-call".

​
index
number

Tool call index.

Constraints: Non-negative integer.

​
id
string

Unique identifier for the tool call.

Constraints: Non-empty string.

​
name
string

Name of the tool being called.

Constraints: Non-empty string.

​
arguments
string

JSON string of tool arguments.

Constraints: Valid string (typically JSON).

const toolCallContent: ToolCallContentType = {
  modality: "tool-call",
  index: 0,
  id: "call_123456789",
  name: "search_database",
  arguments: '{"query": "user information", "limit": 10}'
};

​
Tool Response Content

​
ToolResponseContentType

Represents the response from a tool execution.

​
modality
"tool-response"

Content type discriminator.

Constraints: Must be "tool-response".

​
index
number

Tool response index.

Constraints: Non-negative integer.

​
id
string

Tool call identifier.

Constraints: Non-empty string (matches tool call ID).

​
name
string

Name of the tool that was called.

Constraints: Non-empty string.

​
data
string

Response data from the tool.

Constraints: Any valid string.

const toolResponseContent: ToolResponseContentType = {
  modality: "tool-response",
  index: 0,
  id: "call_123456789",
  name: "search_database",
  data: '{"results": [{"id": 1, "name": "John Doe"}, {"id": 2, "name": "Jane Smith"}]}'
};

​
Reasoning Content

​
ReasoningContentType

Represents LLM reasoning and thinking processes, supporting both full reasoning and redacted content.

​
modality
"reasoning"

Content type discriminator.

Constraints: Must be "reasoning".

​
value
ReasoningContentValueUnionType

Reasoning data.

Constraints: Must be thinking or redacted type.

​
ReasoningContentValueUnionType Discriminated Union

The value field is discriminated by the type field:

​
ReasoningContentValueType

For full reasoning content:

​
type
"thinking"

Value type discriminator.

Constraints: Must be "thinking".

​
thinking
string

The reasoning/thinking text.

Constraints: Any valid string.

​
signature
string

Signature for the reasoning.

Constraints: Any valid string.

​
RedactedReasoningContentValueType

For redacted reasoning content:

​
type
"redacted"

Value type discriminator.

Constraints: Must be "redacted".

​
data
string

Redacted data placeholder.

Constraints: Any valid string

// Full reasoning example
const reasoningContent: ReasoningContentType = {
  modality: "reasoning",
  value: {
    type: "thinking",
    thinking: "I need to analyze the user's request carefully to provide the most helpful response...",
    signature: "reasoning-signature-123",
  }
};

// Redacted reasoning example
const redactedReasoningContent: ReasoningContentType = {
  modality: "reasoning",
  value: {
    type: "redacted",
    data: "This reasoning has been redacted for privacy",
  }
};

​
Complete Example

const complexMessage: MessageType = {
  role: "assistant",
  content: [
    {
      modality: "text",
      value: "I've analyzed the image and found the following information:"
    },
    {
      modality: "reasoning",
      value: {
        type: "thinking",
        thinking: "The image appears to contain a chart. I should extract the data points.",
        signature: "reasoning-signature-456",
      },
    },
    {
      modality: "tool-call",
      index: 0,
      id: "call_987654321",
      name: "analyze_chart",
      arguments: '{"chart_type": "bar", "data_points": ["Q1", "Q2", "Q3", "Q4"]}'
    }
  ]
};