Documentation/Multimodal (Vision)
v2.0.0-beta.1Snapshot bebdcd1Markdown ↗

Multimodal (Vision)

Multimodal (Vision)

Analyzing Images

typescript
import { Agent, createMessageWithImages } from 'oneringai';

const agent = Agent.create({
  connector: 'openai',
  model: 'gpt-5.6-terra',
});

// From file path
const response1 = await agent.run([
  createMessageWithImages('What is in this image?', ['./photo.jpg'])
]);

// From URL
const response2 = await agent.run([
  createMessageWithImages('Describe this image', [
    'https://example.com/image.jpg'
  ])
]);

// From base64
const base64Image = Buffer.from(imageData).toString('base64');
const response3 = await agent.run([
  createMessageWithImages('Analyze this', [
    `data:image/jpeg;base64,${base64Image}`
  ])
]);

// Multiple images
const response4 = await agent.run([
  createMessageWithImages(
    'Compare these two images',
    ['./image1.jpg', './image2.jpg']
  )
]);

Clipboard Images

Paste images directly from clipboard (like Claude Code!):

typescript
import { Agent, readClipboardImage, hasClipboardImage } from 'oneringai';

const agent = Agent.create({
  connector: 'anthropic',
  model: 'claude-opus-5',
});

// Check if clipboard has an image
if (await hasClipboardImage()) {
  // Read clipboard image
  const result = await readClipboardImage();

  if (result.success && result.dataUri) {
    const response = await agent.run([
      createMessageWithImages('What is in this screenshot?', [
        result.dataUri
      ])
    ]);

    console.log(response.output_text);
  }
}

Vision with Tools

typescript
const agent = Agent.create({
  connector: 'openai',
  model: 'gpt-5.6-terra',
  tools: [extractTextTool, identifyObjectsTool],
});

const response = await agent.run([
  createMessageWithImages(
    'Extract all text from this receipt and calculate the total',
    ['./receipt.jpg']
  )
]);

// Agent will:
// 1. Analyze image
// 2. Call extractTextTool to extract text
// 3. Parse numbers
// 4. Calculate total

Documentation snapshot · 2026-11-02Suggest a correction ↗

Search documentation

Type to search the v2 documentation.