Context Management
Context Management
The library includes a powerful, universal context management system that automatically handles the complexity of managing LLM context windows. AgentContextNextGen is the primary context manager with a clean, plugin-based architecture.
Preparation and ownership
Managed run()/stream() prepare context once before each call, then validate the
actual dispatch shape, capacity, and replay owner. Read lastPreparation after
execution for detached budget/selection/compaction diagnostics. Do not call
prepare() again just to inspect a managed request. Standalone context owners may
call it when they are actually constructing their own provider request.
Connected function/native call-result groups, pending dependencies, protected host task state, and unacknowledged evidence cannot be discarded to make a prompt fit. An explicit fit/ownership error is preferable to silently losing them. The preparation recipe describes bounded hook refresh, spill, concurrency, repair, and failure behavior.
AgentContextNextGen - The Modern API
AgentContextNextGen is the modern, plugin-first context manager. It provides clean separation of concerns with composable plugins:
What the plugin architecture is for
A context plugin is a bounded capability, not just a prompt fragment. A plugin can own all of the following as one lifecycle unit:
| Responsibility | Why it belongs in the plugin |
|---|---|
| Static instructions | Teaches the model when and how to use the capability |
| Dynamic context content | Injects only the state needed for the current call |
| Tools | Makes feature-specific operations available through the shared ToolManager |
| Token accounting | Lets the context budget show exactly what each capability costs |
| Compaction | Lets a feature reduce its own content safely when the context fills |
| Persistence | Saves and restores plugin state with the conversation session |
| Cleanup | Releases resources through the common IDisposable lifecycle |
AgentContextNextGen.prepare() assembles one system message from the enabled
plugins and compacts at most once immediately before the LLM call. Conversation
tool-call/result pairs remain atomic, the newest input is never compacted, and
disabling a feature removes its prompt content and feature tools.
The word “memory” appears in several plugin names because they solve different problems:
| Use case | Choose | Data location / retrieval model |
|---|---|---|
| Scratch notes and large intermediate findings | WorkingMemoryPluginNextGen |
Values live outside the prompt; an index is injected and the agent retrieves details with store_* |
| Small state the model must always see | InContextMemoryPluginNextGen |
Full values live directly in the prompt |
| Durable people, organizations, tasks, events, documents, relationships, and learned profiles | MemoryPluginNextGen |
Standalone entity/fact system with ranked, graph, vector, and bitemporal retrieval |
| Let the LLM mutate durable memory | MemoryWritePluginNextGen |
Six controlled write tools layered on the read plugin |
| Learn from conversations without LLM write tools | SessionIngestorPluginNextGen |
Background extraction into the same MemorySystem |
See the Memory Layer Guide for the standalone system and the Self-Learning Memory section for agent wiring.
import { AgentContextNextGen } from 'oneringai';
// Create a context instance
const ctx = AgentContextNextGen.create({
model: 'gpt-4.1',
systemPrompt: 'You are a helpful assistant.',
features: {
workingMemory: true, // WorkingMemoryPluginNextGen
inContextMemory: true, // InContextMemoryPluginNextGen
persistentInstructions: false,
},
strategy: 'algorithmic', // 75% reclaim target; 90% critical trigger
});
// Add user message
ctx.addUserMessage('What is the weather in Paris?');
// Prepare context for LLM call (handles compaction if needed)
const { input, budget, compacted } = await ctx.prepare();
// After LLM call, add response
ctx.addAssistantResponse(response.output);
// Add tool results
ctx.addToolResults([{ tool_use_id: '...', content: '...', state: ToolCallState.COMPLETED }]);
// Access plugins
const memory = ctx.memory; // WorkingMemoryPluginNextGen | null
await memory?.store('key', 'description', value);
// Access tools
ctx.tools.disable('risky_tool');
// Budget information
console.log(`Tokens: ${budget.totalUsed}/${budget.maxTokens}`);
console.log(`Utilization: ${budget.utilizationPercent}%`);
console.log(`Available: ${budget.available}`);
Semantic Rollover at Provider-Session Boundaries
Automatic compaction is token-triggered. Use explicit rollover when the provider session itself must be replaced—for example, before OpenAI Realtime's hard session lifetime—but the same OneRingAI Agent should continue:
const result = await agent.rolloverContext({
preserveRecentTurns: 8,
maxSummaryTokens: 2048,
reason: 'provider-session-expiring',
});
The Agent obtains its normal exclusive execution lease, loads persisted state,
summarizes the older prefix with a tool-free direct call, and commits a
developer continuity brief only after a non-empty summary succeeds. Recent
turns and complete tool-call/result pairs remain exact. Plugin state and the
append-only history journal are not compacted. With session storage,
checkpoint defaults to true; without it, checkpointing defaults to false.
Use a trusted custom summarizer when the Agent model is not text-capable or the summary must run through another service:
await agent.rolloverContext({
summarize: async ({ items, estimatedTokens, preservedRecentTurns, reason }) =>
summaryService.createContinuityBrief({
items,
estimatedTokens,
preservedRecentTurns,
reason,
}),
});
The lower-level context API requires the summarizer explicitly:
const result = await agent.context.rollover({
preserveRecentTurns: 8,
reason: 'manual-handoff',
summarize: ({ items }) => summarizeWithTrustedModel(items),
});
Call either API only between turns, with no pending input. Agent-level rollover
cannot overlap run(), stream(), async continuation, or external execution;
context-level rollover rejects concurrent rollover and aborts if the live
context changes while the summary is being generated. Observe committed
rollovers with context:rolled_over and inspect the returned token/item counts.
Context Structure
AgentContextNextGen organizes context into clear sections:
[Developer Message - All glued together]
# System Prompt
# Persistent Instructions (if plugin enabled)
# Plugin Instructions (for enabled plugins)
# In-Context Memory (if plugin enabled)
# Working Memory Index (if plugin enabled)
[Conversation History]
... messages including tool_use/tool_result pairs ...
[Current Input]
User message OR tool results (newest, never compacted)
AgentContextNextGen Components
AgentContextNextGen uses a plugin architecture with these core components:
| Component | Access | Purpose |
|---|---|---|
| ToolManager | ctx.tools |
Tool registration, execution, circuit breakers |
| WorkingMemoryPluginNextGen | ctx.getPlugin('working_memory') |
Tiered memory (raw/summary/findings) |
| InContextMemoryPluginNextGen | ctx.getPlugin('in_context_memory') |
Live key-value storage in context |
| PersistentInstructionsPluginNextGen | ctx.getPlugin('persistent_instructions') |
Disk-persisted agent instructions |
| UserInfoPluginNextGen | ctx.getPlugin('user_info') |
User-scoped preferences + TODO tracking, auto-injected into context |
| ToolCatalogPluginNextGen | ctx.getPlugin('tool_catalog') |
Scoped discovery and dynamic loading/unloading of tool categories |
| SharedWorkspacePluginNextGen | ctx.getPlugin('shared_workspace') |
Versioned multi-agent coordination board |
| MemoryPluginNextGen | ctx.getPlugin('memory') |
Profile injection plus six graph/vector/document retrieval tools |
| MemoryWritePluginNextGen | ctx.getPlugin('memory_write') |
Six controlled durable-memory mutation tools |
| SessionIngestorPluginNextGen | Manually registered | Optional background extraction from accumulated conversation turns |
| Evidence | Opt-in evidence plugin | Scoped source receipts, projections, and bounded retrieval |
| TaskStatePluginNextGen | ctx.getPlugin('task_state') |
Protected host notes and action receipts |
| Conversation | ctx.getConversation() |
Built-in conversation tracking (InputItem[]) |
Using AgentContextNextGen with Agent
AgentContextNextGen is always available - BaseAgent creates it in the constructor, making it the single source of truth for ToolManager:
import { Agent, AgentContextNextGen } from 'oneringai';
// AgentContextNextGen is auto-created with default config
const agent = Agent.create({
connector: 'openai',
model: 'gpt-4.1',
tools: [weatherTool],
context: {
model: 'gpt-4.1',
strategy: 'algorithmic', // Default strategy; 75% reclaim threshold, 90% critical trigger
features: { workingMemory: true },
},
});
// UNIFIED TOOL MANAGEMENT: agent.tools and agent.context.tools are the SAME instance
console.log(agent.tools === agent.context.tools); // true
console.log(agent.hasContext()); // Always true
// Tool changes via either path are immediately reflected
agent.tools.disable('weather_tool');
console.log(agent.context.tools.listEnabled().includes('weather_tool')); // false
agent.context.tools.enable('weather_tool');
console.log(agent.tools.listEnabled().includes('weather_tool')); // true
// Agent automatically tracks messages and tool calls
await agent.run('What is the weather?');
// Access the context (never null)
const ctx = agent.context;
const conversation = ctx.getConversation(); // InputItem[] - NextGen API
const budget = ctx.lastPreparation?.budget;
if (budget) console.log(`Used: ${budget.totalUsed}/${budget.maxTokens} tokens`);
// Option 2: Pass existing AgentContextNextGen instance
const sharedContext = AgentContextNextGen.create({ model: 'gpt-4.1' });
const agent1 = Agent.create({ connector: 'openai', model: 'gpt-4.1', context: sharedContext });
// Keep this context under one execution owner. Sharing history with another
// provider requires compatible state and explicit authorization; opaque signed
// continuations cannot simply transfer between accounts/providers.
AgentContextNextGen Configuration
interface AgentContextNextGenConfig {
/** Model name (used for token limits) — REQUIRED */
model: string;
/** Max context tokens (overrides model default) */
maxContextTokens?: number;
/** Response token reserve in tokens (default: 4096) */
responseReserve?: number;
/** System prompt */
systemPrompt?: string;
/** Agent ID (used for persistent storage paths) */
agentId?: string;
/** Tools to register */
tools?: ToolFunction[];
/** Feature flags for enabling/disabling plugins */
features?: ContextFeatures;
/** Compaction strategy */
strategy?: string; // 'algorithmic' (default, 75%) or custom registered strategy name
/** Context storage for session persistence */
storage?: IContextStorage;
/** Plugin configurations */
plugins?: PluginConfigs;
}
interface ContextFeatures {
/** Enable WorkingMemoryPluginNextGen (default: true) */
workingMemory?: boolean;
/** Enable InContextMemoryPluginNextGen (default: true) */
inContextMemory?: boolean;
/** Enable PersistentInstructionsPluginNextGen (default: false) */
persistentInstructions?: boolean;
/** Enable UserInfoPluginNextGen (default: false) */
userInfo?: boolean;
/** Enable ToolCatalogPluginNextGen for dynamic tool loading/unloading (default: false) */
toolCatalog?: boolean;
/** Enable SharedWorkspacePluginNextGen (default: false) */
sharedWorkspace?: boolean;
/** Enable MemoryPluginNextGen read tools/profile injection (default: false) */
memory?: boolean;
/** Enable MemoryWritePluginNextGen write tools (default: false; requires memory) */
memoryWrite?: boolean;
}
Feature Configuration
AgentContextNextGen features enable plugins independently. When a feature is disabled, its associated tools are not registered, giving the LLM a cleaner tool set:
import { AgentContextNextGen, DEFAULT_FEATURES } from 'oneringai';
// View default feature settings
console.log(DEFAULT_FEATURES);
// { taskState: false, workingMemory: true, inContextMemory: true, persistentInstructions: false, userInfo: false, toolCatalog: false, sharedWorkspace: false, memory: false, memoryWrite: false }
// evidence is separately opt-in; it is not enabled by these defaults.
Available Features:
| Feature | Default | Plugin | When Disabled |
|---|---|---|---|
workingMemory |
true |
WorkingMemoryPluginNextGen - tiered memory (raw/summary/findings) | store_* tools cannot target "notes" store; ctx.memory returns null |
inContextMemory |
true |
InContextMemoryPluginNextGen - live key-value storage directly in context | store_* tools cannot target "whiteboard" store |
persistentInstructions |
false |
PersistentInstructionsPluginNextGen - agent instructions persisted to disk (KVP entries) | store_* tools cannot target "instructions" store |
userInfo |
false |
UserInfoPluginNextGen - user-scoped preferences + TODO tracking auto-injected into context | store_* tools cannot target "user_info" store; todo_* tools not registered |
toolCatalog |
false |
ToolCatalogPluginNextGen - dynamic tool loading/unloading by category | tool_catalog_* tools not registered; all tools must be pre-loaded |
sharedWorkspace |
false |
SharedWorkspacePluginNextGen - versioned multi-agent coordination board | store_* tools cannot target "workspace" |
memory |
false |
MemoryPluginNextGen - profile injection and six read tools | No durable graph/vector memory content or memory_* read tools; requires plugins.memory.memory when enabled |
memoryWrite |
false |
MemoryWritePluginNextGen - six controlled write tools | Agent remains unable to mutate durable memory; requires memory: true when enabled |
evidence |
disabled | Built-in evidence plugin | No evidence tools, projections, or storage writes |
taskState |
false |
TaskStatePluginNextGen | No protected task view; host scope/task ID required when enabled |
Usage Examples:
// 1. Managed context without scratch notes (history and whiteboard still enabled)
const ctx = AgentContextNextGen.create({
model: 'gpt-4.1',
features: { workingMemory: false }, // Disable working memory
});
console.log(ctx.memory); // null
// 2. Example with legacy compatibility stores enabled
const ctxVariant2 = AgentContextNextGen.create({
model: 'gpt-4.1',
features: {
workingMemory: true, // default: true
inContextMemory: true, // default: true
persistentInstructions: true, // default: false
userInfo: true, // default: false
},
});
// 3. Via Agent.create() - inline config
const agent = Agent.create({
connector: 'openai',
model: 'gpt-4.1',
context: {
model: 'gpt-4.1',
features: { workingMemory: false }, // Disable working memory
},
});
// 4. Agent with selected compatibility stores
const fullAgent = Agent.create({
connector: 'openai',
model: 'gpt-4.1',
userId: 'alice', // Optional for userInfo isolation
context: {
model: 'gpt-4.1',
agentId: 'my-agent',
features: {
workingMemory: true,
inContextMemory: true,
persistentInstructions: true,
userInfo: true,
},
},
});
Feature-Aware APIs:
// Check if a feature is enabled
ctx.features.workingMemory; // boolean
ctx.features.inContextMemory; // boolean
ctx.features.persistentInstructions; // boolean
ctx.features.userInfo; // boolean
// Get read-only feature configuration
ctx.features; // also includes sharedWorkspace, memory, memoryWrite, and registered external feature keys
// Access nullable memory
ctx.memory; // WorkingMemoryPluginNextGen | null
// Access plugins by name
ctx.getPlugin('working_memory'); // WorkingMemoryPluginNextGen | null
ctx.getPlugin('in_context_memory'); // InContextMemoryPluginNextGen | null
ctx.getPlugin('persistent_instructions'); // PersistentInstructionsPluginNextGen | null
ctx.getPlugin('user_info'); // UserInfoPluginNextGen | null
ctx.getPlugin('tool_catalog'); // ToolCatalogPluginNextGen | null
ctx.getPlugin('shared_workspace'); // SharedWorkspacePluginNextGen | null
ctx.getPlugin('memory'); // MemoryPluginNextGen | null
ctx.getPlugin('memory_write'); // MemoryWritePluginNextGen | null
// Check if plugin exists
ctx.hasPlugin('working_memory'); // boolean
Tool Auto-Registration:
AgentContextNextGen automatically registers feature-aware tools based on enabled features:
import { AgentContextNextGen } from 'oneringai';
// With workingMemory enabled (default)
const ctx = AgentContextNextGen.create({ model: 'gpt-4.1' });
console.log(ctx.tools.has('store_set')); // true (store_* tools registered)
// With workingMemory disabled - memory store not available
const ctx2 = AgentContextNextGen.create({
model: 'gpt-4.1',
features: { workingMemory: false },
});
// store_* tools are still registered, but store_set("notes", ...) will fail
Tools registered by feature:
All plugin stores are accessed through 5 unified store_* tools: store_get, store_set, store_delete, store_list, store_action. Each feature flag enables its corresponding store:
- workingMemory=true (default): enables
"notes"store (e.g.,store_set("notes", key, { description, value })) - inContextMemory=true: enables
"whiteboard"store (e.g.,store_set("whiteboard", key, { description, value })) - persistentInstructions=true: enables
"instructions"store (e.g.,store_set("instructions", key, { content })) - userInfo=true: enables
"user_info"store (e.g.,store_set("user_info", key, { value })) - sharedWorkspace=true: enables
"workspace"store (e.g.,store_set("workspace", key, { summary, content })) - memory=true: adds six
memory_*read tools and injects the configured user/profile view (these are notstore_*tools) - memoryWrite=true: adds six
memory_*write tools and requiresmemory=true
TODO tools (todo_add, todo_update, todo_remove) remain separate and are registered when userInfo=true.
Backward Compatibility:
- Default features:
workingMemory: true,inContextMemory: true,persistentInstructions: false,sharedWorkspace: false - Code not using
featuresconfig works unchanged
Conversation Management
AgentContextNextGen provides a simple API for managing conversation history:
import { AgentContextNextGen } from 'oneringai';
const ctx = AgentContextNextGen.create({ model: 'gpt-4.1' });
// Add user message
ctx.addUserMessage('Hello!');
// Prepare for LLM call (handles compaction if needed)
const { input, budget } = await ctx.prepare();
// ... call LLM with input ...
// Add assistant response
ctx.addAssistantResponse(response.output);
// Add tool results
ctx.addToolResults([
{ tool_use_id: 'call_123', content: JSON.stringify({ result: 'success' }), state: ToolCallState.COMPLETED }
]);
// Get conversation history
const conversation = ctx.getConversation();
// Clear conversation
ctx.clearConversation('Starting fresh');
Compaction:
Compaction happens during prepare() when content utilization exceeds the configured
critical threshold (0.9 by default), after subtracting response reserve and tools.
The strategy threshold determines the reclaim target; it does not trigger early
spilling below critical. Each preparation runs at most one strategy pass, rebuilds
all contributions, sanitizes tool groups before counting, and checks the final fit.
The strategy metadata is:
| Strategy | Reclaim threshold | Description |
|---|---|---|
algorithmic |
75% | Moves large tool results to Working Memory, limits tool pairs, applies rolling window (default) |
Custom strategies can be registered via StrategyRegistry.register().
Context Budget:
const { input, budget, compacted } = await ctx.prepare();
console.log(budget.utilizationPercent); // Current usage %
console.log(budget.available); // Remaining tokens
console.log(compacted); // true if compaction occurred
Final checks include synchronous request-event changes, the dispatch model’s capacity (with explicit host limits retained), structured schemas/native tools, and tool-call replay signatures. Managed JSON formatting and repair checks fit without replacing the main prepared revision. Algorithmic spill preserves connected native results and attachments before removal; storage failures leave history intact. Initial preparation also counts known schemas/native tools and resolves the initial provider window, so evictable history can be removed before final validation. Spill allocation checks existing memory keys across turns and save/load; collisions receive distinct suffixes and every new spill is verified before history removal. Function-result errors participate in the same accounting as custom-result errors.
Budgets are estimates, not provider-reported usage. agent.context.lastPreparation
returns a detached snapshot with revision, preparedAt, and budget for the last
selected request. Later context mutations do not rewrite that request's history.
lastBudget and calculateBudget() preserve their cached inspection semantics.
Use these getters for UI inspection instead of calling prepare() again.
Managed run() and stream() send the definitions counted during preparation,
including dynamic descriptions and identity filtering, and recount host-hook changes
without another compaction. Native async calls may remain outstanding; pending
results protect their connected calls. Duplicate final results for one call are
reported as INVALID_TOOL_PROTOCOL, while asynchronous completion messages remain
supported. Protected content that cannot fit produces ContextOverflowError.
The existing user rejection/tool-text emergency truncation policy is preserved.
See Context Preparation for additive adapter options,
legacy strategy behavior, concurrency bounds, estimation limits, and the offline
npm run example:context-freshness, npm run example:context-budget, and
npm run example:structured-budget demonstrations. AMOS labels selected revisions in
/context budget and /context breakdown.
Session Persistence
AgentContextNextGen supports saving and loading sessions:
import { AgentContextNextGen, createFileContextStorage } from 'oneringai';
// Create storage
const storage = createFileContextStorage('my-agent');
// Create context with storage
const ctx = AgentContextNextGen.create({
model: 'gpt-4.1',
features: { workingMemory: true },
storage,
});
// Add messages and data
ctx.addUserMessage('Hello');
// Save session
await ctx.save('session-001', { title: 'My Session' });
// Later: Load session
const ctx2 = AgentContextNextGen.create({ model: 'gpt-4.1', storage });
await ctx2.load('session-001');
// ctx2 now has full conversation and plugin states restored
Plugin System (NextGen)
Extend AgentContextNextGen with custom plugins:
import { type IContextPluginNextGen, BasePluginNextGen, AgentContextNextGen } from 'oneringai';
// Create a custom plugin by extending BasePluginNextGen
class MyPlugin extends BasePluginNextGen {
readonly name = 'my_plugin';
private data: string[] = [];
// Static usage instructions for the LLM (never compacted)
getInstructions(): string | null {
return null;
}
// Return content to be included in context (must update the token cache)
async getContent(): Promise<string | null> {
if (this.data.length === 0) return null;
const content = `## My Plugin Data\n${this.data.join('\n')}`;
this.updateTokenCache(this.estimator.estimateTokens(content));
return content;
}
// Raw data for programmatic inspection
getContents(): unknown {
return this.data;
}
addData(item: string) {
this.data.push(item);
this.invalidateTokenCache();
}
// Opt in to compaction (base class defaults to false)
isCompactable(): boolean {
return this.data.length > 0;
}
// Compact: reduce content to fit within targetTokensToFree
async compact(targetTokensToFree: number): Promise<number> {
const before = this.getTokenSize();
while (this.getTokenSize() > Math.max(0, before - targetTokensToFree) && this.data.length > 1) {
this.data.shift();
await this.getContent(); // refreshes the token cache
}
return before - this.getTokenSize();
}
// Serialize state for persistence
getState(): unknown {
return { data: this.data };
}
// Restore state
restoreState(state: unknown): void {
this.data = ((state as { data?: string[] }).data) || [];
this.invalidateTokenCache();
}
}
// Use the plugin
const ctx = AgentContextNextGen.create({ model: 'gpt-4.1' });
const plugin = new MyPlugin();
ctx.registerPlugin(plugin);
plugin.addData('Custom data');
Events
Monitor AgentContextNextGen activity:
const ctx = AgentContextNextGen.create({ model: 'gpt-4.1' });
// Message events
ctx.on('message:added', ({ role }) => {
console.log(`New ${role} message`);
});
// Compaction events
ctx.on('context:compacted', ({ tokensFreed }) => {
console.log(`Freed ${tokensFreed} tokens`);
});
// Budget events
ctx.on('budget:warning', ({ budget }) => {
console.log(`Context at ${budget.utilizationPercent.toFixed(1)}%`);
});
// Context prepared event
ctx.on('context:prepared', ({ budget }) => {
console.log(`Context prepared: ${budget.totalUsed}/${budget.maxTokens} tokens`);
});
Accessing Context in Agent
Agent uses AgentContextNextGen with plugins for extended functionality:
import { Agent, AgentContextNextGen, WorkingMemoryPluginNextGen } from 'oneringai';
// Create Agent with NextGen context
const agent = Agent.create({
connector: 'openai',
model: 'gpt-4.1',
tools: [myTool],
context: {
model: 'gpt-4.1',
features: { workingMemory: true, inContextMemory: true },
},
});
// Access AgentContextNextGen directly
agent.context.addUserMessage('Hello');
const { input, budget } = await agent.context.prepare();
agent.context.addAssistantResponse(response.output);
// Get conversation
const conversation = agent.context.getConversation(); // InputItem[]
// Access WorkingMemory via plugin
const memoryPlugin = agent.context.getPlugin('working_memory') as WorkingMemoryPluginNextGen;
await memoryPlugin.store('key', 'description', value);
// Access tools via context
agent.context.tools.disable('tool_name');
AgentContextNextGen API:
// AgentContextNextGen provides clean context management:
ctx.addUserMessage(content); // Set current user input
ctx.addAssistantResponse(response.output); // Add response to conversation
await ctx.prepare(); // Prepare context for LLM call, returns { input, budget }
ctx.getConversation(); // Get conversation history
ctx.registerPlugin(plugin); // Register context plugin
ctx.getPlugin(name); // Get registered plugin by name
await ctx.consolidate(); // Post-cycle consolidation (strategy-driven cleanup)
await ctx.save(sessionId); // Save session (if storage configured)
await ctx.load(sessionId); // Load session (if storage configured)
// Access ToolManager:
ctx.tools; // ToolManager instance
NextGen Plugin System:
Use NextGen plugins to extend AgentContextNextGen:
import { BasePluginNextGen, WorkingMemoryPluginNextGen, InContextMemoryPluginNextGen } from 'oneringai';
// Built-in NextGen plugins:
// - WorkingMemoryPluginNextGen: Tiered memory (raw/summary/findings)
// - InContextMemoryPluginNextGen: Live key-value storage in context
// - PersistentInstructionsPluginNextGen: Disk-persisted instructions
// Custom plugin example:
class MyPlugin extends BasePluginNextGen {
readonly name = 'my_plugin';
getInstructions(): string | null {
return null;
}
async getContent(): Promise<string | null> {
const content = 'Custom context content';
this.updateTokenCache(this.estimator.estimateTokens(content));
return content;
}
getContents(): unknown {
return null;
}
}
ctx.registerPlugin(new MyPlugin());
Why Context Management Matters
LLMs have fixed context windows (e.g., 128K tokens for GPT-4, 200K for Claude). As conversations grow, you must:
- Track usage to avoid hitting limits
- Prioritize content (instructions vs history vs memory)
- Compact intelligently when approaching limits
- Preserve critical information while freeing space
The context management system handles all of this automatically.
Basic Context Management
Context management is automatic with AgentContextNextGen:
import { Agent } from 'oneringai';
const agent = Agent.create({
connector: 'openai',
model: 'gpt-4.1',
tools: [myTool],
context: {
model: 'gpt-4.1',
strategy: 'algorithmic', // Default: compact at 75% utilization
features: { workingMemory: true },
},
});
// AgentContextNextGen will automatically:
// 1. Track context usage across all plugins
// 2. Compact when approaching limits (at prepare() time)
// 3. Evict low-priority memory entries when needed
// 4. Call plugin compact() methods in priority order
// 5. Emit events for monitoring
Architecture Overview
The context management system is built around AgentContextNextGen - the clean, plugin-first context manager:
┌─────────────────────────────────────────────────────┐
│ AgentContextNextGen │
│ - Plugin-first architecture │
│ - Clean message flow (addUserMessage → prepare) │
│ - Single compaction point (right before LLM call) │
└─────────────────┬───────────────────────────────────┘
│
┌─────────────┼─────────────┐
│ │ │
▼ ▼ ▼
┌─────────┐ ┌──────────┐ ┌───────────────┐
│ Strategy│ │ Plugins │ │ Context │
│ (how) │ │ (what) │ │ Structure │
└─────────┘ └──────────┘ └───────────────┘
Critical gate: Decides WHEN to compact (90% content utilization by default)
Strategy: Decides HOW to reclaim and its target (algorithmic threshold: 75%)
Plugins: WorkingMemoryPluginNextGen, InContextMemoryPluginNextGen, etc.
Context: Developer Message → Conversation History → Current Input
Manual Context Management
For advanced use cases, use AgentContextNextGen with plugins:
import {
AgentContextNextGen,
WorkingMemoryPluginNextGen,
InContextMemoryPluginNextGen,
} from 'oneringai';
// Create AgentContextNextGen with configuration
const ctx = AgentContextNextGen.create({
model: 'gpt-4.1',
systemPrompt: 'Your system instructions',
maxContextTokens: 128000, // Model's context window
responseReserve: 4096, // Reserve tokens for response
strategy: 'algorithmic', // Default strategy (75% reclaim threshold; critical gate controls triggering)
features: {
workingMemory: true, // Enable WorkingMemoryPluginNextGen
inContextMemory: true, // Enable InContextMemoryPluginNextGen
},
});
// Plugins are auto-registered when features are enabled
// Access them via getPlugin():
const memoryPlugin = ctx.getPlugin('working_memory') as WorkingMemoryPluginNextGen;
const inContextPlugin = ctx.getPlugin('in_context_memory') as InContextMemoryPluginNextGen;
// Add user message (sets _currentInput)
ctx.addUserMessage('Current task description');
// Prepare context before each LLM call
const { input, budget } = await ctx.prepare();
console.log(`Context: ${budget.totalUsed}/${budget.maxTokens} tokens`);
console.log(`Utilization: ${budget.utilizationPercent.toFixed(1)}%`);
// After LLM response, add it to conversation
ctx.addAssistantResponse(llmResponse);
// Get conversation history
const conversation = ctx.getConversation(); // InputItem[]
Compactors Deep Dive
Note:
TruncateCompactor,MemoryEvictionCompactor, andSummarizeCompactorimplement the legacyIContextCompactor/IContextComponentinterfaces (src/core/context/types.ts) and are standalone utility classes —AgentContextNextGennever uses them. Its actual compaction is driven entirely byICompactionStrategyimplementations (e.g.AlgorithmicCompactionStrategy), which contain no reference to these compactor classes.
Compactors determine how content is reduced during compaction. Each compactor handles components with a matching strategy metadata.
Available Compactors
| Compactor | Strategy | Priority | What It Does |
|---|---|---|---|
| TruncateCompactor | truncate |
10 | Removes content from the end |
| MemoryEvictionCompactor | evict |
8 | Evicts low-priority memory entries |
| SummarizeCompactor | summarize |
5 | Uses LLM to create intelligent summaries |
Lower priority number = runs earlier (summarize before truncate).
SummarizeCompactor (LLM-Based)
The SummarizeCompactor uses an LLM to intelligently summarize content, preserving key information while reducing token count.
import { SummarizeCompactor, ApproximateTokenEstimator } from 'oneringai';
const estimator = new ApproximateTokenEstimator();
// Create summarize compactor with LLM
const summarizeCompactor = new SummarizeCompactor(estimator, {
textProvider: myTextProvider, // Required: LLM for summarization
model: 'gpt-4o-mini', // Optional: model to use (default: same as agent)
maxSummaryTokens: 500, // Optional: max tokens for summary
preserveStructure: true, // Optional: keep headings/lists (default: true)
fallbackToTruncate: true, // Optional: truncate if LLM fails (default: true)
});
// Components with strategy: 'summarize' will use this compactor
const component = {
name: 'conversation_history',
content: longConversation,
priority: 6,
compactable: true,
metadata: { strategy: 'summarize' }, // Uses SummarizeCompactor
};
What SummarizeCompactor Preserves:
For conversation history:
- Key decisions made
- Important facts discovered
- User preferences expressed
- Unresolved questions
For tool outputs (search/scrape results):
- Key findings relevant to the task
- Source URLs and main points
- Factual data (numbers, dates, names)
- Contradictions between sources
// Example: Research task with summarization
// AgentContextNextGen is the unified context manager - configure via Agent.create()
const agent = Agent.create({
connector: 'openai',
model: 'gpt-4.1',
context: {
model: 'gpt-4.1',
features: { workingMemory: true },
},
});
// Access context management via agent.context (AgentContextNextGen instance)
const budget = agent.context.lastPreparation?.budget; // Inspect after run/stream; do not prepare twice.
MemoryEvictionCompactor
Evicts low-priority memory entries based on the avgEntrySize metadata.
import { MemoryEvictionCompactor } from 'oneringai';
const evictionCompactor = new MemoryEvictionCompactor(estimator);
// Components with strategy: 'evict' will use this compactor
const memoryComponent = {
name: 'memory_index',
content: memoryIndex,
priority: 8,
compactable: true,
metadata: {
strategy: 'evict',
avgEntrySize: 100, // Average tokens per entry
evict: async (count) => { await evictMemoryEntries(count); },
getUpdatedContent: async () => getMemoryIndex(),
},
};
Pre-Compaction Hooks
The beforeCompaction lifecycle hook allows agents to save important data before compaction occurs. This is critical for research tasks where tool outputs may contain valuable information.
import { Agent } from 'oneringai';
// The callback parameter is inferred from AgentConfig; component summaries contain metadata only.
// Define lifecycle hooks when creating the agent
const agent = Agent.create({
connector: 'openai',
model: 'gpt-4.1',
lifecycleHooks: {
beforeCompaction: async (context) => {
console.log(`Agent ${context.agentId}: Compaction starting`);
console.log(`Current usage: ${context.currentBudget.used}/${context.currentBudget.total}`);
console.log(`Need to free: ${context.estimatedTokensToFree} tokens`);
console.log(`Strategy: ${context.strategy}`);
console.log(`Components to compact: ${context.components.length}`);
// These are metadata summaries, not source bodies.
for (const component of context.components) {
console.log(component.name, component.compactable);
}
// Preserve application-owned findings through the enabled memory plugin;
// see the native-hooks example for a complete bounded implementation.
},
},
});
// AgentContextNextGen handles all context management internally
// No separate ContextManager needed
BeforeCompactionContext
The hook receives detailed context about the upcoming compaction:
interface BeforeCompactionContext {
/** Agent identifier */
agentId: string;
/** Current context budget info (a lightweight summary, not the full ContextBudget) */
currentBudget: {
total: number;
used: number;
available: number;
utilizationPercent: number;
status: 'ok' | 'warning' | 'critical';
};
/** Strategy being used (e.g. 'algorithmic') */
strategy: string;
/** Components about to be compacted (read-only summaries) */
components: ReadonlyArray<{
name: string;
priority: number;
compactable: boolean;
}>;
/** Estimated tokens that need to be freed */
estimatedTokensToFree: number;
}
Error Handling in Hooks
Hooks are designed to be resilient - errors are logged but don't prevent compaction:
const hooks = {
beforeCompaction: async (context) => {
// Even if this throws, compaction will continue
throw new Error('Hook error');
},
};
// Compaction proceeds, error is logged to console
Context Strategies Deep Dive
AgentContextNextGen uses a strategy-based compaction system with the ICompactionStrategy interface. The strategy controls when and how context is compacted.
Built-in Strategies
| Strategy | Reclaim threshold | Description |
|---|---|---|
| algorithmic (default) | 75% | Moves large tool results to Working Memory, limits tool pairs to 10, applies rolling window. Best for tool-heavy agents. |
The algorithmic strategy is the default. With working memory enabled it can
retain removed tool sources there. Without it, that offload step is skipped;
rolling-window compaction still works within protection and fit constraints.
// Default - uses algorithmic strategy automatically
const agent = Agent.create({
connector: 'openai',
model: 'gpt-4.1',
context: { model: 'gpt-4.1', features: { workingMemory: true } },
});
// Explicit strategy selection
const agentVariant2 = Agent.create({
connector: 'openai',
model: 'gpt-4.1',
context: { model: 'gpt-4.1', strategy: 'algorithmic' },
});
Creating Custom Strategies
The existing ICompactionStrategy and StrategyRegistry remain extension points.
Start with a configured built-in strategy so its connected tool-group, pending-source,
and evidence protections continue to apply:
import { Agent, StrategyRegistry } from 'oneringai';
const earlierReclaim = StrategyRegistry.create('algorithmic', { threshold: 0.60 });
const agent = Agent.create({
connector: 'openai', model: 'gpt-4.1',
context: { model: 'gpt-4.1', compactionStrategy: earlierReclaim },
});
The strategy threshold selects a reclaim target; it is not a replacement for the
context's critical trigger. A custom implementation must preserve connected
function/native call-result groups, pending input, opaque provider ownership,
protected task state, and unacknowledged observations. Do not delete an arbitrary
percentage of messages. Final preparation may reject an unsafe or oversized result.
Use context.compactionStrategy for a configured instance; the
native composition recipe uses that same field.
Using Strategies
// Default - algorithmic strategy (recommended for most use cases)
const agent = Agent.create({
connector: 'openai',
model: 'gpt-4.1',
// strategy defaults to 'algorithmic' (75% reclaim target; 90% critical trigger)
});
// Custom registered strategy
const agentVariant2 = Agent.create({
connector: 'openai',
model: 'gpt-4.1',
context: { model: 'gpt-4.1', compactionStrategy: earlierReclaim }, // Instance configured above
});
Token Estimation
The ApproximateTokenEstimator provides content-type-aware estimation:
import { ApproximateTokenEstimator } from 'oneringai';
const estimator = new ApproximateTokenEstimator();
// Basic estimation (mixed content assumed)
const tokens1 = estimator.estimateTokens('Hello, world!');
// Content-type-aware estimation for better accuracy
const codeTokens = estimator.estimateTokens(sourceCode, 'code'); // ~3 chars/token
const proseTokens = estimator.estimateTokens(essay, 'prose'); // ~4 chars/token
const mixedTokens = estimator.estimateTokens(readme, 'mixed'); // ~3.5 chars/token
// Estimate structured data
const dataTokens = estimator.estimateDataTokens({ users: [], config: {} });
Why content type matters:
- Code has more special characters and shorter words → fewer chars/token
- Prose has longer words and punctuation → more chars/token
- Accurate estimation prevents over/under-compaction
Context Budget Monitoring
// Get budget from prepare() call
const { input, budget, compacted } = await agent.context.prepare();
console.log(`Max tokens: ${budget.maxTokens}`);
console.log(`Used tokens: ${budget.totalUsed}`);
console.log(`Available: ${budget.available}`);
console.log(`Utilization: ${budget.utilizationPercent.toFixed(1)}%`);
// Check if compaction occurred
if (compacted) {
console.log('Context was compacted to make room');
}
// Detailed breakdown
console.log('Breakdown:');
console.log(` System prompt: ${budget.breakdown.systemPrompt} tokens`);
console.log(` Plugin instructions: ${budget.breakdown.pluginInstructions} tokens`);
console.log(` Conversation: ${budget.breakdown.conversation} tokens`);
console.log(` Current input: ${budget.breakdown.currentInput} tokens`);
Agent Lifecycle Hooks for Context
Use lifecycle hooks to integrate context management with your application. Caveat: in the current Agent implementation, only beforeCompaction is actually invoked at runtime (wired in Agent's constructor via AgentContextNextGen.setBeforeCompactionCallback). beforeContextPrepare, afterCompaction, beforeToolExecution, afterToolExecution, and onError can be registered via agent.setLifecycleHooks(), but no code path in Agent/BaseAgent currently calls their invoke* helpers, so they will not fire.
import type { AgentConfig } from 'oneringai';
type LifecycleHooks = NonNullable<AgentConfig['lifecycleHooks']>;
const hooks: LifecycleHooks = {
// This is the lifecycle hook currently wired by Agent.
beforeCompaction: async (context) => {
await monitoring.record({
event: 'context_compaction_starting',
agentId: context.agentId,
utilizationPercent: context.currentBudget.utilizationPercent,
strategy: context.strategy,
estimatedTokensToFree: context.estimatedTokensToFree,
});
},
};
// Apply hooks to agent
agent.setLifecycleHooks(hooks);
Best Practices for Context Management
1. Use the Default Strategy
The algorithmic strategy (default, 75% reclaim threshold; 90% critical trigger) works well for most use cases. It automatically offloads large tool results to Working Memory and manages conversation history:
import { Agent } from 'oneringai';
// Default algorithmic strategy - recommended for most use cases
const agent = Agent.create({
connector: 'openai',
model: 'gpt-4.1',
context: { model: 'gpt-4.1', features: { workingMemory: true } },
});
// For custom compaction behavior, register a custom strategy via StrategyRegistry
2. Monitor in Production
// Set up monitoring with AgentContextNextGen events
const ctx = agent.context;
ctx.on('context:compacted', async ({ tokensFreed }) => {
await metrics.gauge('context.tokens_freed', tokensFreed);
});
ctx.on('context:prepared', async ({ budget }) => {
await metrics.gauge('context.usage', budget.totalUsed);
await metrics.gauge('context.total', budget.maxTokens);
});
ctx.on('budget:warning', async ({ budget }) => {
const utilization = Math.round(budget.utilizationPercent);
await alerts.warn(`Context warning: ${utilization}%`);
});
3. Use WorkingMemory Tiers
// Store data in appropriate tiers based on importance
const memoryPlugin = ctx.getPlugin('working_memory') as WorkingMemoryPluginNextGen;
// Raw tier: Large, unprocessed data (evicted first)
await memoryPlugin.store('search.results', 'Raw search results', largeResults, { tier: 'raw' });
// Summary tier: Condensed information
await memoryPlugin.store('search.summary', 'Search summary', summaryData, { tier: 'summary' });
// Findings tier: Key insights (evicted last)
await memoryPlugin.store('search.findings', 'Key findings', findings, { tier: 'findings' });
4. Plan for Compaction
// Structure data for efficient compaction using tiers
const memoryPlugin = ctx.getPlugin('working_memory') as WorkingMemoryPluginNextGen;
// BAD: Single large object in findings (won't be evicted easily)
await memoryPlugin.store('all.data', 'All data', hugeObject, { tier: 'findings' });
// GOOD: Split by importance using tiers
// Raw tier: Evicted first during compaction
await memoryPlugin.store('data.raw', 'Raw data', rawData, { tier: 'raw' });
// Summary tier: Evicted second
await memoryPlugin.store('data.summary', 'Summarized data', summaryData, { tier: 'summary' });
// Findings tier: Evicted last (most important)
await memoryPlugin.store('data.findings', 'Key findings', findings, { tier: 'findings' });