Resilience & Observability
Resilience & Observability
Circuit breakers, retries, logging, and metrics
CircuitBreaker class
📍 src/infrastructure/resilience/CircuitBreaker.ts:114
Generic circuit breaker for any async operation
Constructor
constructor
constructor(
public readonly name: string,
config: Partial<CircuitBreakerConfig> = {}
)
Parameters:
name:stringconfig:Partial<CircuitBreakerConfig>(optional) (default:{})
Methods
execute()
Execute function with circuit breaker protection
async execute(fn: () => Promise<T>): Promise<T>
Parameters:
fn:() => Promise<T>
Returns: Promise<T>
getState()
Get current state
getState(): CircuitState
Returns: CircuitState
getMetrics()
Get current metrics
getMetrics(): CircuitBreakerMetrics
Returns: CircuitBreakerMetrics
reset()
Manually reset circuit breaker (force close)
reset(): void
Returns: void
isOpen()
Check if circuit is allowing requests
isOpen(): boolean
Returns: boolean
getConfig()
Get configuration
getConfig(): CircuitBreakerConfig
Returns: CircuitBreakerConfig
CircuitOpenError class
📍 src/infrastructure/resilience/CircuitBreaker.ts:94
Circuit breaker error - thrown when circuit is open
Constructor
constructor
constructor(
public readonly breakerName: string,
public readonly nextRetryTime: number,
public readonly failureCount: number,
public readonly lastError: string
)
Parameters:
breakerName:stringnextRetryTime:numberfailureCount:numberlastError:string
ConsoleMetrics class
📍 src/infrastructure/observability/Metrics.ts:50
Console metrics collector (development/debugging)
Constructor
constructor
constructor(prefix: string = 'oneringai')
Parameters:
prefix:string(optional) (default:'oneringai')
Methods
increment()
increment(metric: string, value: number = 1, tags?: MetricTags): void
Parameters:
metric:stringvalue:number(optional) (default:1)tags:MetricTags | undefined(optional)
Returns: void
gauge()
gauge(metric: string, value: number, tags?: MetricTags): void
Parameters:
metric:stringvalue:numbertags:MetricTags | undefined(optional)
Returns: void
timing()
timing(metric: string, duration: number, tags?: MetricTags): void
Parameters:
metric:stringduration:numbertags:MetricTags | undefined(optional)
Returns: void
histogram()
histogram(metric: string, value: number, tags?: MetricTags): void
Parameters:
metric:stringvalue:numbertags:MetricTags | undefined(optional)
Returns: void
FrameworkLogger class
📍 src/infrastructure/observability/Logger.ts:158
Framework logger
Constructor
constructor
constructor(config: LoggerConfig = {})
Parameters:
config:LoggerConfig(optional) (default:{})
Methods
child()
Create child logger with additional context
child(context: Record<string, any>): FrameworkLogger
Parameters:
context:Record<string, any>
Returns: FrameworkLogger
trace()
Trace log
trace(obj: Record<string, any> | string, msg?: string): void
Parameters:
obj:string | Record<string, any>msg:string | undefined(optional)
Returns: void
debug()
Debug log
debug(obj: Record<string, any> | string, msg?: string): void
Parameters:
obj:string | Record<string, any>msg:string | undefined(optional)
Returns: void
info()
Info log
info(obj: Record<string, any> | string, msg?: string): void
Parameters:
obj:string | Record<string, any>msg:string | undefined(optional)
Returns: void
warn()
Warn log
warn(obj: Record<string, any> | string, msg?: string): void
Parameters:
obj:string | Record<string, any>msg:string | undefined(optional)
Returns: void
error()
Error log
error(obj: Record<string, any> | string, msg?: string): void
Parameters:
obj:string | Record<string, any>msg:string | undefined(optional)
Returns: void
updateConfig()
Update configuration
updateConfig(config: Partial<LoggerConfig>): void
Parameters:
config:Partial<LoggerConfig>
Returns: void
close()
Cleanup resources (call before process exit)
close(): void
Returns: void
getLevel()
Get current log level
getLevel(): LogLevel
Returns: LogLevel
isLevelEnabled()
Check if level is enabled
isLevelEnabled(level: LogLevel): boolean
Parameters:
level:LogLevel
Returns: boolean
NoOpMetrics class
📍 src/infrastructure/observability/Metrics.ts:40
No-op metrics collector (default - zero overhead)
Methods
increment()
increment(): void
Returns: void
gauge()
gauge(): void
Returns: void
timing()
timing(): void
Returns: void
histogram()
histogram(): void
Returns: void
RateLimitError class
📍 src/infrastructure/resilience/RateLimiter.ts:13
Error thrown when rate limit is exceeded and onLimit is 'throw'
Constructor
constructor
constructor(
public readonly retryAfterMs: number,
message?: string
)
Parameters:
retryAfterMs:numbermessage:string | undefined(optional)
TokenBucketRateLimiter class
📍 src/infrastructure/resilience/RateLimiter.ts:75
Token bucket rate limiter implementation
Uses a sliding window approach where tokens are refilled completely when the time window expires.
Constructor
constructor
constructor(config: Partial<RateLimiterConfig> = {})
Parameters:
config:Partial<RateLimiterConfig>(optional) (default:{})
Methods
acquire()
Acquire a token (request permission to make an LLM call)
async acquire(): Promise<void>
Returns: Promise<void>
tryAcquire()
Try to acquire without waiting
tryAcquire(): boolean
Returns: boolean
getAvailableTokens()
Get current available tokens
getAvailableTokens(): number
Returns: number
getWaitTime()
Get time until next token is available
getWaitTime(): number
Returns: number
getMetrics()
Get rate limiter metrics
getMetrics(): RateLimiterMetrics
Returns: RateLimiterMetrics
reset()
Reset the rate limiter state
reset(): void
Returns: void
resetMetrics()
Reset metrics
resetMetrics(): void
Returns: void
getConfig()
Get the current configuration
getConfig(): Required<RateLimiterConfig>
Returns: Required<RateLimiterConfig>
AgentMetrics interface
📍 src/domain/entities/AgentState.ts:43
Agent execution metrics
Properties
| Property | Type | Description |
|---|---|---|
totalLLMCalls |
totalLLMCalls: number; |
- |
totalToolCalls |
totalToolCalls: number; |
- |
totalTokensUsed |
totalTokensUsed: number; |
- |
totalCost |
totalCost: number; |
- |
BackoffConfig interface
📍 src/infrastructure/resilience/BackoffStrategy.ts:13
Backoff configuration
Properties
| Property | Type | Description |
|---|---|---|
strategy |
strategy: BackoffStrategyType; |
Strategy type |
initialDelayMs |
initialDelayMs: number; |
Initial delay in ms |
maxDelayMs |
maxDelayMs: number; |
Maximum delay in ms |
multiplier? |
multiplier?: number; |
Multiplier for exponential (default: 2) |
incrementMs? |
incrementMs?: number; |
Increment for linear (default: 1000ms) |
jitter? |
jitter?: boolean; |
Add random jitter to prevent thundering herd |
jitterFactor? |
jitterFactor?: number; |
Jitter factor (0-1, default: 0.1 = ±10%) |
isRetryable? |
isRetryable?: (error: Error) => boolean; |
Classify errors - return true if error should be retried |
CircuitBreakerConfig interface
📍 src/infrastructure/resilience/CircuitBreaker.ts:26
Circuit breaker configuration
Properties
| Property | Type | Description |
|---|---|---|
failureThreshold |
failureThreshold: number; |
Number of failures before opening circuit |
successThreshold |
successThreshold: number; |
Number of successes to close from half-open |
resetTimeoutMs |
resetTimeoutMs: number; |
Time to wait in open state before trying half-open (ms) |
windowMs |
windowMs: number; |
Time window for counting failures (ms) |
isRetryable? |
isRetryable?: (error: Error) => boolean; |
Classify errors - return true if error should count as failure |
CircuitBreakerEvents interface
📍 src/infrastructure/resilience/CircuitBreaker.ts:74
Circuit breaker events
Properties
| Property | Type | Description |
|---|---|---|
opened |
opened: { name: string; failureCount: number; lastError: string; nextRetryTime: number }; |
- |
'half-open' |
'half-open': { name: string; timestamp: number }; |
- |
closed |
closed: { name: string; successCount: number; timestamp: number }; |
- |
CircuitBreakerMetrics interface
📍 src/infrastructure/resilience/CircuitBreaker.ts:46
Circuit breaker metrics
Properties
| Property | Type | Description |
|---|---|---|
name |
name: string; |
- |
state |
state: CircuitState; |
- |
totalRequests |
totalRequests: number; |
- |
successCount |
successCount: number; |
- |
failureCount |
failureCount: number; |
- |
rejectedCount |
rejectedCount: number; |
- |
recentFailures |
recentFailures: number; |
- |
consecutiveSuccesses |
consecutiveSuccesses: number; |
- |
lastFailureTime? |
lastFailureTime?: number; |
- |
lastSuccessTime? |
lastSuccessTime?: number; |
- |
lastStateChange |
lastStateChange: number; |
- |
nextRetryTime? |
nextRetryTime?: number; |
- |
failureRate |
failureRate: number; |
- |
successRate |
successRate: number; |
- |
ExecutionMetrics interface
📍 src/capabilities/agents/ExecutionContext.ts:36
Properties
| Property | Type | Description |
|---|---|---|
totalDuration |
totalDuration: number; |
- |
llmDuration |
llmDuration: number; |
- |
toolDuration |
toolDuration: number; |
- |
hookDuration |
hookDuration: number; |
- |
iterationCount |
iterationCount: number; |
- |
toolCallCount |
toolCallCount: number; |
- |
toolSuccessCount |
toolSuccessCount: number; |
- |
toolFailureCount |
toolFailureCount: number; |
- |
toolTimeoutCount |
toolTimeoutCount: number; |
- |
inputTokens |
inputTokens: number; |
- |
outputTokens |
outputTokens: number; |
- |
totalTokens |
totalTokens: number; |
- |
cachedInputTokens? |
cachedInputTokens?: number; |
Optional for source compatibility; ExecutionContext instances always initialize this to 0. |
cacheCreationInputTokens? |
cacheCreationInputTokens?: number; |
Optional for source compatibility; ExecutionContext instances always initialize this to 0. |
reasoningTokens? |
reasoningTokens?: number; |
Optional for source compatibility; ExecutionContext instances always initialize this to 0. |
nativeToolCalls? |
nativeToolCalls?: Record<string, number>; |
Optional for source compatibility; ExecutionContext instances always initialize this to {}. |
errors |
errors: Array<{ type: string; message: string; timestamp: Date }>; |
- |
LogEntry interface
📍 src/infrastructure/observability/Logger.ts:148
Log entry
Properties
| Property | Type | Description |
|---|---|---|
level |
level: LogLevel; |
- |
time |
time: number; |
- |
msg |
msg: string; |
- |
LoggerConfig interface
📍 src/infrastructure/observability/Logger.ts:128
Logger configuration
Properties
| Property | Type | Description |
|---|---|---|
level? |
level?: LogLevel; |
Log level |
pretty? |
pretty?: boolean; |
Pretty print for development |
context? |
context?: Record<string, any>; |
Base context added to all logs |
destination? |
`destination?: 'console' | 'stdout' |
filePath? |
filePath?: string; |
File path for file logging |
MetricsCollector interface
📍 src/infrastructure/observability/Metrics.ts:15
Metrics collector interface
Methods
increment()
Increment a counter
increment(metric: string, value?: number, tags?: MetricTags): void;
Parameters:
metric:stringvalue:number | undefined(optional)tags:MetricTags | undefined(optional)
Returns: void
gauge()
Set a gauge value
gauge(metric: string, value: number, tags?: MetricTags): void;
Parameters:
metric:stringvalue:numbertags:MetricTags | undefined(optional)
Returns: void
timing()
Record a timing/duration
timing(metric: string, duration: number, tags?: MetricTags): void;
Parameters:
metric:stringduration:numbertags:MetricTags | undefined(optional)
Returns: void
histogram()
Record a histogram value
histogram(metric: string, value: number, tags?: MetricTags): void;
Parameters:
metric:stringvalue:numbertags:MetricTags | undefined(optional)
Returns: void
RateLimiterConfig interface
📍 src/infrastructure/resilience/RateLimiter.ts:27
Configuration for the rate limiter
Properties
| Property | Type | Description |
|---|---|---|
maxRequests |
maxRequests: number; |
Max requests allowed in window |
windowMs? |
windowMs?: number; |
Time window in ms (default: 60000 = 1 minute) |
onLimit |
`onLimit: 'wait' | 'throw';` |
maxWaitMs? |
maxWaitMs?: number; |
Max wait time in ms (for 'wait' mode, default: 60000) |
maxQueueSize? |
maxQueueSize?: number; |
Max queued waiters (default: 500). Rejects new requests when exceeded. |
RateLimiterMetrics interface
📍 src/infrastructure/resilience/RateLimiter.ts:58
Rate limiter metrics
Properties
| Property | Type | Description |
|---|---|---|
totalRequests |
totalRequests: number; |
Total requests made |
throttledRequests |
throttledRequests: number; |
Total requests throttled |
totalWaitMs |
totalWaitMs: number; |
Total wait time in ms |
avgWaitMs |
avgWaitMs: number; |
Average wait time in ms |
BackoffStrategyType type
📍 src/infrastructure/resilience/BackoffStrategy.ts:8
Backoff strategies for retry logic
type BackoffStrategyType = 'exponential' | 'linear' | 'constant'
CircuitState type
📍 src/infrastructure/resilience/CircuitBreaker.ts:13
Circuit breaker states
type CircuitState = 'closed' | 'open' | 'half-open'
LogLevel type
📍 src/infrastructure/observability/Logger.ts:19
Log level
type LogLevel = 'trace' | 'debug' | 'info' | 'warn' | 'error' | 'silent'
MetricsCollectorType type
📍 src/infrastructure/observability/Metrics.ts:187
Metrics collector type
type MetricsCollectorType = 'noop' | 'console' | 'inmemory'
MetricTags type
📍 src/infrastructure/observability/Metrics.ts:10
Metrics collection infrastructure
Pluggable metrics system with support for various backends.
type MetricTags = Record<string, string | number | boolean>
addJitter function
📍 src/infrastructure/resilience/BackoffStrategy.ts:92
Add random jitter to a delay
export function addJitter(delay: number, factor: number = 0.1): number
backoffSequence function
📍 src/infrastructure/resilience/BackoffStrategy.ts:116
Backoff iterator - generates delays for each attempt
export function* backoffSequence(
config: BackoffConfig = DEFAULT_BACKOFF_CONFIG,
maxAttempts?: number
): Generator<number, void, unknown>
backoffWait function
📍 src/infrastructure/resilience/BackoffStrategy.ts:105
Wait for backoff delay
export async function backoffWait(attempt: number, config: BackoffConfig = DEFAULT_BACKOFF_CONFIG): Promise<number>
calculateBackoff function
📍 src/infrastructure/resilience/BackoffStrategy.ts:54
Calculate backoff delay for given attempt
export function calculateBackoff(attempt: number, config: BackoffConfig = DEFAULT_BACKOFF_CONFIG): number
createMetricsCollector function
📍 src/infrastructure/observability/Metrics.ts:192
Create metrics collector from type
export function createMetricsCollector(type?: MetricsCollectorType, prefix?: string): MetricsCollector
retryWithBackoff function
📍 src/infrastructure/resilience/BackoffStrategy.ts:140
Retry with backoff
export async function retryWithBackoff<T>(
fn: () => Promise<T>,
config: BackoffConfig = DEFAULT_BACKOFF_CONFIG,
maxAttempts?: number
): Promise<T>
setMetricsCollector function
📍 src/infrastructure/observability/Metrics.ts:216
Update global metrics collector
export function setMetricsCollector(collector: MetricsCollector): void
DEFAULT_BACKOFF_CONFIG const
📍 src/infrastructure/resilience/BackoffStrategy.ts:42
Default backoff configuration
Properties
| Property | Type | Description |
|---|---|---|
strategy |
'exponential' |
- |
initialDelayMs |
1000 |
- |
maxDelayMs |
30000 |
- |
multiplier |
2 |
- |
jitter |
true |
- |
jitterFactor |
0.1 |
- |
DEFAULT_CIRCUIT_BREAKER_CONFIG const
📍 src/infrastructure/resilience/CircuitBreaker.ts:83
Default configuration
Properties
| Property | Type | Description |
|---|---|---|
failureThreshold |
5 |
- |
successThreshold |
2 |
- |
resetTimeoutMs |
30000 |
- |
windowMs |
60000 |
- |
isRetryable |
() => true |
- |
DEFAULT_RATE_LIMITER_CONFIG const
📍 src/infrastructure/resilience/RateLimiter.ts:47
Default rate limiter configuration
Properties
| Property | Type | Description |
|---|---|---|
maxRequests |
60 |
- |
windowMs |
60000 |
- |
onLimit |
'wait' |
- |
maxWaitMs |
60000 |
- |
maxQueueSize |
500 |
- |