Agent · Real-time Video Intelligence

LiveOps Real-time Vision Agent

Continuously understand live video from warehouses, factories, and stores, turning changes involving people, equipment, and goods into searchable and actionable operational events.

Vision LLMObject TrackingVideo RAGBLE / IoTMCP Tools

Try the interactive demo

Solution Case

Solution overview

From passive monitoring to proactive operations intelligence

The platform ingests live and archived video, performs edge-side detection, cross-frame tracking, and privacy processing, then uses vision-language models to understand spatiotemporal relationships between workers, forklifts, pallets, and operating zones. Video events are correlated with BLE, RFID, IoT sensors, orders, and warehouse data. The Agent can retrieve key clips in natural language, build incident timelines, verify alerts, and use MCP tools to create work orders, notify owners, or trigger review workflows. High-risk events retain timestamps, trajectories, and business evidence for authorized human approval.

Reduce continuous manual screen monitoring and bring anomaly discovery, evidence retrieval, and coordinated response into one workflow while preserving real-time performance, traceability, and human control.

VLM + TrackingReal-time scene understanding

Video RAGNatural-language event retrieval

MCP ActionsAlerts and work-order execution

Implementation flow

Closed loop from video perception to operational action

Perceive live

Ingest cameras and archived video, then decode, redact, detect, and continuously track objects at the edge.

Understand scenes

Use VLMs to identify objects, actions, zones, and temporal relationships as searchable semantic events.

Correlate sources

Fuse BLE, RFID, IoT, order, and warehouse data to verify visual events and reduce false positives.

Let the Agent act

Generate summaries and evidence, invoke MCP tools for work orders and notifications, and request approval for high-risk actions.

Production Architecture

Always-on lightweight perception, on-demand multimodal reasoning

Hardware decoding, compact detectors, and tracking run continuously. Only anomalies selected by operational rules and business data reach the VLM, controlling GPU cost, response latency, and false positives by design.

Edge Ingest

Camera video

Reuse existing RTSP / ONVIF cameras, ingest live streams at the edge, and retain raw recordings and incident clips according to access policy.

Decode & Sample

Hardware decode and frame sampling

Batch multiple streams with hardware decoders such as NVDEC and adapt sampling to object speed and scene risk instead of sending every frame to a model.

Always-on CV

Compact detection models + tracking

Lightweight detectors continuously identify people, forklifts, pallets, and vehicles while trackers maintain IDs, trajectories, speed, and dwell time across frames.

Event Gate

Rules and operational-data filtering

Correlate zone crossings, dwell, and congestion rules with BLE, RFID, WMS, orders, and shift data, retaining only candidate events that need deeper explanation.

Only anomalous clips enter high-cost inference

On-demand VLM

VLM verification and scene explanation

Analyze only key frames or short clips around an event to understand spatiotemporal relationships, explain the anomaly, and create semantic events with timestamps and evidence.

Agent Action

Agent work orders, notifications, and reports

Retrieve evidence and invoke WMS, ticketing, and notification systems through MCP while keeping human approval and full audit trails for high-risk actions.

Engineering Challenges

The hard engineering problems behind production deployment

Success depends not only on model accuracy, but also on video-pipeline reliability, site data quality, and whether response workflows form a measurable closed loop.

Multi-stream throughput and latency

Decouple decoding, preprocessing, and inference; apply batching, adaptive sampling, and backpressure so a slow stream cannot stall the entire pipeline.

False positives and evidence

Cross-check continuous trajectories, zone rules, and sensor data, and retain the clip, timestamp, confidence, and trigger rationale for every alert.

Cross-camera tracking and identity

Occlusion and viewpoint changes can break visual re-identification, so critical assets also need barcode, BLE, or RFID identity binding.

VLM cost and peak latency

Use event triggers, key-frame sampling, request aggregation, and small/large model routing to maintain predictable latency during incident bursts.

Privacy, authorization, and retention

Redact at the edge, upload only necessary events, enforce role-based video access, and audit search, export, and Agent actions.

Scene change and model drift

Lighting, rack layouts, uniforms, and camera angles affect accuracy, requiring site-specific evaluation sets, drift monitoring, and continuous tuning.

Interactive experience

Real-time visual operations command center

Switch between misplaced pallet, restricted-zone entry, and dock congestion to see tracking, VLM understanding, event search, and MCP actions in motion.

LiveOps VisionReal-time operations command center

System healthy12 / 12 cameras

CAM 12 · Aisle 7Live

10:24:18

FORKLIFT 02 · 98%

WORKER 03 · 96%

WORKER 07 · 94%

PALLET P-2048

DOCK 04

Privacy-safe synthetic feedNo real customer footage

Misplaced pallet P-2048 detected10:24:21

4 objects38 msRTSP · 30 FPS

00:00

00:12