COMPEL Glossary / latency
Latency
Latency is the time delay between sending a request to an AI system and receiving a response, typically measured in milliseconds.
What this means in practice
Low latency is critical for real-time applications: fraud detection systems must evaluate transactions in under 100 milliseconds to avoid blocking legitimate purchases, conversational AI must respond within 1-2 seconds to feel natural, and autonomous systems must react in real time to environmental changes. Latency is affected by model complexity, infrastructure performance, network distance, data retrieval time, and request queuing. In the COMPEL Technology pillar, latency requirements inform platform architecture decisions, deployment location (cloud vs. edge), and model optimization strategies. SLAs for AI systems must specify maximum acceptable latency, and monitoring systems must track latency in production to detect degradation before it impacts user experience.
Why it matters
Latency requirements directly determine AI architecture decisions and user experience. Fraud detection needs sub-100ms response times, conversational AI needs 1-2 seconds, and batch analytics can tolerate minutes. Organizations that do not specify latency requirements upfront build systems that may be technically accurate but operationally unusable because response times exceed what the business context demands.
How COMPEL uses it
Latency requirements inform platform architecture decisions during the Model stage, including deployment location (cloud vs. edge) and model optimization strategies. SLAs for AI systems specify maximum acceptable latency. The Produce stage implements monitoring that tracks latency in production. The Evaluate stage measures latency performance against SLAs, and latency degradation triggers investigation through the operational resilience framework.
Related Terms
Other glossary terms mentioned in this entry's definition and context.