# Aviz Networks details AI-Parser for LLM token metering

URL: https://decisioninsights.ai/aviz-networks-details-ai-parser-for-llm-token-metering/

Aviz Networks describes AI-Parser, a passive meter that records LLM calls from mirrored network traffic and attributes tokens and latency to tenants and applications. It targets operators that provide AI services and must track their consumption.Why visibility becomes the provider's responsibilityThe blog says NeoClouds, sovereign AI programs and enterprise AI factories share a common shift once they provide AI rather than consume it. Commercial APIs return token counts, model identity and latency for each call, while self-hosted engines such as vLLM report usage in aggregate by model only. GPU, server and network metrics show utilization but not which tenant, model or application is consuming capacity.How AI-Parser captures and measures callsAccording to the blog, AI-Parser operates as a passive tap on a SPAN copy of traffic between applications and inference engines, not as an inline proxy. The process rebuilds each request and streamed response, and a lightweight agent obtains the session key from the inference engine so encrypted calls can be read. Tokens, latency, model and application are recorded for each call and streamed to tools such as Elastic, Splunk, Prometheus or Grafana.AI-Parser runs on x86 or the NVIDIA BlueField DPU in front of the GPU server, with no host footprint. The company states that nothing sits in the live path, so the product adds no latency and creates no new point of failure.Metrics recorded per callEach LLM call produces one record attributed to the tenant and application that made it. The blog lists input, output and total tokens read from the server's usage report, along with time-to-first-token, end-to-end latency, inter-token latency and tokens per second as the client experiences them. Completed and aborted calls, wasted tokens and truncated responses are also tracked.Aggregating records by tenant yields token totals per tenant, per application and per model, which the blog describes as usable for chargeback. Visibility also extends inside a single tenant, where usage breaks down by application, inference engine and LLM.Prompt visibility and data governanceAI-Parser records a fingerprint of each prompt by default and captures full prompt text only when that option is enabled. The blog identifies use cases including data loss prevention, governance and audit, deduplication to identify caching opportunities, and prompt analysis. Fingerprints allow repeated prompts to be identified without storing the text users typed.Scope for sovereign and enterprise deploymentsThe blog states that AI-Parser runs on infrastructure controlled by the operator and sends records only to tools the customer selects. It is positioned alongside Aviz ONES, Aviz Service Gateway and Aviz AI-NOC, which handle network services, multi-tenant operation and root-cause correlation. The stated goal is to make the GPU, inference and workload stacks easier to operate together rather than to replace them.This Blog Summary is a fact-based summary of the vendor blog. For NeoClouds, sovereign AI programs and enterprises building their own AI factories, the article presents per-call attribution of tokens, latency and prompt signals from network traffic as a way to meter and govern AI service consumption.
