·10 min read·Blog·Security Operations

What Is Security Telemetry, Why Structure Matters

S
SpecterForce

Threat Research Team

Security telemetry is every signal that enterprise systems generate about security-relevant activity. Authentication events. Firewall decisions. Endpoint process creation. DNS queries. Cloud API calls. Identity changes. Configuration modifications. Network flows.

In aggregate, security telemetry is the most complete record of what is happening, and what has happened, across the enterprise. It is the raw material from which detections are built, investigations are conducted, compliance is demonstrated, and behavioral understanding is maintained.

And in most organizations, it is a mess. Telemetry arrives in dozens of formats from hundreds of sources. Schemas are inconsistent. Identifiers are ambiguous. Context is missing. The data exists, but it is not structured for the work it needs to support.

The difference between raw security telemetry and structured security telemetry is the difference between having evidence and being able to use it.

Defining security telemetry: what it includes and what it doesn't

Security telemetry is a subset of enterprise telemetry, specifically, the signals relevant to security detection, investigation, compliance, and risk management.

It includes authentication and identity signals: login attempts, MFA challenges, token issuance, privilege changes, directory modifications, group membership changes, service account activity. These signals are the primary source for detecting credential compromise, privilege escalation, and insider threats.

It includes network telemetry: firewall logs, proxy logs, DNS queries, NetFlow records, IDS/IPS alerts, and VPN connection logs. Network telemetry provides visibility into lateral movement, data exfiltration, command-and-control communication, and policy violations.

It includes endpoint telemetry: process creation events, file system changes, registry modifications, driver loads, network connections initiated by processes, and behavioral signals from EDR platforms. Endpoint telemetry is where malware execution, persistence mechanisms, and post-compromise activity are visible.

It includes cloud and application telemetry: cloud provider audit logs (CloudTrail, Activity Logs, Audit Logs), SaaS application logs, API call records, configuration changes, and deployment events. As enterprise workloads move to cloud and SaaS, this category grows faster than any other.

It includes audit and compliance signals: access logs for regulated data, policy change records, administrative actions, and governance events. These signals satisfy regulatory retention requirements and support compliance audits.

What security telemetry does not include, at least not in its primary scope, is performance metrics (CPU utilization, memory usage, request latency) and application traces (distributed transaction paths). These belong to the observability domain. The distinction matters because the retention, structure, and consumption patterns for security telemetry differ fundamentally from those for observability data.

Raw telemetry vs. structured telemetry: why the difference matters

Raw telemetry is the data as it is generated by the source system. A firewall log line. A JSON blob from CloudTrail. A syslog message from an authentication server. Raw telemetry is verbose, inconsistent in format, and requires interpretation to extract meaning.

Structured telemetry is the same data after it has been parsed, normalized, enriched, and organized into a consistent schema. Fields are typed. Values are normalized (a timestamp is a timestamp, not a string that looks like a timestamp). Entity identifiers are resolved (an IP address is linked to a device, a device to a user, a user to a role).

The difference matters operationally in three ways.

For detection, structured telemetry enables detection rules that are portable and reliable. A detection rule that looks for "authentication failure followed by success from the same user within five minutes" is straightforward when the data has a consistent schema with typed fields for event_type, user_id, timestamp, and outcome. The same rule against raw telemetry requires per-source parsing logic that breaks when the source format changes.

For investigation, structured telemetry enables queries that return meaningful results without manual data wrangling. "Show me all activity for user jdoe across all sources in the last 30 days" is a single query when telemetry is structured with resolved entity identifiers. Against raw telemetry, it requires searching for "jdoe" across multiple fields in multiple formats, and hoping that no source uses a different identifier for the same person.

For machine consumption, structured telemetry is the prerequisite for AI agents that reason over security data. An agent that receives a structured entity history, a timeline of typed events linked to a resolved identity, can reason over it directly. An agent that receives raw log lines must spend its context window on parsing and interpretation instead of analysis. The practical difference is between an agent that produces useful conclusions and an agent that produces noise.

How telemetry structure affects detection fidelity

Detection fidelity, the ability of a detection system to correctly identify threats while minimizing false positives, is a function of data quality as much as rule quality.

Consider a detection rule designed to identify credential stuffing: "more than N failed authentication attempts from the same source IP against more than M distinct accounts within T minutes." This rule requires three fields to be consistently extracted and normalized: source IP, target account, and authentication outcome. If any of these fields are missing, inconsistently named, or differently formatted across sources, the rule either misses events or produces false positives.

Now consider a more sophisticated detection: "user authenticates from a geographic location inconsistent with their last 90 days of login history." This detection requires not just structured current events but structured historical data, a maintained record of the user's authentication locations over time, with geographic enrichment applied.

As detection sophistication increases, from simple threshold rules to behavioral analytics to machine learning models, the demand for structured, enriched, complete telemetry increases correspondingly. Simple rules can tolerate inconsistent data. Behavioral models cannot.

This is why detection engineering teams often spend more time on data engineering than on rule writing. The rule is the easy part. Getting the right data, in the right structure, with the right enrichment, at the right retention depth, that is the hard part. And it is a data layer problem, not a detection layer problem.

Entity-centric telemetry: organizing events around entities, not time

Traditional telemetry storage is time-centric. Events are indexed by timestamp and stored in chronological order. Queries ask "what happened between time A and time B?" and return a list of events sorted by time.

Entity-centric telemetry inverts this model. Events are organized around the entities they involve, users, devices, services, network segments, cloud resources. Queries ask "what has this entity done?" and return a structured timeline of that entity's activity across all data sources.

The difference is profound for investigation and reasoning.

In a time-centric model, investigating a user requires searching for the user's identifiers across all data sources within a time window, then manually correlating the results. If the user has multiple identifiers (username in Active Directory, email address in the identity provider, employee ID in HR systems), the investigator must search for each one separately and merge the results.

In an entity-centric model, entity resolution has already been performed at ingest. The user's multiple identifiers are linked to a single resolved entity. Querying for the entity returns a unified timeline of activity across all sources, authentication events, endpoint activity, cloud API calls, network connections, and everything else, already correlated and structured.

For human analysts, entity-centric telemetry reduces investigation time from hours to minutes. The manual correlation work, the tedious, error-prone process of finding the same person across different systems, is eliminated.

For AI agents, entity-centric telemetry is the difference between functional and nonfunctional. An agent cannot perform manual correlation. It cannot search across multiple systems with different schemas and merge the results. It needs the pre-correlated, entity-centric view to reason over. Without it, the agent produces fragmented, incomplete, or incorrect analysis.

Entity-centric telemetry is not a feature. It is an architectural requirement for any modern security operation, human or machine.

From security telemetry to institutional memory: the architecture

The progression from raw security telemetry to institutional memory follows a clear architectural path.

Stage 1: Collection. Telemetry is captured from all sources, endpoints, network, identity, cloud, and applications. The collection layer handles format normalization at the transport level and ensures no sources are excluded due to volume constraints. This stage addresses completeness: every relevant signal enters the system.

Stage 2: Enrichment. Captured telemetry is parsed into structured records. Metadata is extracted, event types, entity identifiers, timestamps, outcomes, and attributes are normalized into typed fields. Entity resolution links each event to the entities it involves. External enrichment adds context, threat intelligence, asset criticality, geographic data, organizational hierarchy. This stage addresses structure: every event becomes a machine-consumable record linked to resolved entities.

Stage 3: Retention. Enriched, structured telemetry is stored in hot, searchable storage. Full fidelity, no sampling, no aggregation, no tiering to cold archive. The retention horizon spans months to years, ensuring that behavioral baselines, retrospective analysis, and compliance audits have access to complete historical data. This stage addresses accessibility: every retained event is queryable in seconds regardless of age.

Stage 4: Consumption. Structured, retained telemetry is exposed through interfaces optimized for both human analysts and AI agents. Entity history queries return structured timelines. Cross-domain correlations are pre-computed. The data is ready for detection, investigation, hunting, compliance, and autonomous reasoning without additional transformation.

When all four stages operate continuously, the result is institutional memory, a maintained, structured, complete record of enterprise security activity that compounds in value over time. Each day of retained data adds to behavioral baselines, enriches entity histories, and provides additional context for future detections and investigations.

Security telemetry and agentic AI: what machines need to reason correctly

The emergence of AI agents in security operations creates a new and exacting set of requirements for security telemetry.

Agents reason over data. The quality of the data determines the quality of the reasoning. This is not an abstract principle, it has specific, measurable consequences.

An agent investigating a suspicious login needs the user's authentication history. If that history is structured, entity-resolved, and spans six months of hot data, the agent can determine in seconds whether this login pattern is consistent with the user's normal behavior. If the history is fragmented across three systems, partially in cold archive, and requires manual correlation, the agent either fails to gather sufficient context or produces a shallow analysis that misses the pattern.

An agent assessing a cloud configuration change needs the change history for the affected resource. If that history is maintained as an entity timeline with enrichment (who made each change, what was changed, what was the business context), the agent can evaluate the change in context. If the change history is a series of raw CloudTrail events with no entity resolution, the agent must spend its reasoning capacity on data interpretation instead of risk assessment.

The requirements are clear. Agents need telemetry that is complete (all sources, unsampled), structured (typed fields, normalized values, consistent schemas), entity-resolved (events linked to resolved identities and resources), enriched (context embedded at ingest), and accessible (hot, queryable in seconds, spanning months of history).

Bloo provides exactly this. It captures all security telemetry, applies metadata extraction and entity resolution at ingest, retains the structured result in hot searchable storage at predictable cost, and serves it through machine-consumable interfaces. For AI agents, Bloo is the memory layer, the infrastructure that transforms raw security telemetry into the institutional memory that autonomous reasoning requires.

Security telemetry is the foundation. Structure is what makes it useful. And the architecture that delivers structured, retained, machine-consumable telemetry is what makes the modern SOC, human and machine, possible.

Related articles

The Death of Static Lineage: Fusing Co-Occurrence Math with Call Stack Anomalies

The security industry has historically relied on monitoring parent-child process trees to identify malicious execution. If Microsoft Word spawns a command shell, a static rule triggers. However, advanced adversaries - particularly those operating in high-stakes financial and telecommunications sectors - are fully aware of these static registries.

Fileless Malware and Process-Based Attacks Analysis

Fileless malware is one of the most dangerous and evasive attack techniques. Unlike traditional malware, it leaves no files on disk; instead, it hides inside the system's own trusted processes and tools, making it nearly invisible to conventional security software. In this article, we break down how fileless and process-based attacks work, how attackers use built-in Windows utilities like PowerShell and WMI to execute malicious code entirely in memory, and what defenders need to do to detect and stop them before it's too late.

We use cookies to provide essential site functionality and, with your consent, to analyze site usage and enhance your experience. View our Privacy Policy