AI Threat Hunting: Why Normalizing Data Before the Hunt Breaks Your Evidence

When normalization shreds your evidence, the hunt stalls before it starts.

Key Takeaways

• Normalizing security data before the hunt strips or flattens the specific fields threat hunters need, such as full command-line arguments and vendor-specific metadata.
• AI threat hunting stalls when it relies on normalized data because the evidence required to validate a hypothesis may have been discarded before the hunt even starts.
• Querying security platforms in their native formats (like SPL for Splunk or KQL for Sentinel) preserves the evidence trail and allows for deeper, more accurate hunts.
• For regulated environments, model sovereignty is an operational requirement. Hunt queries reveal attack surfaces, so the AI and model must run inside the customer's boundary.
• An effective AI-assisted hunt produces an auditable and repeatable evidence record in the customer's own case management system, not a vendor's cloud tenant.

A threat hunter receives a CISA advisory for a new living-off-the-land technique. The hunt requires searching 90 days of data across a Splunk instance, Microsoft Sentinel, and a data lake. The team's AI threat hunting tool, however, relies on a normalized schema that has stripped the parent process command-line arguments and specific registry key paths. This is the exact detail needed to distinguish malicious LOLBin use from legitimate admin activity. The hunt stalls. AI threat hunting breaks when you normalize security data before the hunt, because normalization discards the field-level context that separates a true positive from noise.

The AI performed as designed. The data architecture it was given discarded the evidence before the hunt started.  This guide explains why native-format queries matter for threat hunting, how AI changes the economics of the hunt hypothesis backlog, and what a real CVE hunt looks like across fragmented data. It also covers why model sovereignty is a critical hunting requirement in regulated and air-gapped environments.

Why Schema Normalization Destroys the Evidence Trail Before the Hunt Starts

Threat hunting depends on a level of field-level specificity that most detection rules do not require. A detection rule for Sysmon Event ID 1 can fire on process creation generically. A hunt for LOLBin abuse needs the exact command-line string and the full parent-child process chain. A hunt for C2 beaconing needs timestamps precise enough to measure interval and jitter. Normalization pipelines often flatten or discard both. This is an architectural decision that determines whether AI-assisted hunts produce usable evidence or a false sense of coverage. The difference between a detection rule and a hunt hypothesis lies in their directionality. A detection waits for data to match a known pattern, while a hypothesis asks whether a suspected pattern is present in the data.

What Normalization Strips From Hunt-Quality Evidence

Normalization pipelines built for dashboarding and detection optimize for aggregation, not investigation. Consider a hunt team responding to an advisory for a VPN appliance CVE. The SIEM had normalized the appliance's authentication logs into a generic logon category, stripping the firmware-version field. That field existed in the raw syslog, but it did not survive the normalization pipeline. The team spent hours pulling raw logs to reconstruct what the SIEM had already discarded.

This process consistently removes three types of hunt-quality evidence. First, it truncates or hashes command-line arguments to fit a common schema, losing the URL that distinguishes a malicious certutil.exe download from legitimate use. Second, it drops vendor-specific fields that have no slot in the common schema, like the firmware version in the VPN example above. Third, it can reduce timestamp precision from milliseconds to seconds, which breaks analysis of beacon jitter. The data your AI hunts against may have already lost the evidence you need.

Three categories of hunt-quality evidence that normalization pipelines discard.

Read more: our patented approach to analyzing non-normalized security data

How Native-Format Queries Preserve Field-Level Context

An alternative approach is federated search that queries each data source in its native format. Instead of forcing all data into one schema, the AI system queries Splunk in SPL, Sentinel in KQL, and a data lake using its own query syntax. This architectural choice preserves every field the source originally recorded. In the certutil.exe example, a native-format query returns the full command line, the parent process, and the download URL. A query against normalized data returns a generic process event.

This approach shifts complexity from the data pipeline to the platform's reasoning layer. The AI must understand multiple query languages and how to correlate findings across different data structures. Crogl's architecture makes this tradeoff deliberately. It queries sources where they live because recovering fields after a normalization pipeline has discarded them is slow, expensive, or impossible. The goal is to give the hunter the complete evidence bag.

The Hunt Hypothesis Backlog and How AI Changes the Economics

Every established SOC maintains a hunt hypothesis backlog, whether in a formal spreadsheet or an analyst's notes. Experienced hunters generate hypotheses faster than the team can execute them. A single CISA advisory might generate dozens of hunt leads, but a team can only execute a few per hunt sprint. Each one requires writing queries across multiple platforms, waiting for results, pivoting on new indicators, and documenting every finding.

Independent research by Ponemon Institute, commissioned by Crogl, found that organizations face an average of 4,330 alerts per day but only investigate 37% of them . When alert triage consumes available analyst hours, the hypothesis backlog grows. At a major US electric utility, that pressure showed up in advisory work. With Crogl, the team now turns around CRISP advisories in under an hour, down from more than 24.

AI changes this equation by executing the cross-platform queries and evidence gathering that consume most of the hunt sprint, while preserving the hunter's judgment. The hunter still forms the hypothesis and evaluates the evidence. The AI handles the manual query work across platforms. Without federated search, hunts stall waiting for access. This makes AI threat hunting an economics problem above all else.

What an AI Threat Hunt Looks like for a Specific CVE

Imagine a CISA advisory is released for a CVE affecting a widely deployed network appliance. The hunt lead needs to determine exposure, check for exploitation, and document the evidence. The environment is typical for a large enterprise: Splunk for network logs, CrowdStrike Falcon for endpoint telemetry, and a ServiceNow instance for asset and patch management records. This is what an AI-assisted hunt looks like in practice.

Querying Three Platforms Without Memorizing Three Query Languages

The first phase is the search. The hunter states the hypothesis: are we exposed to this CVE, and is there evidence of exploitation? The AI translates this into platform-native queries. It queries Splunk in SPL for network connections to known C2 infrastructure from the advisory. It queries the CrowdStrike API for process execution on hosts running the vulnerable appliance's management agent. It also queries ServiceNow for asset records identifying hosts with the vulnerable firmware version.

The hunter can ask Crogl to show the queries before they run. Running autonomously, Crogl executes them and shows the evidence chain: every query, the source it ran against, and what came back. This is a key distinction. A system that shows its work allows the hunter to validate the hunt logic. A system that hides its queries forces the hunter to trust a black box. The analyst remains in control of the hunt, directing the AI to gather evidence from across the toolchain.

Read more: watch a video of Crogl investigating a real threat

The Analyst Decision and the Evidence Record

The second phase is analysis and judgment. The AI returns its findings. Three hosts have the vulnerable firmware. Two of them show network connections to a suspicious IP address from the advisory. However, no matching malicious process execution is found on the endpoints. The hunter evaluates the evidence. The connections could be benign. The IP sits on shared CDN infrastructure, so traffic to it may belong to another tenant entirely.

The hunter directs the AI to perform a diamond model pivot on the IP address, querying threat intelligence feeds and performing frequency analysis on internal DNS logs. Based on the consolidated evidence, the hunter makes a call. The two hosts require immediate patching and monitoring, but the evidence does not confirm active exploitation. The hunter documents this decision, the supporting evidence, the sources queried, and the reasoning in a ServiceNow ticket. This becomes the evidence bag for the compliance team, recording what was searched, what was found, and what was ruled out.

How an AI-assisted hunt flows from hypothesis to documented evidence record.

Model Sovereignty as a Threat Hunting Requirement

Threat hunting queries expose more sensitive context than alert triage does. A triage workflow processes an alert that has already fired. A hunt query is exploratory. It searches for lateral movement patterns, queries DNS logs for C2 beaconing intervals, and correlates user authentication events across directories. These queries reveal an organization's network topology, naming conventions, and defensive gaps. If an AI system sends those queries or their results to a public cloud-hosted model, the model's provider now has a map of the organization's attack surface.

For defense, intelligence, critical infrastructure, and financial services, this is not an acceptable risk. Model sovereignty has three dimensions for hunting. First, the model runs inside the customer's boundary, so hunt queries and sensitive data never leave the environment. Second, the customer selects and controls the model, allowing them to evaluate its behavior and update it on their schedule. Third, the system must operate in fully air-gapped environments where cloud connectivity is not an option. A U.S. Department of War deployment of Crogl runs fully air-gapped and classified across three SIEMs, two SOARs, and 100TB of data, investigating more than 1,000 alerts per day. Sovereignty is an operational requirement for hunting in sensitive environments.

Model sovereignty for threat hunting spans three operational dimensions.

Building an Auditable Evidence Trail for AI-Assisted Hunts

An AI-assisted hunt has to hold up when someone asks how a finding was reached. Crogl reasons inside a governed harness, grounded in a knowledge graph built from the customer’s own data. Every query, the source it ran against, the result, and the hunter’s decision are logged. That makes each hunt auditable, repeatable, and inspectable.

Consider a compliance scenario where an auditor asks to see how a hunt finding was reached six months ago. If the AI system logged its work in the customer's own case system, like ServiceNow or Jira, the SOC can produce the full, inspectable chain of evidence. If the logs are in the vendor's cloud, the SOC depends on the vendor's retention policies. For regulated organizations, only the first option is acceptable. Audit readiness is an architecture decision  that must be made before the first hunt runs.

How Crogl Hunts Across Fragmented Data Without Normalizing It

Each of these problems is architectural. Normalization destroys evidence before the hunt starts, the backlog outgrows the team, hunt queries expose sensitive context, and auditability depends on where the AI logs its work.

Crogl was built around those four constraints. Federated search queries each source in its native format, so the AI hunts across Splunk, Sentinel, CrowdStrike, data lakes, and OT platforms without a normalization pipeline. Crogl builds a knowledge graph of your data and its cross-tool relationships when you add a connector, then keeps it current on its own as datasets, fields, and data change. The model runs inside the customer's boundary, on-prem or fully air-gapped, so hunt queries and sensitive data never leave the environment. Every query, result, and reasoning step is logged in the customer's own case system for a complete audit trail.

This architecture changes the economics of the hunt. When the AI handles the cross-platform queries and evidence gathering, the hunter can execute more hypotheses per sprint. Crogl's pricing has no per-alert, per-investigation, or per-user fees, so expanding hunt coverage does not increase cost. You can run your first hunt against your own SIEM in minutes.

Download Crogl  to hunt across your SIEM, EDR, and data lake in their native formats without building a normalization pipeline first.

The Architectural Choice of Modern Threat Hunting

The most important shift in thinking about AI threat hunting centers on an architectural decision about where data lives, who controls the model, and whether the evidence trail survives the hunt. The teams that hunt effectively across fragmented data are the ones that stopped normalizing before the hunt and started querying sources where they live. The teams that trust their AI-assisted findings are the ones that can hand an auditor the full, inspectable chain from hypothesis to decision. As threat advisories accelerate, the hunt hypothesis backlog will continue to grow. The question is whether your architecture lets your team execute more of those hypotheses without losing the evidence along the way.