Root-Causing a Misbehaving Server: Two Servers, One Method

August 21, 2026 · InfraScout Team

A server has been misbehaving for weeks, and the team already has a favorite suspect. Hand the problem to InfraScout in a chat, and you end up with a root cause that has been checked against the host, a fix you approved step by step, and an answer to the next question: is anything else in the fleet affected?

The two scenarios below are composites of a common pattern: a Windows member server stuck in a memory loop, and a Linux SIEM server that first ran out of memory and later out of CPU.

The question you ask

You don't need a playbook to start. Describe the symptom, name your suspicion, and say how much the AI may change.

One of our Windows member servers has had memory problems for about two weeks.
We suspect the security agents installed on it. Find the root cause.
Investigate read-only first, and ask me before you change anything.

From there the AI resolves the host, pulls its inventory summary and any open insights, and opens a session so every command it runs is recorded against that server.

Test the hypothesis before you chase it

In the first scenario, the AI first reads live memory, commit charge, and per-process usage. The live values didn't match the inventory: the server had been rebooted recently and had a different amount of RAM than recorded. The AI worked from the live data.

Next it checked the suspects. One of the two security agents wasn't installed at all, so half the hypothesis was wrong before anyone spent an afternoon on it.

Following the evidence to an install loop

You steer where you know more than the host shows. Here, the admin pointed out that the missing agent is deployed through Group Policy. The AI checked the applied GPOs and found one that pushes a scheduled task, and the task runs an installer. Task Scheduler history showed the task firing again and again: once when it was registered, then at every reboot. The installer's log, several megabytes long, ended in an out-of-memory exception at start-up, and the AI watched a PowerShell process grow in real time.

When the admin asked why PowerShell needed that much memory, the AI decoded the error. It wasn't a shortage of RAM. The commit limit had been reached because the paging file was too small, and the paging file lived on a tiny dedicated volume. It also traced the orphaned child processes each failed attempt left behind.

Recognizing a commit-limit failure

0x800705AF is Win32 error 1455, ERROR_COMMITMENT_LIMIT: the paging file is too small. In .NET and PowerShell it often shows up as System.OutOfMemoryException even when free RAM looks fine. Event 2004 from Resource-Exhaustion-Detector in the System log names the top memory consumers at the moment the limit was hit.

Fixes you approve, verified on the host

Everything so far was read-only. For the fix, the AI proposed specific, scoped changes and waited for approval of each. After a read-only check of the disk, it proposed exact paging-file sizes. The admin approved and the AI applied them. The admin rebooted, and the AI confirmed the commit limit had grown. Then the admin grew the virtual disk in the hypervisor and asked which Windows disk to extend. The AI didn't guess. It mapped the hypervisor's disk order to the Windows disk, partition, and drive letter, and extended the partition only after approval.

The Group Policy fix showed why you check the host rather than taking anyone's word for it. The admin changed the GPO, and the AI checked the server: the change hadn't taken effect. Reading the policy definition itself, rather than the "applied" summary, showed why. The Group Policy Preferences item used the Create action, which never modifies a task that already exists. Switching it to Replace fixed that, but the edit broke the policy's security filtering. The AI spotted that in the policy results, and the admin corrected it.

Technical notes from this scenario

A Group Policy Preferences scheduled task set to Create never updates an existing task, so use Replace or Update. If a GPO edit doesn't seem to apply, run gpresult /r /scope:computer /v and look for "Denied (Security)". Background refresh takes 90 to 120 minutes, and gpupdate /force applies changes immediately. In the hypervisor, "Hard Disk N" follows SCSI target order, not size, so map disks by SCSI target ID. Growing a virtual disk doesn't grow the partition inside it.

From one host to the fleet

Once you know the failure signature, the next question is scope. With a virtualization MCP server and a SIEM MCP server connected alongside InfraScout, the AI confirmed the VM's real RAM allocation and then searched the fleet for the same crash. One other memory-constrained server showed it. The domain controllers did not. The whole investigation took about four hours on two systems, plus one query across the rest of the fleet. The findings were saved as insights, along with a change log and host notes for next time.

The second server: a SIEM under load

The second scenario ran on Linux. A Splunk server ran out of memory and was OOM-killed. An emergency restart hadn't helped, and the admin wanted to know what to switch off.

On first look, the AI found the systemd unit marked dead after the OOM kill, while Splunk processes kept running without supervision. It also found one very large index, and saved both as insights. When the admin asked whether to just restart, the AI advised against it: the cause was unknown, free memory was low, and nothing was monitoring the service. The admin chose to prepare and analyze first. The AI dropped caches, set a Java heap limit, and found that a single index held a large share of the data. With approval, it removed old buckets from that index and capped an internal index. One of its own config edits broke start-up once. It diagnosed the problem and repaired it in the same session.

Weeks later the server sat at 100% CPU. The AI read its saved memory notes before running anything. Live load was about four times the core count, and three causes compounded each other: a generous search-concurrency setting, a monitoring app with many frequent scheduled searches, and data-model acceleration jobs.

The AI proposed disabling an app that looked like a demo. The admin questioned that, and an audit showed the app contained production detections. The AI withdrew the recommendation and dismissed its own insight. With approval, it backed up the configuration file, halved the concurrency settings, restarted Splunk, and verified the effective values with btool. Load dropped by about half. It then categorized the scheduled searches, left the core ones alone, and moved a subset to hourly.

Within the hour, the service was back, disk usage had dropped substantially, CPU load had roughly halved, and the remaining acceleration load was recorded for follow-up. The sessions went to review.

Technical notes from this scenario

max_searches_per_cpu and the related settings in limits.conf control search concurrency. Check the effective values with btool rather than trusting a single file. Disk usage is controlled by bucket retention and index size caps. Under heavy load, a graceful Splunk shutdown can take longer than the systemd stop timeout, which is one way a unit ends up dead while its processes keep running.

The method, and where you stay in control

Both investigations followed the same four habits, and you can ask for them explicitly in any troubleshooting chat:

• Test the hypothesis first. The suspect may not even be installed.
• Verify against the host, not against inventory, a summary, or someone's recollection.
• Read the policy or config definition directly instead of the "applied" view.
• Once you have a failure signature, search the fleet for it.

Investigation used read-only tools. Changes happened only after you approved a specific, scoped action, with a backup or preview first and verification afterward. The AI turned down a risky restart, refused to guess a disk mapping, accepted pushback and reversed its own recommendation, and never guessed or reused credentials. Tool annotations and roles explain how read-only and state-changing tools are kept apart.

Next steps

To try this on your own misbehaving server, check what your agents can reach under agent capabilities, then learn how findings carry over between sessions in Insights and Memory. If your hypervisor or SIEM offers an MCP server, connect it so the AI can take the investigation from one host to the whole fleet.