Examples
The process is up, but the URL stops answering. The process is gone. A port stops answering. Logs fill the disk. Nine common alerts on web and app servers, and what Virca does with each.
Each example uses ActionBooks in the product today. An ActionBook is a fix written down as one file your team can read and approve. The Virca Worker is a process on your server. It watches the server and runs the fixes your team approves.
Known problems
Your team approves each fix. The Worker checks that the alert is real, gathers the facts, runs the fix and re-checks its target. With approval mode on, every run an alert starts waits for approval in Virca.
Every example, the same frame
Known problems · 01
When your team links this alert to a restart. Until then, the Worker looks first, then stops and tells your team.
214/url_down
If no one acts: Users of that service can keep getting errors.
What the record shows
If /healthz already answers http_status: 200 Already answering again. Virca changed nothing.
Known problems · 02
With an ActionBook your team approved, for nginx on web-01.
219/process_down
If no one acts: The service can stay down until someone restarts it.
What the record shows
If nginx is already back is_service_active: true Already running again. Virca changed nothing.
Known problems · 03
With an ActionBook your team approved, started by your team, for /data/logs.
201/host_filesystem_high
If no one acts: Once the disk is full, writes fail and the app can stop.
What the record shows
If the growth is not in /data/logs No approved fix fits. Virca stops and tells your team.
Known problems · 04
With an ActionBook your team approved, for the process using the most CPU.
201/host_cpu_high
If no one acts: Requests to that server can slow down or time out.
What the record shows
init, systemd and sshd are never touched. SIGTERM only. For your main app, link the alert to a service restart instead.
Known problems · 05
With an ActionBook your team approved, for the process using the most memory.
201/host_memory_high
If no one acts: To free memory, Linux may stop a process you need.
What the record shows
If that process is your main app, link the alert to a restart of its service instead.
Looked into first
These alerts have no set fix. The Worker looks into the server and saves the findings to the record. Your team decides the next step.
Not every event ends in an action, and that is the point.
Looked into first · 06
Ports close for different reasons on each server. No single fix fits.
217/port_probe_down
If no one acts:Services that use that port can't connect to it.
What the record shows
Your team takes over from the record.
Looked into first
If no one acts: The server can slow down as it moves memory to and from disk.
Saved to the record: server memory and the processes using the most memory.
If no one acts: Reads and writes queue up, and apps on the server slow down.
Saved to the record: a snapshot of CPU, memory, disk and load.
If no one acts: The network can fill up, and normal requests slow down or fail.
Saved to the record: connection counts, listening ports and waiting connections.
