Triage DevOps incidents from Slack with GPT-4.1, Prometheus, Jira and Terraform

Description

Quick overview
This workflow listens for incident reports in Slack, uses an OpenAI agent with tool-calling to query Prometheus, Loki, and a vector-store RAG knowledge base, logs the analysis to PostgreSQL, and—after explicit human approval—creates a Jira or GitHub ticket, runs Terraform remediation, and verifies recovery via Prometheus and Grafana.

How it works
Triggers when a new message is posted in the configured Slack incident-response channel.
Normalizes the incident details (text, channel, thread timestamp, reporter) and creates a session ID for thread-based memory.
Uses an OpenAI GPT-4.1 agent with session memory to investigate by calling Prometheus (metrics), Loki (logs), and a vector-store RAG search over past incidents as needed, returning a structured RCA in JSON.
Parses the agent’s JSON output and stores the incident analysis (root cause, confidence, evidence, remediation, risk level, and service) in a PostgreSQL incidents table.
If no concrete remediation is proposed, posts an investigation update back to Slack and stops.
If a remediation is proposed, posts an approval request to Slack and waits for a reviewer decision via a resume webhook.
When approved, creates an incident ticket in Jira or GitHub Issues, triggers a Terraform Cloud run to apply the remediation, re-queries Prometheus, adds a Grafana annotation, and posts a resolution summary to the original Slack thread.
When rejected, posts a rejection notice to Slack and stops.

Setup
Connect Slack credentials for the Slack Trigger and messaging nodes, and set the target channel (default: #incident-response).
Add an OpenAI API credential for the GPT-4.1 chat model and configure the vector store used by the RAG search tool.
Update the Prometheus, Loki, and Grafana base URLs (and add any required auth headers/tokens) in the HTTP request nodes.
Add PostgreSQL credentials and create a public incidents table with columns matching the insert mapping used by the workflow.
Configure either Jira or GitHub credentials (project/issue type for Jira or owner/repo for GitHub) and set ticketingSystem to jira or github in the configuration step.
Add Terraform Cloud API authentication, replace REPLACE_WITH_WORKSPACE_ID, and configure Slack interactive Approve/Reject actions to call the workflow’s wait/resume webhook with a reviewerApproved boolean.