New ask Hacker News story: Terraform requires a DAG. AWS allows cycles. Here's how I map the difference.
Terraform requires a DAG. AWS allows cycles. Here's how I map the difference.
3 by davidlu1001 | 0 comments on Hacker News.
Error: Cycle: aws_security_group.app -> aws_security_group.db -> aws_security_group.app If you've ever seen this error while importing AWS infrastructure to Terraform, you know the pain. Terraform's core engine relies on a Directed Acyclic Graph (DAG). It needs to know: "Create A first, then B." But AWS is eventually consistent and happily allows cycles. The Deadlock The most common culprit is Security Groups. Imagine two microservices: - SG-App allows outbound traffic to SG-DB - SG-DB allows inbound traffic from SG-App If you write this with inline rules (which is what terraform import defaults to), you create a cycle: resource "aws_security_group" "app" { egress { security_groups = [aws_security_group.db.id] } } resource "aws_security_group" "db" { ingress { security_groups = [aws_security_group.app.id] } } Terraform cannot apply this. It can't create app without db's ID, and vice versa. The Graph Theory View When building an infrastructure reverse-engineering tool, I realized I couldn't just dump API responses to HCL. We model AWS as a graph: Nodes are Resources, Edges are Dependencies. In a healthy config, dependencies are a DAG: [VPC] --> [Subnet] --> [EC2] But Security Groups often form cycles: ┌──────────────┐ ▼ │ [SG-App] [SG-DB] │ ▲ └──────────────┘ Finding the Knots To solve this for thousands of resources, we use Tarjan's algorithm to find Strongly Connected Components (SCCs). It identifies "knots" — clusters of nodes that are circularly dependent — and flags them for surgery. In our testing, a typical enterprise AWS account with 500+ SGs contains 3-7 of these clusters. The Fix: "Shell & Fill" We use a strategy to break the cycle: 1. Create Empty Shells: Generate SGs with no rules. Terraform creates these instantly. 2. Fill with Rules: Extract rules into separate aws_security_group_rule resources that reference the shells. Step 1: Create Shells [SG-App (empty)] [SG-DB (empty)] Step 2: Create Rules ▲ ▲ │ │ [Rule: egress->DB] [Rule: ingress<-App] The graph is now acyclic. "Why not just always use separate rules?" Fair question. The problem is: 1. terraform import often generates inline rules. 2. Many existing codebases prefer inline rules for readability. 3. The AWS API presents the "logical" view (rules bundled inside). The tool needs to detect cycles and surgically convert only the problematic ones. Why terraform import isn't enough Standard import reads state as-is. It doesn't build a global dependency graph or perform topological sorting before generating code. It places the burden of refactoring on the human. For brownfield migrations with 2,000+ resources, that's not feasible. --- I've implemented this graph engine in a tool called RepliMap. I've open-sourced the documentation and IAM policies needed to run read-only scans safely. If you're interested in edge cases like this (or the root_block_device trap), the repo is here: https://github.com/RepliMap/replimap-community Happy to answer questions.
3 by davidlu1001 | 0 comments on Hacker News.
Error: Cycle: aws_security_group.app -> aws_security_group.db -> aws_security_group.app If you've ever seen this error while importing AWS infrastructure to Terraform, you know the pain. Terraform's core engine relies on a Directed Acyclic Graph (DAG). It needs to know: "Create A first, then B." But AWS is eventually consistent and happily allows cycles. The Deadlock The most common culprit is Security Groups. Imagine two microservices: - SG-App allows outbound traffic to SG-DB - SG-DB allows inbound traffic from SG-App If you write this with inline rules (which is what terraform import defaults to), you create a cycle: resource "aws_security_group" "app" { egress { security_groups = [aws_security_group.db.id] } } resource "aws_security_group" "db" { ingress { security_groups = [aws_security_group.app.id] } } Terraform cannot apply this. It can't create app without db's ID, and vice versa. The Graph Theory View When building an infrastructure reverse-engineering tool, I realized I couldn't just dump API responses to HCL. We model AWS as a graph: Nodes are Resources, Edges are Dependencies. In a healthy config, dependencies are a DAG: [VPC] --> [Subnet] --> [EC2] But Security Groups often form cycles: ┌──────────────┐ ▼ │ [SG-App] [SG-DB] │ ▲ └──────────────┘ Finding the Knots To solve this for thousands of resources, we use Tarjan's algorithm to find Strongly Connected Components (SCCs). It identifies "knots" — clusters of nodes that are circularly dependent — and flags them for surgery. In our testing, a typical enterprise AWS account with 500+ SGs contains 3-7 of these clusters. The Fix: "Shell & Fill" We use a strategy to break the cycle: 1. Create Empty Shells: Generate SGs with no rules. Terraform creates these instantly. 2. Fill with Rules: Extract rules into separate aws_security_group_rule resources that reference the shells. Step 1: Create Shells [SG-App (empty)] [SG-DB (empty)] Step 2: Create Rules ▲ ▲ │ │ [Rule: egress->DB] [Rule: ingress<-App] The graph is now acyclic. "Why not just always use separate rules?" Fair question. The problem is: 1. terraform import often generates inline rules. 2. Many existing codebases prefer inline rules for readability. 3. The AWS API presents the "logical" view (rules bundled inside). The tool needs to detect cycles and surgically convert only the problematic ones. Why terraform import isn't enough Standard import reads state as-is. It doesn't build a global dependency graph or perform topological sorting before generating code. It places the burden of refactoring on the human. For brownfield migrations with 2,000+ resources, that's not feasible. --- I've implemented this graph engine in a tool called RepliMap. I've open-sourced the documentation and IAM policies needed to run read-only scans safely. If you're interested in edge cases like this (or the root_block_device trap), the repo is here: https://github.com/RepliMap/replimap-community Happy to answer questions.
Comments
Post a Comment