← All Posts
backendawsqueuegolang

Why Your System Will Break Without a Queue

VOL. 2026 · NO. 088·By Start Frist Byte·March 29, 2026·8 min read

300 simultaneous requests, a held database connection, and a cascade of 500 errors. Here's why synchronous architectures fail under load — and how a message queue fixes it.

Why Your System Will Break Without a Queue

Why Your System Will Break Without a Queue

You've built a hotel booking platform. The API is clean, the database is solid, emails go out reliably. Everything works perfectly — in staging, with five concurrent users.

Then launch day arrives. Three hundred people hit "Book Now" at the same time.

Within seconds: DB connection pool exhausted. 500 errors cascade. Angry users refresh, retrying and making it worse. Your on-call phone starts ringing.

The culprit isn't your code quality. It's your architecture. You're forcing everything to happen synchronously when reality doesn't require that.

The Synchronous Trap

Here's what happens in a typical booking flow without a queue:

  1. User submits a booking form
  2. API receives the request
  3. API writes to the database
  4. API calls the email service to send a confirmation
  5. Email service responds
  6. API returns a success response to the user

Steps 3 and 4 happen while the user is waiting. The HTTP connection stays open. A database transaction is held. If the email service is slow or the DB is under load, everything backs up.

Under normal traffic, this is invisible. Under a spike, it's catastrophic.

"You're holding a database connection hostage while waiting for an email to send. That's the bottleneck."

300 simultaneous requests × one held DB connection each = pool exhausted → new requests fail immediately → users see errors → they retry → it gets worse.

The Fix: Decouple with a Queue

The insight is simple: not everything needs to happen before you respond.

The user needs to know their booking was received. They don't need the confirmation email to be sent before you return a response. Those are two different things.

Here's the redesigned flow:

  1. User submits a booking form
  2. API receives the request
  3. API writes to the database
  4. API publishes a message to the queue
  5. API returns 202 Accepted immediately
  6. A worker picks up the message asynchronously
  7. Worker sends the confirmation email

The API is now fast and stateless. The queue absorbs the spike. Workers process at their own pace. No connection held hostage, no cascading failures.

AWS SQS in Practice

For this pattern, AWS SQS (Simple Queue Service) is a natural fit. It's managed, durable, and scales without any configuration on your end.

The architecture has three components:

ComponentResponsibility
Producer (API)Receives booking, writes to DB, enqueues message, returns response
Queue (SQS)Buffers work, decouples producer from consumer
Consumer (Worker)Reads messages, sends emails, deletes processed messages

The Message

When a booking is confirmed, the API sends a JSON message to SQS:

{
  "bookingId": "BK-2025-001",
  "guestName": "Nguyen Van A",
  "roomType": "Deluxe Sea View",
  "checkIn": "2025-07-15",
  "checkOut": "2025-07-18",
  "totalAmount": 4500000,
  "email": "nguyenvana@gmail.com"
}

The Consumer (Go)

Here's a complete Go consumer that pulls from SQS, processes bookings, and handles graceful shutdown:

package main

import (
    "context"
    "encoding/json"
    "fmt"
    "log"
    "os"
    "os/signal"
    "syscall"
    "time"

    "github.com/aws/aws-sdk-go-v2/aws"
    "github.com/aws/aws-sdk-go-v2/config"
    "github.com/aws/aws-sdk-go-v2/service/sqs"
)

type Booking struct {
    BookingID   string  `json:"bookingId"`
    GuestName   string  `json:"guestName"`
    RoomType    string  `json:"roomType"`
    CheckIn     string  `json:"checkIn"`
    CheckOut    string  `json:"checkOut"`
    TotalAmount float64 `json:"totalAmount"`
    Email       string  `json:"email"`
}

func processBooking(booking Booking) error {
    fmt.Printf("\n[%s] Processing new booking:\n", time.Now().Format("15:04:05"))
    fmt.Printf("  Guest : %s (%s)\n", booking.GuestName, booking.Email)
    fmt.Printf("  Room  : %s\n", booking.RoomType)
    fmt.Printf("  Dates : %s → %s\n", booking.CheckIn, booking.CheckOut)
    fmt.Printf("  Total : %.0f\n", booking.TotalAmount)
    fmt.Printf("  → Sending confirmation email... OK\n")
    return nil
}

func main() {
    queueURL := os.Getenv("QUEUE_URL")
    if queueURL == "" {
        log.Fatal("Missing QUEUE_URL — export QUEUE_URL=https://sqs.ap-southeast-1.amazonaws.com/...")
    }

    ctx, cancel := context.WithCancel(context.Background())
    defer cancel()

    go func() {
        sig := make(chan os.Signal, 1)
        signal.Notify(sig, os.Interrupt, syscall.SIGTERM)
        <-sig
        fmt.Println("\nShutting down consumer...")
        cancel()
    }()

    cfg, err := config.LoadDefaultConfig(ctx)
    if err != nil {
        log.Fatalf("Failed to load AWS config: %v", err)
    }

    client := sqs.NewFromConfig(cfg)
    fmt.Printf("Consumer running. Listening to queue...\n")
    fmt.Printf("Queue: %s\n\n", queueURL)

    for {
        select {
        case <-ctx.Done():
            fmt.Println("Consumer stopped cleanly.")
            return
        default:
        }

        result, err := client.ReceiveMessage(ctx, &sqs.ReceiveMessageInput{
            QueueUrl:            aws.String(queueURL),
            MaxNumberOfMessages: 10,
            WaitTimeSeconds:     20, // long polling — avoids empty requests
        })
        if err != nil {
            if ctx.Err() != nil {
                return
            }
            log.Printf("ReceiveMessage error: %v — retrying in 3s\n", err)
            time.Sleep(3 * time.Second)
            continue
        }

        if len(result.Messages) == 0 {
            fmt.Print(".")
            continue
        }

        fmt.Printf("\nReceived %d message(s)\n", len(result.Messages))

        for _, msg := range result.Messages {
            var booking Booking
            if err := json.Unmarshal([]byte(*msg.Body), &booking); err != nil {
                log.Printf("JSON parse failed: %v — skipping message\n", err)
                continue
            }

            if err := processBooking(booking); err != nil {
                log.Printf("Processing failed: %v\n", err)
                continue
            }

            // Only delete AFTER successful processing
            _, err = client.DeleteMessage(ctx, &sqs.DeleteMessageInput{
                QueueUrl:      aws.String(queueURL),
                ReceiptHandle: msg.ReceiptHandle,
            })
            if err != nil {
                log.Printf("DeleteMessage failed: %v\n", err)
            }
        }
    }
}

Running It

mkdir hotel-booking-consumer
cd hotel-booking-consumer
go mod init hotel-booking-consumer
go get github.com/aws/aws-sdk-go-v2/config
go get github.com/aws/aws-sdk-go-v2/service/sqs
export QUEUE_URL="https://sqs.ap-southeast-1.amazonaws.com/123456789012/hotel-booking-queue"
go run main.go

Five Traps That Will Bite You

Getting a queue working is easy. Getting it right is where most teams make mistakes.

1. Visibility Timeout Shorter Than Processing Time

When SQS delivers a message to your worker, it hides that message from other workers for a window of time — the visibility timeout. If your worker hasn't deleted the message by the time that window expires, SQS assumes the worker failed and makes the message visible again.

If your processing takes 45 seconds but your visibility timeout is 30 seconds, the message gets redelivered while the first worker is still processing it. Now two workers are doing the same job.

Fix: Set the visibility timeout to at least 2× your expected processing time. For unpredictable workloads, extend it programmatically during processing.

2. Deleting the Message Before You Finish

It's tempting to delete the message from the queue as soon as you receive it — "I'll handle it now, no need to keep it around." But if your process crashes mid-way, the message is gone forever.

// Wrong: delete first, then process
client.DeleteMessage(...)
processBooking(booking) // if this panics, the booking is lost

// Right: process first, then delete
processBooking(booking)
client.DeleteMessage(...) // only reached on success

Fix: Always delete after successful processing. This is why the code above puts DeleteMessage at the end of the loop body.

3. Assuming Standard Queues Preserve Order

SQS Standard queues offer at-least-once delivery with best-effort ordering — which means messages can arrive out of order. If you send "booking created" followed by "booking updated", the update might be processed first.

For status machines or event-sourced systems, this is a silent data corruption bug. Your booking status reverts to "created" after being "confirmed".

Fix: Use SQS FIFO queues if ordering matters. Or design your consumers to be idempotent — able to handle out-of-order messages without corrupting state.

4. No Dead Letter Queue

What happens to a message that fails to process every time? Without a Dead Letter Queue (DLQ), it retries forever — or until its retention period expires and it silently disappears.

A DLQ is a separate queue where messages go after a configurable number of failed attempts (e.g., after 3 retries). Once there, you can:

  • Trigger a CloudWatch alarm to alert on-call
  • Inspect the message to understand why it failed
  • Replay it after fixing the bug

Without a DLQ, failures are invisible. With one, you catch problems before they become incidents.

Fix: Always configure a DLQ. Set maxReceiveCount to 3–5. Set a CloudWatch alarm on the DLQ's ApproximateNumberOfMessagesVisible metric.

5. Short Polling Without Wait Times

By default, ReceiveMessage returns immediately even if the queue is empty. If your consumer calls this in a tight loop, you're making hundreds of API calls per minute — paying for empty responses.

The code above uses WaitTimeSeconds: 20. This is long polling: SQS waits up to 20 seconds for a message to arrive before returning an empty response. It reduces empty polls by ~95% and cuts costs significantly.

Fix: Always set WaitTimeSeconds to 20 (the maximum). There is almost no reason to use short polling in a production consumer.

The Dead Letter Queue Pattern

Here's a practical DLQ setup worth understanding:

Main Queue ──► Worker
                  │
              (fails 3×)
                  │
                  ▼
            Dead Letter Queue ──► CloudWatch Alarm ──► SNS ──► On-Call

Configure your main queue's redrive policy:

{
  "deadLetterTargetArn": "arn:aws:sqs:ap-southeast-1:123456789:hotel-booking-dlq",
  "maxReceiveCount": 3
}

When a message fails three times, SQS moves it to the DLQ automatically. A CloudWatch alarm fires when the DLQ depth exceeds zero. You wake up to a Slack notification with the failed message in the DLQ — not to a flood of 500 errors in production.

The difference between a 3am emergency and a 9am debugging session is usually a DLQ.

When Not to Use a Queue

Queues add operational complexity. They're not the right tool for everything.

Use a queue when:

  • The work can be deferred (emails, notifications, reports, thumbnails)
  • You need to absorb traffic spikes
  • Processing is slow relative to the request/response cycle
  • You want to decouple services that scale independently

Don't use a queue when:

  • The user needs the result immediately (e.g., payment authorization)
  • The operation is cheap and fast
  • You're early in development and optimizing prematurely

The goal isn't to queue everything. It's to identify the work that doesn't need to block your users — and move it out of the critical path.

Summary

The pattern is straightforward: receive fast, process async.

Your API should do the minimum required to guarantee the request was accepted — write to the database, enqueue a message, return a response. Everything else — emails, webhooks, downstream syncs, report generation — belongs in a worker.

When traffic spikes, your API stays fast because it has nothing expensive to do. Workers slow down gracefully. The queue absorbs the difference. No cascading failures.

That's the contract a queue gives you: a buffer between the rate at which work arrives and the rate at which you can do it.