Graceful Shutdown - What Actually Happens

The Problem

Originally the gateway looked something like:

grpcServer.Serve(listener)

The issue is that Serve() is a blocking call.

Internally it is continuously accepting connections and handling requests.

Serve()
  ├─ accept connection
  ├─ handle request
  ├─ accept connection
  ├─ handle request
  └─ ...

As long as the server is running, execution never moves to the next line.

So if I wrote:

grpcServer.Serve(listener)
 
signal.Notify(...)

the signal handling code would never execute.

---

Moving the Server to a Goroutine

go func() {
	if err := grpcServer.Serve(listener); err != nil &&
		err != grpc.ErrServerStopped {
		log.Fatalf("gRPC server error: %v", err)
	}
}()

Now there are two execution paths.

Main Goroutine

Register signals
Wait for shutdown
Trigger graceful stop

Server Goroutine

Serve requests
Serve requests
Serve requests
Serve requests

Both run concurrently.

---

Why Do We Need Shutdown Handling?

Imagine the gateway is running.

Client
   |
Gateway
   |
Redis

Now someone presses Ctrl+C.

Without shutdown handling:

Process dies immediately.

Any active requests may be interrupted.

Connections may close abruptly.

Logs may not flush.

Cleanup code may never run.

---

Graceful Shutdown vs Hard Shutdown

Hard Stop

STOP NOW

Current request:

50% complete

Process:

Killed

Request never finishes.

---

Graceful Stop

Stop accepting new work
Finish current work
Exit

Current request:

50% complete

becomes

100% complete

then server exits.

This is much safer in production systems.

---

Creating the Signal Channel

shutdownChan := make(chan os.Signal, 1)

This creates a buffered channel.

Think of it as:

Mailbox size = 1

The operating system can place one signal inside.

---

Why os.Signal?

Channels have types.

Examples:

chan int
chan string
chan bool

carry integers, strings and booleans.

This channel carries operating-system signals.

chan os.Signal

Examples of values:

interrupt
terminated

---

Registering Signals

signal.Notify(
	shutdownChan,
	os.Interrupt,
	syscall.SIGTERM,
)

This is registration.

You’re telling the Go runtime:

If Ctrl+C happens,
send it to shutdownChan.
 
If SIGTERM happens,
send it to shutdownChan.

---

What Is os.Interrupt?

Usually generated by:

Ctrl+C

inside the terminal.

Flow:

Keyboard
   |
Ctrl+C
   |
Operating System
   |
SIGINT
   |
Go Runtime
   |
shutdownChan

---

What Is SIGTERM?

Usually sent by:

kill PID

or

Docker
Kubernetes
systemd

when stopping a service.

Meaning:

Please terminate gracefully.

---

The Most Important Line

sh := <-shutdownChan

Read it as:

Wait here until something arrives.

This line blocks.

Nothing below it executes.

The gateway simply waits.

---

Understanding <-

Send:

ch <- value

Receive:

value := <-ch

Receive and ignore:

<-ch

In the shutdown flow:

sh := <-shutdownChan

means:

Take the signal out of the channel
and store it in sh.

---

Example Timeline

Program starts:

Gateway online

Server goroutine:

Serving requests

Main goroutine:

Waiting on shutdownChan

---

User presses:

Ctrl+C

Operating system sends:

SIGINT

Go runtime does roughly:

shutdownChan <- os.Interrupt

Channel:

[ interrupt ]

---

Main goroutine wakes up:

sh := <-shutdownChan

Now:

log.Printf("received shutdown signal: %v", sh)

prints:

received shutdown signal: interrupt

---

Triggering GracefulStop

grpcServer.GracefulStop()

This tells gRPC:

Do not accept new requests.

But:

Finish current requests.

Then:

Shutdown server.

---

What Happens to Serve()?

Earlier:

grpcServer.Serve(listener)

was running forever.

After:

grpcServer.GracefulStop()

the Serve loop exits.

It returns:

grpc.ErrServerStopped

---

Why Is That Returned As An Error?

Go APIs often return an error to explain why they stopped.

Examples:

Port closed
Network failure
Listener failure

are real errors.

But:

Server stopped because you asked it to stop.

is not a failure.

gRPC represents that case as:

grpc.ErrServerStopped

---

Why Ignore It?

Without filtering:

if err != nil {
	log.Println(err)
}

logs:

grpc: server stopped

every shutdown.

That looks like a crash even though shutdown was intentional.

Instead:

if err != nil &&
	err != grpc.ErrServerStopped {
	log.Println(err)
}

means:

Ignore expected shutdowns.
 
Only log unexpected failures.

---

Final Flow

Start Gateway
      |
Create Redis Router
      |
Create Listener
      |
Create gRPC Server
      |
Register Service
      |
Start Serve() in Goroutine
      |
Register Ctrl+C and SIGTERM
      |
Wait on shutdownChan
      |
Ctrl+C Pressed
      |
Signal arrives
      |
Log signal
      |
GracefulStop()
      |
Serve() exits
      |
grpc.ErrServerStopped returned
      |
Ignored
      |
Gateway exits cleanly

Mental Model

The entire pattern can be summarized as:

Server runs in background.
 
Main goroutine waits for shutdown request.
 
When shutdown request arrives:
 
    stop accepting work
    finish current work
    exit cleanly