Learn / Go Service Engineering
Storage, Workers, and Release
Persist data, run bounded background work, and ship a small service binary with repeatable checks.
Course: Go Service Engineering. Level: Intermediate. Topic: Reliable services.
Stage: intermediate - Intermediate storage and workers - Storage, workers, and release evidence. Connect database access, background work, and release commands into one reviewable service path.
Outcomes
- Use repository interfaces over SQL code.
- Stop worker pools cleanly.
- Run tests, race checks, and container builds.
- Start goroutines and wait for completion safely.
- Use channels for communication and mutexes for shared state.
- Avoid leaks, races, and unbounded concurrency.
- Use gofmt, go test, go vet, go doc, and go list confidently.
- Maintain dependency metadata with go mod tidy.
- Understand how tooling supports repeatable builds.
Concepts
- database/sql
- migration
- worker pool
- race detector
- Concurrency
- Goroutines are cheap, not free
- Channels communicate ownership
- Mutexes are often the simplest answer
- Go Programming foundations
- Guided practice
- Tooling
- The standard workflow
- Docs are source-adjacent
Concept flow
Show how storage, workers, and release evidence moves from trigger to implementation outcome in Go Services.
- Handler
- Repository
- Database
- Job queue
- Worker pool
Session flow
- Model database/sql (concept, 12 min) — Name the decisions behind database/sql before writing code.
- Use repository interfaces over SQL code.
- Explain where database/sql belongs in ticket processing platform.
- Build the vertical slice (walkthrough, 21 min) — Implement the smallest useful slice in internal/store/tickets.go.
- Stop worker pools cleanly.
- Connect migration to a working example.
- Verify and harden (exercise, 14 min) — Run go test with the race detector on worker code.
- Run tests, race checks, and container builds.
- Record one risk or follow-up before moving on.
- Goroutines, Channels, and Shared State: Goroutines are cheap, not free (concept, 33 min) — A goroutine is a lightweight concurrent execution path managed by the Go runtime. You can create many, but each needs a clear lifetime. A goroutine blocked forever on send, receive, lock, or I/O is a leak.
- Retained source example: Wait for goroutines
var wg sync.WaitGroup
for _, id := range []int{1, 2, 3} {
id := id
wg.Add(1)
go func() {
defer wg.Done()
fmt.Println("processed", id)
}()
}
wg.Wait()
The id := id line gives each goroutine its own loop value.
- Goroutines, Channels, and Shared State: Channels communicate ownership (walkthrough, 33 min) — Channels are best when they transfer data, ownership, or events between goroutines. Closing a channel signals that no more values will be sent. Do not close a channel from the receiver side unless the receiver also owns sending.
- Retained source example: Worker pool shape
jobs := make(chan int)
results := make(chan int)
go func() {
defer close(results)
for job := range jobs {
results <- job * job
}
}()
The goroutine consumes jobs until jobs is closed, then closes results.
- Goroutines, Channels, and Shared State: Mutexes are often the simplest answer (walkthrough, 33 min) — Shared memory is not forbidden in Go. A mutex is clear and efficient when multiple goroutines need to protect a small critical section. Channels are for coordination; mutexes are for protecting shared state.
- Retained source example: Protected counter
type Counter struct {
mu sync.Mutex
n int
}
func (c *Counter) Inc() {
c.mu.Lock()
defer c.mu.Unlock()
c.n++
}
The mutex protects the invariant around n.
- Goroutines, Channels, and Shared State: practices and mistakes (review, 5 min) — Retained guidance from the legacy PTesting lesson.
- Practice: Give every goroutine a clear owner and shutdown path.
- Practice: Use buffered channels deliberately, not as a hidden fix for deadlock.
- Practice: Run the race detector on meaningful test paths.
- Avoid: Capturing loop variables incorrectly in goroutines.
- Avoid: Closing channels from receivers that do not own sending.
- Avoid: Using channels where a mutex would be simpler and clearer.
- Goroutines, Channels, and Shared State: references (review, 2 min) — Original references retained from the legacy library.
- Go Blog: Share Memory By Communicating: https://go.dev/blog/codelab-share
- Data Race Detector: https://go.dev/doc/articles/race_detector
- Toolchain Workflows: The standard workflow (concept, 35 min) — Most Go teams share a small command vocabulary. gofmt formats code, go test verifies behavior, go vet finds suspicious constructs, go mod tidy synchronizes dependency metadata, and go list gives structured package information for scripts.
- Retained source example: Pre-commit style check
gofmt -w .
go mod tidy
go test ./...
go vet ./...
These commands are common local checks before review.
- Toolchain Workflows: Docs are source-adjacent (walkthrough, 35 min) — Exported identifiers should have comments that start with the identifier name. These comments feed godoc and pkg.go.dev, making API documentation part of the code review surface.
- Retained source example: Exported comment
// Store persists and retrieves users.
type Store interface {
FindUser(ctx context.Context, id string) (User, error)
}
The comment begins with Store and explains the exported API.
- Toolchain Workflows: practices and mistakes (review, 5 min) — Retained guidance from the legacy PTesting lesson.
- Practice: Make gofmt non-negotiable.
- Practice: Run go test ./... before sharing changes.
- Practice: Use go vet and static analysis as early warning systems, not as replacements for tests.
- Avoid: Manually editing go.sum.
- Avoid: Adding dependencies without running go mod tidy.
- Avoid: Leaving exported APIs undocumented in reusable packages.
- Toolchain Workflows: references (review, 2 min) — Original references retained from the legacy library.
- Command go: https://pkg.go.dev/cmd/go
- Go Modules Reference: https://go.dev/ref/mod
Code example
Go in internal/store/tickets.go.
package store
import (
"context"
"database/sql"
)
type Store struct {
db *sql.DB
}
type Ticket struct {
ID string
Title string
Status string
}
func (s Store) CreateTicket(ctx context.Context, ticket Ticket) error {
_, err := s.db.ExecContext(ctx, `
insert into tickets (id, title, status)
values ($1, $2, $3)
`, ticket.ID, ticket.Title, ticket.Status)
return err
}
Walkthrough examples
- Storage, Workers, and Release in a ticket processing platform — A team is extending a small Go service plus worker and needs this lesson's pattern to be clear enough for review, testing, and future maintenance.
- File: internal/store/tickets.go
- File: tests/storage-workers-and-release.spec
- File: docs/go-services/storage-workers-and-release.md
- Start from the provided Go snippet and make the intent visible in names and boundaries.
- Apply the checklist item "Pass context to SQL" before adding extra behavior.
- Write down how the implementation changes when migration fails or becomes slow.
- Wait for goroutines — The id := id line gives each goroutine its own loop value.
- Retained source code:
var wg sync.WaitGroup
for _, id := range []int{1, 2, 3} {
id := id
wg.Add(1)
go func() {
defer wg.Done()
fmt.Println("processed", id)
}()
}
wg.Wait()
- Explain the expected behavior.
- Compare the example with the canonical PTLearn implementation.
- Worker pool shape — The goroutine consumes jobs until jobs is closed, then closes results.
- Retained source code:
jobs := make(chan int)
results := make(chan int)
go func() {
defer close(results)
for job := range jobs {
results <- job * job
}
}()
- Explain the expected behavior.
- Compare the example with the canonical PTLearn implementation.
- Protected counter — The mutex protects the invariant around n.
- Retained source code:
type Counter struct {
mu sync.Mutex
n int
}
func (c *Counter) Inc() {
c.mu.Lock()
defer c.mu.Unlock()
c.n++
}
- Explain the expected behavior.
- Compare the example with the canonical PTLearn implementation.
- Race detector — The race detector finds many data races in tests and integration runs.
- Retained source code:
go test -race ./...
- Explain the expected behavior.
- Compare the example with the canonical PTLearn implementation.
- Pre-commit style check — These commands are common local checks before review.
- Retained source code:
gofmt -w .
go mod tidy
go test ./...
go vet ./...
- Explain the expected behavior.
- Compare the example with the canonical PTLearn implementation.
- Exported comment — The comment begins with Store and explains the exported API.
- Retained source code:
// Store persists and retrieves users.
type Store interface {
FindUser(ctx context.Context, id string) (User, error)
}
- Explain the expected behavior.
- Compare the example with the canonical PTLearn implementation.
- Structured package listing — go list is useful for tooling because it exposes package metadata.
- Retained source code:
go list -json ./...
- Explain the expected behavior.
- Compare the example with the canonical PTLearn implementation.
Practice
- Add one migration.
- Write a repository test.
- Run go test with the race detector on worker code.
- Find data races: Which go test flag enables the race detector?
- Expected output: -race
- Hint: It is commonly run as go test -race ./...
- Reference solution: Use go test -race ./...
- Accepted answers: -race | race
- Formatting command: Which standard command formats Go source files?
- Expected output: gofmt
- Hint: One command rewrites files; the other invokes formatting by package.
- Reference solution: Use gofmt for source formatting, commonly gofmt -w .
- Accepted answers: gofmt | go fmt
Checklist
- Pass context to SQL
- Keep transactions short
- Bound worker queues
- Document build and run commands
Quiz prompts
- What is one sign a goroutine may leak? — Long-running goroutines need a clear shutdown signal so deploys and tests can finish cleanly.
- A teammate wants to hide database/sql inside a convenient helper. What should you check first? — Place database/sql at the boundary that keeps ticket processing platform behavior explicit, testable, and reviewable.
- Which artifact best proves this Go Services lesson is ready for review? — Production-ready learning needs evidence: a test, trace, command, screenshot, or log that catches the risk again.
- Intermediate storage and workers: a teammate says the happy path works, but "Storage contract" is still implicit. What should you ask for before merging? — Storage contract belongs in the intermediate stage only when the decision is visible, testable, and tied to a realistic failure mode.
- A reviewer has five minutes to evaluate this intermediate Go Services slice. Which evidence is strongest? — Ship an intermediate worker-backed Go feature with storage transaction proof, bounded queue behavior, and release command evidence.
- Who should usually close a channel? — Closing announces no more sends, so the sender side normally owns it.
- What is a goroutine leak? — Leaked goroutines consume resources and can hold references that prevent collection.
- What does go mod tidy do? — It adds missing and removes unused module requirements.
- Why write comments for exported identifiers? — Go documentation tools extract comments from source.
Flashcards
- Intermediate storage and workers: what decision does "Storage contract" force you to make? Make query ownership, transaction scope, and returned DTOs visible at the repository edge. Evidence prompt: Wrap one write in a transaction and document the rollback behavior.
- Intermediate storage and workers: what decision does "Worker backpressure" force you to make? Use queues, channel bounds, and retry rules that fail predictably under load. Evidence prompt: Add one bounded queue or retry policy and capture what happens when it fills.
- Intermediate storage and workers: what decision does "Release command log" force you to make? Keep build, test, migrate, and run commands repeatable for a teammate. Evidence prompt: Save the exact command log that proves the service is ready to release.
- In Go Services, what should you remember about database/sql? database/sql matters here because it supports "Use repository interfaces over SQL code.".
- In Go Services, what should you remember about migration? migration matters here because it supports "Stop worker pools cleanly.".
- In Go Services, what should you remember about worker pool? worker pool matters here because it supports "Run tests, race checks, and container builds.".
- In Go Services, what should you remember about race detector? race detector matters here because it supports "Use repository interfaces over SQL code.".
Labs
- Ship a storage, workers, and release slice — Extend a small Go service plus worker with a small but reviewable feature that proves the lesson's architecture in code.
- Ship an intermediate worker-backed Go feature with storage transaction proof, bounded queue behavior, and release command evidence.
- Wrap one write in a transaction and document the rollback behavior.
- Add one bounded queue or retry policy and capture what happens when it fills.
- Save the exact command log that proves the service is ready to release.
- Add one migration.
- Write a repository test.
- The lab demonstrates the intermediate storage and workers outcome without skipping earlier contract evidence.
- Each subtopic has a concrete artifact: Storage contract, Worker backpressure, Release command log.
- The review notes explain how this level changes ownership, verification, or operational risk.
Challenge
- Review-ready storage, workers, and release (Stretch) — Ship an intermediate worker-backed Go feature with storage transaction proof, bounded queue behavior, and release command evidence.
- All checklist items are either implemented or documented with a reason.
- The change can be understood from internal/store/tickets.go plus one short note.
- The concept diagram names ownership, failure handling, and verification points.
- A teammate could run the verification steps without asking for hidden context.
Canonical lesson URL