golib/mail/smtp/TESTING.md

Testing Guide

Comprehensive testing documentation for the SMTP package, covering test execution, race detection, coverage analysis, and quality assurance.

---

Table of Contents

• Overview
• Quick Start
• Test Framework
• Running Tests
• Test Coverage
• Thread Safety
• Test Architecture
• Writing Tests
• Best Practices
• Troubleshooting
• CI Integration

---

Overview

The SMTP package uses Ginkgo v2 (BDD testing framework) and Gomega (matcher library) for comprehensive testing with expressive, behavior-driven test specifications.

Test Suite Summary

Package Statistics:

Package	Specs	Coverage	Duration	Race-Safe
smtp	104	86.6%	~27s	✅
config	110	96.7%	~0.03s	✅
tlsmode	165	98.8%	~0.02s	✅
Total	379	93.4%	~27s	✅

With Race Detector: ~40s total execution time

Coverage Areas:

• SMTP client operations (connection, authentication, sending)
• TLS mode handling (None, STARTTLS, Strict TLS)
• Configuration parsing and validation (DSN, network protocols)
• Health monitoring integration
• Error handling and edge cases
• Thread safety and concurrency
• Performance benchmarking

---

Quick Start

# Install Ginkgo CLI (optional)
go install github.com/onsi/ginkgo/v2/ginkgo@latest

# Run all tests
go test ./...

# Run with coverage
go test -timeout=10m -v -cover -covermode=atomic ./...

# Run with race detection (recommended)
CGO_ENABLED=1 go test -race -timeout=10m -v -cover -covermode=atomic ./...

# Using Ginkgo CLI
ginkgo -r -cover -race

# Generate coverage report
go test -coverprofile=coverage.out ./...
go tool cover -html=coverage.out

---

Test Framework

Ginkgo v2 - BDD testing framework (docs)

• Hierarchical test organization (Describe, Context, It)
• Setup/teardown hooks (BeforeEach, AfterEach)
• Focus and skip mechanisms (FDescribe, XIt, Skip())
• Parallel execution support
• Rich reporting with custom formatters

Gomega - Matcher library (docs)

• Readable assertion syntax (Expect(...).To(Equal(...)))
• Extensive built-in matchers
• Eventually/Consistently for async testing
• Detailed failure messages with context

gmeasure - Performance benchmarking (docs)

• Statistical analysis of measurements
• Duration and memory profiling
• Comparative benchmarks
• Automatic report generation

---

Running Tests

Basic Commands

# Run all tests in package and subpackages
go test ./...

# Verbose output with test names
go test -v ./...

# With coverage reporting
go test -cover ./...

# Specific subpackage
go test ./config
go test ./tlsmode

# Run specific test file
go test -v -run TestSMTP
go test -v -run TestConfig

Advanced Options

# Timeout for long-running tests
go test -timeout=10m ./...

# Coverage with atomic mode
go test -covermode=atomic -coverprofile=coverage.out ./...

# Generate HTML coverage report
go tool cover -html=coverage.out -o coverage.html

# Verbose with race detection
CGO_ENABLED=1 go test -race -v ./...

Ginkgo CLI Commands

# Run all tests recursively
ginkgo -r

# With coverage
ginkgo -r -cover

# Parallel execution
ginkgo -r -p

# Focus on specific test
ginkgo --focus="SMTP.*Send"

# Skip specific tests
ginkgo --skip="Integration"

# Generate JUnit XML report
ginkgo -r --junit-report=results.xml

# Watch mode (rerun on file changes)
ginkgo watch -r

Package-Specific Tests

# SMTP main package (104 specs)
go test -v -run TestSMTP

# Configuration tests (110 specs)
go test -v ./config -run TestConfig

# TLS mode tests (165 specs)
go test -v ./tlsmode -run TestTLSMode

---

Test Coverage

Current Coverage Statistics

Overall Coverage: 93.4%

Detailed Breakdown:

SMTP Core Package (86.6%)

File              Coverage    Uncovered Lines
─────────────────────────────────────────────
interface.go      100%        -
model.go          95.8%       (error paths)
client.go         89.4%       (retry logic)
dial.go           88.2%       (TLS edge cases)
monitor.go        100%        -
error.go          100%        -

Not Covered:

• Commented CRAM-MD5 authentication code (lines 237-241 in dial.go)
• Some error recovery paths that require specific network failures
• Certain TLS handshake edge cases

Config Package (96.7%)

File              Coverage    Details
─────────────────────────────────────────────
interface.go      100%        All methods tested
model.go          98.5%       DSN parsing comprehensive
config.go         95.2%       Validation paths covered
error.go          100%        All error codes tested

Not Covered:

• Some invalid DSN format edge cases
• Certain network protocol error paths

TLSMode Package (98.8%)

File              Coverage    Details
─────────────────────────────────────────────
interface.go      100%        All conversions tested
model.go          100%        All modes tested
encode.go         97.6%       JSON/YAML/TOML/CBOR covered
format.go         100%        String formatting tested

Not Covered:

• Some CBOR decoding error paths

Generating Coverage Reports

HTML Report:

# Generate coverage profile
go test -coverprofile=coverage.out ./...

# View in browser
go tool cover -html=coverage.out

# Save as HTML file
go tool cover -html=coverage.out -o coverage.html

Terminal Summary:

# Show coverage per package
go test -cover ./...

# Detailed function-level coverage
go test -coverprofile=coverage.out ./...
go tool cover -func=coverage.out

Coverage by Function:

go tool cover -func=coverage.out | grep -E "dial.go|client.go"

---

Thread Safety

Race Detection

Critical for concurrent operations validation

# Enable race detector (requires CGO)
CGO_ENABLED=1 go test -race ./...

# With timeout for long-running tests
CGO_ENABLED=1 go test -race -timeout=10m ./...

# Specific package
CGO_ENABLED=1 go test -race ./config

What Race Detector Validates:

• Mutex protection of connection state (sync.Mutex in smtpClient)
• Atomic operations (connection validity checks)
• Concurrent method calls on same client
• Independent client clones
• Configuration read/write safety

Expected Output:

# ✅ Success (no races detected)
ok  	github.com/nabbar/golib/mail/smtp	27.445s

# ❌ Data race detected
WARNING: DATA RACE
Write at 0x00c00012c180 by goroutine 8:
  github.com/nabbar/golib/mail/smtp.(*smtpClient)._close()
      /path/to/model.go:79 +0x44

Current Status: Zero data races detected across all 379 test specs.

Concurrency Tests

The test suite includes specific concurrency scenarios:

SMTP Package:

• Concurrent Check() calls
• Concurrent Send() operations with different clients
• Clone() with concurrent operations
• Config updates during active connections

Config Package:

• Concurrent read operations
• DSN parsing under concurrent load
• Thread-safe getters

TLSMode Package:

• Concurrent encoding/decoding
• Parallel parsing operations
• Immutable value guarantees

---

Test Architecture

Test Organization

smtp/
├── *_test.go              # Main SMTP tests
│   ├── smtp_suite_test.go     # Suite setup and helpers
│   ├── client_test.go          # Client operations
│   ├── dial_test.go            # Connection tests
│   ├── send_test.go            # Email sending (skipped - needs real server)
│   ├── monitor_test.go         # Health monitoring
│   ├── integration_test.go     # End-to-end scenarios
│   ├── benchmark_test.go       # Performance tests
│   └── helper_test.go          # Test SMTP server + utilities
├── config/
│   ├── *_test.go
│   │   ├── config_suite_test.go        # Suite setup
│   │   ├── config_test.go              # Basic operations
│   │   ├── config_dsn_test.go          # DSN parsing
│   │   ├── config_validation_test.go   # Validation rules
│   │   ├── config_errors_test.go       # Error handling
│   │   ├── config_edge_cases_test.go   # Edge cases
│   │   ├── config_coverage_test.go     # Coverage improvement
│   │   ├── config_benchmark_test.go    # Performance
│   │   └── example_test.go             # Runnable examples
└── tlsmode/
    ├── *_test.go
        ├── tlsmode_suite_test.go       # Suite setup
        ├── format_test.go              # String formatting
        ├── parsing_test.go             # String/number parsing
        ├── encoding_test.go            # JSON/YAML/TOML
        ├── edge_cases_test.go          # Edge cases
        ├── viper_test.go               # Viper integration
        └── benchmark_test.go           # Performance

Test Helpers

SMTP Package Helpers (helper_test.go):

• startTestSMTPServer(): Embedded SMTP server using github.com/emersion/go-smtp
• createTLSConfig(): Generate self-signed certificates for TLS testing
• newTestConfig(): Create test configurations
• newTestSMTPClient(): Initialize test clients
• contextWithTimeout(): Context helpers
• getFreePort(): Dynamic port allocation

Test Server Features:

• TLS and non-TLS modes
• PLAIN authentication support
• Message capture for verification
• Thread-safe message storage
• Configurable auth requirements

Test Categories

Unit Tests:

• Individual method testing
• Error condition validation
• Input validation
• State management

Integration Tests:

• Full email flow (connect → auth → send → disconnect)
• Multiple recipients
• Large email content
• Server restart recovery
• Authentication scenarios

Concurrency Tests:

• Parallel client operations
• Clone safety
• Connection lifecycle races
• Config update races

Benchmark Tests:

• Connection performance
• TLS handshake overhead
• Parsing throughput
• Encoding/decoding speed

Edge Case Tests:

• Invalid inputs
• Network failures
• Timeout scenarios
• Resource exhaustion

---

Writing Tests

Test Structure

package smtp_test

import (
    . "github.com/onsi/ginkgo/v2"
    . "github.com/onsi/gomega"
)

var _ = Describe("Feature Name", func() {
    var (
        client smtp.SMTP
        cfg    config.Config
    )
    
    BeforeEach(func() {
        // Setup before each test
        cfg, _ = config.New(config.ConfigModel{
            DSN: "tcp(localhost:25)/",
        })
        client, _ = smtp.New(cfg, nil)
    })
    
    AfterEach(func() {
        // Cleanup after each test
        if client != nil {
            client.Close()
        }
    })
    
    Context("when condition X", func() {
        It("should behave Y", func() {
            // Test implementation
            result := client.SomeMethod()
            Expect(result).To(Equal(expected))
        })
    })
})

Assertion Examples

// Basic assertions
Expect(err).ToNot(HaveOccurred())
Expect(result).To(BeNil())
Expect(value).To(Equal(expected))

// String matchers
Expect(msg).To(ContainSubstring("error"))
Expect(msg).To(MatchRegexp(`\d{3}`))

// Numeric matchers
Expect(count).To(BeNumerically(">", 0))
Expect(duration).To(BeNumerically("~", 100*time.Millisecond, 10*time.Millisecond))

// Collection matchers
Expect(slice).To(HaveLen(5))
Expect(slice).To(ContainElement("item"))
Expect(slice).To(ConsistOf("a", "b", "c"))

// Async matchers
Eventually(func() bool {
    return server.IsReady()
}, 5*time.Second).Should(BeTrue())

Consistently(func() error {
    return client.Check(ctx)
}, 2*time.Second, 100*time.Millisecond).Should(Succeed())

Testing SMTP Operations

It("should send email successfully", func() {
    // Start test SMTP server
    backend := &testBackend{
        requireAuth: false,
        messages:    make([]testMessage, 0),
    }
    server, err := startTestSMTPServer(backend, false)
    Expect(err).ToNot(HaveOccurred())
    defer server.Close()
    
    // Get server address
    host, port, err := getServerHostPort(server)
    Expect(err).ToNot(HaveOccurred())
    
    // Create client
    cfg := newTestConfig(host, port, tlsmode.TLSNone)
    client := newTestSMTPClient(cfg)
    defer client.Close()
    
    // Send email
    email := newTestEmail("from@test.com", "to@test.com", "Subject", "Body")
    err = client.Send(ctx, "from@test.com", []string{"to@test.com"}, email)
    Expect(err).ToNot(HaveOccurred())
    
    // Verify
    Eventually(func() int {
        return len(backend.messages)
    }, 2*time.Second).Should(Equal(1))
})

Benchmarking

It("should measure connection performance", func() {
    experiment := gmeasure.NewExperiment("Connection Benchmark")
    AddReportEntry(experiment.Name, experiment)
    
    experiment.Sample(func(idx int) {
        experiment.MeasureDuration("connect", func() {
            client, _ := smtp.New(cfg, nil)
            defer client.Close()
            _ = client.Check(ctx)
        })
    }, gmeasure.SamplingConfig{N: 100})
    
    stats := experiment.GetStats("connect")
    AddReportEntry("Mean connection time", stats.DurationFor(gmeasure.StatMean))
})

---

Best Practices

Test Organization

1. Group Related Tests

Describe("SMTP Client", func() {
    Context("Connection", func() {
        It("should connect to server")
        It("should handle connection failure")
    })
    
    Context("Authentication", func() {
        It("should authenticate with valid credentials")
        It("should reject invalid credentials")
    })
})

2. Use Descriptive Test Names

// ❌ Bad
It("test 1", func() { ... })

// ✅ Good
It("should reject email addresses containing CR/LF characters", func() { ... })

3. Test One Thing Per Spec

// ❌ Bad - tests multiple things
It("should work", func() {
    client.Connect()
    client.Auth()
    client.Send()
})

// ✅ Good - focused tests
It("should establish connection successfully", func() {
    err := client.Connect()
    Expect(err).ToNot(HaveOccurred())
})

Test Data Management

1. Use Test Fixtures

const (
    testHost     = "localhost"
    testPort     = 2525
    testUser     = "testuser"
    testPassword = "testpass"
)

2. Generate Dynamic Data

func newTestEmail(from, to, subject, body string) *testWriter {
    data := fmt.Sprintf("From: %s\r\nTo: %s\r\nSubject: %s\r\n\r\n%s",
        from, to, subject, body)
    return &testWriter{data: data}
}

3. Clean Up Resources

BeforeEach(func() {
    server, _ = startTestServer()
})

AfterEach(func() {
    if server != nil {
        server.Close()
    }
})

Performance Testing

1. Use gmeasure for Benchmarks

experiment := gmeasure.NewExperiment("Feature Name")
AddReportEntry(experiment.Name, experiment)

experiment.Sample(func(idx int) {
    experiment.MeasureDuration("operation", func() {
        // Code to measure
    })
}, gmeasure.SamplingConfig{N: 1000})

2. Report Meaningful Metrics

stats := experiment.GetStats("operation")
AddReportEntry("Mean", stats.DurationFor(gmeasure.StatMean))
AddReportEntry("P95", stats.DurationFor(gmeasure.StatP95))
AddReportEntry("Max", stats.DurationFor(gmeasure.StatMax))

Async Testing

1. Use Eventually for Async Operations

Eventually(func() bool {
    return condition()
}, timeout, pollingInterval).Should(BeTrue())

2. Use Consistently for Stability Checks

Consistently(func() error {
    return client.Check(ctx)
}, duration, interval).Should(Succeed())

---

Troubleshooting

Common Issues

1. Tests Hang or Timeout

# Increase timeout
go test -timeout=20m ./...

# Check for deadlocks with pprof
go test -timeout=20m -cpuprofile=cpu.prof ./...

2. Race Detector Warnings

# Run specific test with race detector
CGO_ENABLED=1 go test -race -run TestSpecificName

# Check goroutine traces
CGO_ENABLED=1 go test -race -v 2>&1 | tee race.log

3. Port Conflicts

# Check for port in use
lsof -i :25
netstat -an | grep 25

# Tests use dynamic port allocation
# Failure indicates system resource exhaustion

4. TLS Certificate Errors

# Verify test certificates are generated
ls -la *_test.go | grep helper

# Check certificate validity
openssl x509 -in cert.pem -text -noout

5. Coverage Not Updating

# Clean test cache
go clean -testcache

# Regenerate coverage
go test -coverprofile=coverage.out ./...

Debugging Tests

1. Verbose Output

go test -v ./...
ginkgo -v

2. Focus on Specific Test

# Focus in code
FDescribe("Only This", func() { ... })
FIt("Only This Test", func() { ... })

# Focus via CLI
ginkgo --focus="test name pattern"

3. Skip Problematic Tests

# Skip in code
XDescribe("Skip This", func() { ... })
XIt("Skip This Test", func() { ... })

# Skip via CLI
ginkgo --skip="test name pattern"

4. Add Debug Output

It("should work", func() {
    GinkgoWriter.Printf("Debug: value=%v\n", value)
    // or
    fmt.Fprintf(GinkgoWriter, "Debug: %+v\n", struct)
})

---

CI Integration

GitHub Actions

name: Tests

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        go: ['1.18', '1.19', '1.20', '1.21']
    
    steps:
      - uses: actions/checkout@v3
      
      - name: Set up Go
        uses: actions/setup-go@v4
        with:
          go-version: ${{ matrix.go }}
      
      - name: Run Tests
        run: go test -timeout=10m -v -cover ./...
      
      - name: Run Race Detector
        run: CGO_ENABLED=1 go test -race -timeout=10m ./...
      
      - name: Generate Coverage
        run: |
          go test -coverprofile=coverage.out -covermode=atomic ./...
          go tool cover -html=coverage.out -o coverage.html
      
      - name: Upload Coverage
        uses: codecov/codecov-action@v3
        with:
          files: ./coverage.out
          flags: unittests

GitLab CI

test:
  image: golang:1.21
  script:
    - go test -timeout=10m -v -cover ./...
    - CGO_ENABLED=1 go test -race -timeout=10m ./...
  coverage: '/coverage: \d+\.\d+% of statements/'

Jenkins

pipeline {
    agent any
    stages {
        stage('Test') {
            steps {
                sh 'go test -timeout=10m -v -cover ./...'
            }
        }
        stage('Race Detection') {
            steps {
                sh 'CGO_ENABLED=1 go test -race -timeout=10m ./...'
            }
        }
    }
}

---

Test Metrics

Performance Benchmarks

Parsing Performance (TLSMode):

Parse string:  64ns/op
Parse bytes:   66ns/op  
Parse int:     54ns/op

Encoding Performance (TLSMode):

String roundtrip:  76ns/op
Int64 roundtrip:   51ns/op
JSON roundtrip:    735ns/op

Connection Performance (SMTP):

Health check:      50-100ms
Send email:        150-300ms
TLS handshake:     100-200ms
Authentication:    50-100ms

Memory Usage:

Client instance:   ~400 bytes
Config instance:   ~200 bytes
TLS mode:          8 bytes

Test Execution Times

Package         Time      Tests    Coverage
─────────────────────────────────────────────
smtp            26.9s     104      86.6%
smtp/config     0.03s     110      96.7%
smtp/tlsmode    0.02s     165      98.8%
─────────────────────────────────────────────
Total           ~27s      379      93.4%

With Race Detector: ~40s total

---

Additional Resources

• Ginkgo Documentation: https://onsi.github.io/ginkgo/
• Gomega Matchers: https://onsi.github.io/gomega/
• Go Testing: https://golang.org/pkg/testing/
• Race Detector: https://golang.org/doc/articles/race_detector.html
• Coverage Tools: https://go.dev/blog/cover

---

Questions? Open an issue or consult the main README.