Assist_Design/docs/LOGGING.md

📊 Production-Ready Logging Guide

This guide covers the modern, high-performance logging system implemented using Pino - the fastest JSON logger for Node.js applications.

🎯 Key Features

• ✅ High Performance: Pino is 5x faster than winston
• ✅ Structured JSON Logs: Perfect for log aggregation tools
• ✅ Automatic Correlation IDs: Request tracing across services
• ✅ Sensitive Data Sanitization: Automatic redaction of secrets
• ✅ Environment-Specific Output: Pretty dev logs, structured production logs
• ✅ File Rotation: Automatic log rotation in production
• ✅ Zero Dependencies: Minimal overhead

🚀 Quick Start

Setting Log Levels

# Using the helper script (recommended)
./tools/dev/set-log-level.sh info

# Or manually edit .env
LOG_LEVEL="info"

# Then restart
pnpm dev:restart

Available Log Levels

Level	Description	Use Case
error	Only errors	Production monitoring
warn	Errors + warnings	Production with warnings
info	Standard logging	Development (recommended)
debug	Detailed debugging	Troubleshooting
verbose	All possible logs	Deep debugging

🏗️ Architecture

Development Mode

• Console Output: Pretty-printed, colorized logs
• Format: Human-readable with timestamps
• Performance: Optimized for development experience

Production Mode

• Console Output: Structured JSON for container logs
• File Output: Separate files for different log levels
• Log Rotation: Automatic rotation (10MB per file, 5-10 files retained)
• Performance: Optimized for speed and parsing

📁 File Structure

logs/
├── customer-portal-bff-combined.log    # All logs (info+)
├── customer-portal-bff-error.log       # Error logs only
├── customer-portal-bff-combined.log.1  # Rotated files
└── customer-portal-bff-combined.log.2

🔒 Security Features

Automatic Data Sanitization

The logging system automatically redacts sensitive information:

// These headers/fields are automatically redacted:
const sensitiveKeys = [
  'authorization', 'cookie', 'set-cookie', 'x-api-key', 
  'x-auth-token', 'password', 'secret', 'token', 'jwt', 'bearer'
];

// Example log output:
{
  "headers": {
    "authorization": "[REDACTED]",
    "content-type": "application/json"
  }
}

Correlation IDs

Every HTTP request gets a unique correlation ID for tracing:

// Automatic generation or use existing header
const correlationId = req.headers['x-correlation-id'] || generateId();

// Available in all logs for that request
{
  "correlationId": "1629789123456-abc123def",
  "message": "GET /api/users 200",
  "duration": 45
}

🛠️ Configuration

Environment Variables

# Application name (affects log file names)
APP_NAME="customer-portal-bff"

# Log level
LOG_LEVEL="info"  # error|warn|info|debug|verbose

# Environment
NODE_ENV="development"  # affects log format

Advanced Configuration

The logging is configured in apps/bff/src/common/logging/logging.config.ts:

// Custom log levels per request
customLogLevel: (req, res, err) => {
  if (res.statusCode >= 400 && res.statusCode < 500) return 'warn';
  if (res.statusCode >= 500 || err) return 'error';
  if (res.statusCode >= 300 && res.statusCode < 400) return 'debug';
  return 'info';
}

📊 Log Examples

Development Output

2025-08-21 10:30:15 [INFO] GET /api/health 200 - 5ms
2025-08-21 10:30:16 [WARN] GET /api/users 404 - User not found
2025-08-21 10:30:17 [ERROR] POST /api/auth/login 500 - Database connection failed

Production Output

{
  "level": "info",
  "time": "2025-08-21T10:30:15.123Z",
  "service": "customer-portal-bff",
  "environment": "production",
  "correlationId": "1629789123456-abc123def",
  "msg": "GET /api/health 200",
  "req": {
    "method": "GET",
    "url": "/api/health",
    "headers": { "user-agent": "...", "authorization": "[REDACTED]" }
  },
  "res": {
    "statusCode": 200
  },
  "duration": 5
}

🔍 Monitoring & Observability

Integration with Log Aggregation Tools

The structured JSON format works perfectly with:

• ELK Stack (Elasticsearch, Logstash, Kibana)
• Grafana Loki
• Fluentd
• AWS CloudWatch
• Google Cloud Logging
• Azure Monitor

Search Queries

Common search patterns:

# Find all errors for a specific user
correlationId:"1629789123456-abc123def" AND level:"error"

# Find slow requests
duration:>1000 AND level:"info"

# Find authentication failures
msg:*"auth"* AND level:"error"

# Find specific API endpoints
req.url:"/api/users*" AND res.statusCode:>=400

🚀 Performance

Benchmarks (Pino vs Winston)

Operation	Pino	Winston	Improvement
JSON logging	5.2M ops/sec	1.1M ops/sec	5x faster
Object logging	3.8M ops/sec	0.9M ops/sec	4x faster
String logging	6.1M ops/sec	1.2M ops/sec	5x faster

Memory Usage

• Lower memory footprint: ~40% less memory usage
• Faster garbage collection: Fewer objects created
• Asynchronous by default: Non-blocking I/O operations

📚 Best Practices

1. Use Structured Logging

// Good ✅
logger.info({ userId, action: 'login', duration: 150 }, 'User login successful');

// Bad ❌  
logger.info(`User ${userId} login successful in ${duration}ms`);

2. Include Context

// Good ✅
logger.error({ 
  userId, 
  endpoint: '/api/users', 
  error: err.message,
  stack: err.stack 
}, 'Database query failed');

// Bad ❌
logger.error('Database error');

3. Use Appropriate Log Levels

// Errors: System failures, exceptions
logger.error({ err }, 'Database connection failed');

// Warnings: Recoverable issues, deprecations
logger.warn({ userId }, 'User attempted invalid action');

// Info: Business logic, successful operations
logger.info({ userId, action }, 'User action completed');

// Debug: Detailed debugging information
logger.debug({ query, params }, 'Database query executed');

4. Performance in Loops

// Good ✅ - Use conditional logging
if (logger.isLevelEnabled('debug')) {
  logger.debug({ largeObject }, 'Processing item');
}

// Bad ❌ - Always serializes object
logger.debug({ largeObject }, 'Processing item');

🔧 Troubleshooting

Common Issues

1. Logs Not Appearing

# Check log level
echo $LOG_LEVEL

# Increase verbosity
./tools/dev/set-log-level.sh debug
pnpm dev:restart

2. File Permissions (Production)

# Ensure logs directory exists and is writable
mkdir -p logs
chmod 755 logs

3. Performance Issues

# Reduce log level in production
LOG_LEVEL="warn"

# Use log level checks in hot paths
if (logger.isLevelEnabled('debug')) {
  logger.debug(expensiveToSerialize);
}

🔄 Migration from Winston

If migrating from winston:

1. Remove winston dependencies:

   pnpm remove winston @types/winston nest-winston
   

2. Install Pino:

   pnpm add pino nestjs-pino pino-pretty
   

3. Update logger usage:

   // Old winston way
   import { Logger } from '@nestjs/common';
   private readonly logger = new Logger(ServiceName.name);
   
   // New Pino way
   import { PinoLogger } from 'nestjs-pino';
   constructor(private readonly logger: PinoLogger) {}
   

📈 Future Enhancements

• Metrics Integration: Add Prometheus metrics
• Tracing: OpenTelemetry integration
• Alerts: Automated error alerting
• Dashboard: Real-time log monitoring dashboard

---

Need help? Check the tools/README.md for log level management or the main README.md for general setup.