Assist_Design/README.md

# Customer Portal Project

A modern customer portal where users can self-register, log in, browse & buy subscriptions, view/pay invoices, and manage support cases.

## Architecture Overview

### Systems of Record

- **WHMCS**: Billing, subscriptions, invoices, and **authoritative address storage**
- **Salesforce**: CRM (Accounts, Contacts, Cases) and **order address snapshots**
- **Portal**: Modern UI with backend for frontend (BFF) architecture

### Identity Management

- Portal-native authentication (email + password, optional MFA)
- One-time WHMCS user verification with forced password reset
- User mapping: `user_id ↔ whmcs_client_id ↔ sf_contact_id/sf_account_id`

## Tech Stack

### Frontend (Portal UI)

- **Next.js 15** with App Router
- **Turbopack** for ultra-fast development and builds
- **React 19** with TypeScript
- **Tailwind CSS** with shadcn/ui components
- **TanStack Query** for data fetching and caching
- **Zod** for validation
- **React Hook Form** for form management

### Backend (BFF API)

- **NestJS 11** (Node 24 Current or 22 LTS)
- **Prisma 6** ORM with PostgreSQL 17
- **jsforce** for Salesforce REST API integration
- **salesforce-pubsub-api-client** for Salesforce Platform Events
- **p-queue** for request throttling and queue management
- **WHMCS** custom API client with comprehensive service layer
- **Freebit** SIM management integration
- **Zod-first validation** shared via the domain package
- **Bcrypt** for password hashing

### Queue Management

- **p-queue** for intelligent request throttling to external APIs
- Separate queues for Salesforce (standard + long-running) and WHMCS
- Configurable concurrency, rate limiting, and timeout handling
- Prevents API rate limit violations and resource exhaustion

### Logging

- Centralized structured logging via Pino using `nestjs-pino` in the BFF
- Sensitive fields are redacted; each request has a correlation ID
- Usage pattern in services:
  - Inject `Logger` from `nestjs-pino`: `constructor(@Inject(Logger) private readonly logger: Logger) {}`
  - Log with structured objects: `this.logger.error('Message', { error })`
  - See `docs/LOGGING.md` for full guidelines

### Data & Infrastructure

- **PostgreSQL 17** for users, ID mappings, and optional mirrors
- **Redis 7** for cache, token blacklists, and rate limiting
- **Docker** for local development (Postgres/Redis)

## Project Structure

```
customer-portal/
├── apps/
│   ├── portal/            # Next.js 15 frontend (React 19, Tailwind, shadcn/ui)
│   └── bff/               # NestJS 11 backend (Prisma, p-queue, Zod validation)
├── packages/
│   ├── domain/            # Unified domain layer with contracts & schemas
│   ├── validation/        # Unified validation service (NestJS + React)
│   ├── logging/           # Centralized logging utilities (Pino)
│   └── integrations/      # External service integrations
│       ├── whmcs/         # WHMCS API client
│       └── freebit/       # Freebit SIM management
├── scripts/
│   ├── dev/               # Development management scripts
│   └── prod/              # Production deployment scripts
├── docker/
│   └── dev/               # Docker Compose for local development
│       └── docker-compose.yml  # PostgreSQL 17 + Redis 7
├── docs/                  # Comprehensive documentation
├── secrets/               # Private keys (git ignored)
├── env/                   # Environment file templates
├── package.json           # Root workspace configuration
├── pnpm-workspace.yaml    # pnpm workspace definition
└── README.md              # This file
```

## Getting Started

### Prerequisites

- **Node.js**: Version 22 (LTS) or 24 (Current) - specified in `package.json` engines
- **pnpm**: Version 10.0.0+ (managed via `packageManager` field)
- **Docker & Docker Compose**: For local PostgreSQL and Redis services
- **Git**: For version control

### Quick Start (2 minutes)

1. **Clone and Install Dependencies**

   ```bash
   git clone <repository-url>
   cd customer-portal
   pnpm install
   ```

2. **Setup Environment**

   ```bash
   # Copy development environment template (if available)
   # Note: .env files may be filtered by .cursorignore
   # Contact your team for environment configuration

   # Edit with your values (most defaults work for local development)
   # Required: DATABASE_URL, REDIS_URL, JWT_SECRET
   # Optional for basic dev: WHMCS, Salesforce, Freebit credentials
   ```

3. **Start Development Environment**

   ```bash
   # Start database and Redis services
   pnpm dev:start

   # In another terminal, start the applications with hot reload
   pnpm dev
   ```

4. **Access Your Applications**
   - **Frontend**: http://localhost:3000
   - **Backend API**: http://localhost:4000/api

### Development Commands

```bash
# === Daily Development ===
pnpm dev:start          # Start PostgreSQL + Redis services
pnpm dev                # Start both apps with hot reload
pnpm dev:stop           # Stop all services

# === Database Management ===
pnpm dev:migrate        # Run database migrations
pnpm db:studio          # Open Prisma Studio (database GUI)
pnpm dev:tools          # Start admin tools (Adminer + Redis Commander)

# === Utilities ===
pnpm dev:status         # Check service status
pnpm dev:logs           # View service logs
pnpm dev:reset          # Reset development environment
pnpm lint               # Run linting across all packages
pnpm type-check         # Run TypeScript checks
```

### Build and Export Images (for Plesk upload)

```bash
# Frontend
docker build -t customer-portal-frontend:latest -f apps/portal/Dockerfile .
docker save -o customer-portal-frontend.latest.tar customer-portal-frontend:latest

# Backend
docker build -t customer-portal-backend:latest -f apps/bff/Dockerfile .
docker save -o customer-portal-backend.latest.tar customer-portal-backend:latest
```

Upload the tar files in Plesk → Docker → Images → Upload, then deploy using the appropriate compose stack configuration.

### API Client

The portal uses TanStack Query with a lightweight fetch client that shares request/response contracts from
`@customer-portal/domain` and validates them with Zod:

```ts
import { apiClient } from "@/lib/api-client";
import type { DashboardSummary } from "@customer-portal/domain/dashboard";

const { data: summary } = useQuery({
  queryKey: ["dashboard", "summary"],
  queryFn: () => apiClient.get<DashboardSummary>("/api/me/summary"),
});
```

Because the schemas and types live in the shared domain package there is no separate
code generation step.

### Environment Configuration

- Local development: configure environment variables (contact team for template)
- Docker services use defaults: PostgreSQL (dev/dev/portal_dev), Redis (no auth)
- Plesk production: use split env files (no secrets under `httpdocs`)
  - Frontend: ensure `NEXT_PUBLIC_API_BASE=/api`
  - Backend: ensure `TRUST_PROXY=true`, DB uses `database:5432`, Redis uses `cache:6379`
  - See deployment documentation for full instructions

#### Key Environment Variables

Required environment variables (contact your team for specific values):

```env
# === Application ===
NODE_ENV=development
BFF_PORT=4000
NEXT_PORT=3000

# === Database & Cache ===
DATABASE_URL=postgresql://dev:dev@localhost:5432/portal_dev?schema=public
REDIS_URL=redis://localhost:6379

# === Frontend (exposed to browser) ===
NEXT_PUBLIC_API_BASE=http://localhost:4000
NEXT_PUBLIC_APP_NAME=Customer Portal (Dev)
NEXT_PUBLIC_ENABLE_DEVTOOLS=true

# === Security ===
JWT_SECRET=<secure_secret_minimum_32_chars>
JWT_REFRESH_SECRET=<different_secure_secret_minimum_32_chars>
BCRYPT_ROUNDS=12

# === External APIs (required for full functionality) ===
WHMCS_BASE_URL=<your_whmcs_url>
WHMCS_API_IDENTIFIER=<your_identifier>
WHMCS_API_SECRET=<your_secret>

SF_LOGIN_URL=<salesforce_instance_url>
SF_CLIENT_ID=<oauth_client_id>
SF_PRIVATE_KEY_PATH=./secrets/sf-private.key
SF_USERNAME=<salesforce_username>

FREEBIT_API_BASE_URL=<freebit_api_url>
FREEBIT_CLIENT_ID=<freebit_client_id>
FREEBIT_CLIENT_SECRET=<freebit_secret>
```

#### Salesforce Pub/Sub (Events)

```env
# Enable Pub/Sub subscription for order provisioning
SF_EVENTS_ENABLED=true
SF_PROVISION_EVENT_CHANNEL=/event/Order_Fulfilment_Requested__e
SF_EVENTS_REPLAY=LATEST        # or ALL for retention replay
SF_PUBSUB_ENDPOINT=api.pubsub.salesforce.com:7443
SF_PUBSUB_NUM_REQUESTED=25     # flow control window
```

- Verify subscriber status: `GET /api/health/sf-events`
  - `enabled`: whether Pub/Sub is enabled
  - `channel`: topic name
  - `replay.lastReplayId`: last committed cursor
  - `subscriber.status`: connected | disconnected | unknown

Read more about the provisioning flow in `docs/provisioning/RUNBOOK_PROVISIONING.md`.

### Development Tools Access

When running `pnpm dev:tools`, you get access to:

- **Adminer** (Database GUI): http://localhost:8080
  - Server: `postgres`, User: `dev`, Password: `dev`, Database: `portal_dev`
- **Redis Commander**: http://localhost:8081
  - User: `admin`, Password: `dev`

## Data Model

### Core Tables (PostgreSQL)

- `users` - Portal user accounts with auth credentials
- `id_mappings` - Cross-system user ID mappings
- `invoices_mirror` - Optional WHMCS invoice cache
- `subscriptions_mirror` - Optional WHMCS service cache
- `idempotency_keys` - Prevent duplicate operations

## API Surface (BFF)

### Authentication

- `POST /api/auth/signup` - Create portal user → WHMCS AddClient → SF upsert
- `POST /api/auth/login` - Portal authentication
- `POST /api/auth/migrate` - Account migration from legacy portal
- `POST /api/auth/set-password` - Required after WHMCS link

### User Management

- `GET /api/me` - Current user profile
- `GET /api/me/summary` - Dashboard summary
- `PATCH /api/me` - Update profile
- `PATCH /api/me/address` - Update address fields

### Catalog & Orders

- `GET /api/services/*` - Services catalog endpoints (internet/sim/vpn)
- `POST /api/orders` - WHMCS AddOrder with idempotency

### Invoices

- `GET /api/invoices` - Paginated invoice list (cached 60-120s)
- `GET /api/invoices/:id` - Invoice details
- `POST /api/invoices/:id/sso-link` - WHMCS CreateSsoToken

### Subscriptions

- `GET /api/subscriptions` - WHMCS GetClientsProducts

### Support Cases (Salesforce)

- `GET /api/cases` - Cases list (cached 30-60s)
- `GET /api/cases/:id` - Case details with comments
- `POST /api/cases` - Create new case
- `POST /api/cases/:id/comments` - Add comment to case

### Webhooks & Events

- `POST /api/webhooks/whmcs` - WHMCS action hooks → update mirrors + bust cache
- **Salesforce Platform Events** - Real-time order provisioning via gRPC Pub/Sub

## Frontend Pages

### Public Pages

- `/` - Landing page for non-authenticated users
- `/auth/login` - Sign in
- `/auth/signup` - Create account
- `/auth/set-password` - Set password after WHMCS link

### Authenticated Pages

- `/dashboard` - Dashboard (invoices, active subs, orders)
- `/catalog` - Product catalog home
- `/catalog/internet` - Internet plans
- `/catalog/vpn` - VPN products
- `/checkout` - Checkout flow
- `/orders` - Order list
- `/orders/[id]` - Order details
- `/subscriptions` - Active subscriptions
- `/subscriptions/[id]` - Subscription details
- `/billing/invoices` - Invoice list
- `/billing/invoices/[id]` - Invoice details
- `/billing/payments` - Payment methods
- `/support/cases` - Support cases list
- `/support/cases/[id]` - Case details
- `/support/new` - Create new case
- `/account` - User account management

## Development Milestones

### Milestone 1: Identity & Linking

- [x] Portal login/signup with JWT authentication
- [x] One-time WHMCS verification with SSO
- [x] Set new portal password (Bcrypt)
- [x] Store id_mappings (user ↔ WHMCS ↔ Salesforce)
- [x] Refresh token rotation
- [x] Account lockout after failed attempts
- [x] Rate limiting on auth endpoints

### Milestone 2: Catalog & Orders

- [x] Product catalog (GetProducts from WHMCS)
- [x] Catalog caching with CDC invalidation
- [x] Internet plan catalog with address verification
- [x] VPN product catalog
- [x] Checkout flow with cart management
- [x] Order creation (WHMCS AddOrder)
- [x] Order list and details from Salesforce
- [x] Real-time order provisioning via Salesforce Platform Events

### Milestone 3: Billing & Invoices

- [x] Invoice list/detail (GetInvoices from WHMCS)
- [x] Invoice caching with TTL
- [x] WHMCS SSO deep links for payment
- [x] Payment method management
- [x] Payment gateway listing
- [x] Subscription list and details
- [x] WHMCS webhooks → cache bust + mirror updates

### Milestone 4: Support & Cases

- [x] Salesforce case list (cached)
- [x] Case details with comments
- [x] Create new support case
- [x] Add comments to existing cases
- [ ] Case file attachments
- [ ] Email notifications for case updates

### Milestone 5: SIM Management & Provisioning

- [x] Freebit SIM management integration
- [x] Order provisioning workflow
- [x] Real-time event processing via Salesforce Platform Events
- [x] Comprehensive error handling and retry logic
- [ ] Customer-facing SIM management UI

## Security Features

- HTTPS only with HttpOnly/SameSite cookies
- JWT access + refresh tokens with Redis-backed blacklist
- Bcrypt password hashing (configurable rounds)
- Account lockout after failed login attempts
- Rate limiting on auth endpoints and external API calls
- Idempotency keys on all mutating operations
- Row-level security (user must own resources)
- PII minimization with encryption at rest/in transit
- Audit logging for security-critical actions
- No WHMCS/SF credentials exposed to browser

## Caching Strategy

- **Invoices**: TTL-based (90s); bust on WHMCS webhook
- **Catalog**: CDC-driven (no TTL); manual bust on data changes
- **Orders**: CDC-driven (no TTL); real-time invalidation via Salesforce Platform Events
- **WHMCS API responses**: TTL-based caching (configurable per endpoint)
- **Redis-backed** with request coalescing to prevent thundering herd
- **Keys include user_id** to prevent cross-user leakage
- **Metrics tracking** for cache hits, misses, and invalidations

## Troubleshooting

### Common Issues

**Port Already in Use**

```bash
# Check what's using the port
lsof -i :3000  # or :4000, :5432, :6379
# Kill the process or change ports in .env
```

**Database Connection Issues**

```bash
# Check if PostgreSQL is running
pnpm dev:status
# Restart services
pnpm dev:restart
# Reset everything
pnpm dev:reset
```

### Environment Variables Not Loading

- Ensure environment variables are configured (contact team for configuration)
- Restart applications after changing environment variables
- Check for typos in variable names
- Frontend variables must start with `NEXT_PUBLIC_`

**Docker Issues**

```bash
# Clean up Docker resources
docker system prune -f
# Rebuild containers
pnpm dev:stop && pnpm dev:start
```

**pnpm Issues**

```bash
# Clear pnpm cache
pnpm store prune
# Reinstall dependencies
rm -rf node_modules && pnpm install
```

### Getting Help

1. Check the logs: `pnpm dev:logs`
2. Verify service status: `pnpm dev:status`
3. Review environment configuration in `.env`
4. Check the documentation in `docs/` folder
5. Look for similar issues in the project's issue tracker

## Documentation

- **[Getting Started](docs/GETTING_STARTED.md)** - Detailed setup guide
- **[Development Guide](docs/RUN.md)** - Quick reference for daily development
- **[Deployment Guide](docs/DEPLOY.md)** - Production deployment instructions
- **[Architecture](docs/STRUCTURE.md)** - Code organization and conventions
- **[Logging](docs/LOGGING.md)** - Logging configuration and best practices
- **Portal Guides** - High-level flow, data ownership, and error handling (`docs/how-it-works/README.md`)

## Contributing

1. **Setup Development Environment**

   ```bash
   # Configure environment variables (contact team)
   pnpm install
   pnpm dev:start
   ```

2. **Follow Code Standards**
   - Run `pnpm lint` before committing
   - Use `pnpm format` to format code
   - Ensure `pnpm type-check` passes
   - Write tests for new features

3. **Development Workflow**
   - Create feature branches from `main`
   - Make small, focused commits
   - Update documentation as needed
   - Test thoroughly before submitting PRs

4. **Code Quality**
   - Follow TypeScript strict mode
   - Use proper error handling (no sensitive info exposed)
   - Implement clean, minimal UI designs
   - Avoid 'V2' suffixes in service names
   - Verify API integration against official documentation

## Codebase Coding Standard

1. Have types and validation in the shared domain layer.
2. Keep business logic out of the frontend; use services and APIs instead.
3. Reuse existing types and functions; extend them when additional behavior is needed.
4. Follow the established folder structures documented in `docs/STRUCTURE.md`.

## Documentation

📚 **[Complete Documentation](docs/README.md)** - Full documentation index

### Quick Links

- **[Getting Started](docs/GETTING_STARTED.md)** - Setup and configuration
- **[Development Commands](docs/RUN.md)** - Daily workflow
- **[Address System](docs/ADDRESS_SYSTEM.md)** - Address management
- **[Product Catalog](docs/PRODUCT-CATALOG-ARCHITECTURE.md)** - SKU-based catalog
- **[Deployment Guide](docs/DEPLOY.md)** - Production deployment

### Key Features

- ✅ **Required address at signup** - No incomplete profiles
- ✅ **Order-type specific flows** - Internet orders require verification
- ✅ **Real-time WHMCS sync** - Address updates
- ✅ **Salesforce snapshots** - Point-in-time order addresses
- ✅ **Clean architecture** - Modular, maintainable code

## License

[Your License Here]
See `docs/RUNBOOK_PROVISIONING.md` for the provisioning runbook.