ankhbayar/Assist_Design
1
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
Assist_Design/README.md
barsa a61c2dd68b Enhance Opportunity Management and Eligibility Handling 
- Updated SalesforceOpportunityService to allow filtering by stages during opportunity retrieval, improving flexibility in eligibility checks.
- Integrated DistributedLockService into InternetCatalogService and OrderOrchestrator to prevent race conditions when creating or reusing opportunities.
- Refactored opportunity matching logic to ensure proper handling of stages during eligibility requests and order placements.
- Improved documentation to clarify the opportunity lifecycle and eligibility verification processes, ensuring better understanding for future development.
2025-12-23 16:44:45 +09:00

546 lines
17 KiB
Markdown
Raw Blame History

# Customer Portal Project
A modern customer portal where users can self-register, log in, browse & buy subscriptions, view/pay invoices, and manage support cases.
## Architecture Overview
### Systems of Record
- **WHMCS**: Billing, subscriptions, invoices, and **authoritative address storage**
- **Salesforce**: CRM (Accounts, Contacts, Cases) and **order address snapshots**
- **Portal**: Modern UI with backend for frontend (BFF) architecture
### Identity Management
- Portal-native authentication (email + password, optional MFA)
- One-time WHMCS user verification with forced password reset
- User mapping: `user_id ↔ whmcs_client_id ↔ sf_contact_id/sf_account_id`
## Tech Stack
### Frontend (Portal UI)
- **Next.js 15** with App Router
- **Turbopack** for ultra-fast development and builds
- **React 19** with TypeScript
- **Tailwind CSS** with shadcn/ui components
- **TanStack Query** for data fetching and caching
- **Zod** for validation
- **React Hook Form** for form management
### Backend (BFF API)
- **NestJS 11** (Node 24 Current or 22 LTS)
- **Prisma 6** ORM with PostgreSQL 17
- **jsforce** for Salesforce REST API integration
- **salesforce-pubsub-api-client** for Salesforce Platform Events
- **p-queue** for request throttling and queue management
- **WHMCS** custom API client with comprehensive service layer
- **Freebit** SIM management integration
- **Zod-first validation** shared via the domain package
- **Bcrypt** for password hashing
### Queue Management
- **p-queue** for intelligent request throttling to external APIs
- Separate queues for Salesforce (standard + long-running) and WHMCS
- Configurable concurrency, rate limiting, and timeout handling
- Prevents API rate limit violations and resource exhaustion
### Logging
- Centralized structured logging via Pino using `nestjs-pino` in the BFF
- Sensitive fields are redacted; each request has a correlation ID
- Usage pattern in services:
- Inject `Logger` from `nestjs-pino`: `constructor(@Inject(Logger) private readonly logger: Logger) {}`
- Log with structured objects: `this.logger.error('Message', { error })`
- See `docs/LOGGING.md` for full guidelines
### Data & Infrastructure
- **PostgreSQL 17** for users, ID mappings, and optional mirrors
- **Redis 7** for cache, token blacklists, and rate limiting
- **Docker** for local development (Postgres/Redis)
## Project Structure
```
customer-portal/
├── apps/
│ ├── portal/ # Next.js 15 frontend (React 19, Tailwind, shadcn/ui)
│ └── bff/ # NestJS 11 backend (Prisma, p-queue, Zod validation)
├── packages/
│ ├── domain/ # Unified domain layer with contracts & schemas
│ ├── validation/ # Unified validation service (NestJS + React)
│ ├── logging/ # Centralized logging utilities (Pino)
│ └── integrations/ # External service integrations
│ ├── whmcs/ # WHMCS API client
│ └── freebit/ # Freebit SIM management
├── scripts/
│ ├── dev/ # Development management scripts
│ └── prod/ # Production deployment scripts
├── docker/
│ └── dev/ # Docker Compose for local development
│ └── docker-compose.yml # PostgreSQL 17 + Redis 7
├── docs/ # Comprehensive documentation
├── secrets/ # Private keys (git ignored)
├── env/ # Environment file templates
├── package.json # Root workspace configuration
├── pnpm-workspace.yaml # pnpm workspace definition
└── README.md # This file
```
## Getting Started
### Prerequisites
- **Node.js**: Version 22 (LTS) or 24 (Current) - specified in `package.json` engines
- **pnpm**: Version 10.0.0+ (managed via `packageManager` field)
- **Docker & Docker Compose**: For local PostgreSQL and Redis services
- **Git**: For version control
### Quick Start (2 minutes)
1. **Clone and Install Dependencies**
```bash
git clone <repository-url>
cd customer-portal
pnpm install
```
2. **Setup Environment**
```bash
# Copy development environment template (if available)
# Note: .env files may be filtered by .cursorignore
# Contact your team for environment configuration
# Edit with your values (most defaults work for local development)
# Required: DATABASE_URL, REDIS_URL, JWT_SECRET
# Optional for basic dev: WHMCS, Salesforce, Freebit credentials
```
3. **Start Development Environment**
```bash
# Start database and Redis services
pnpm dev:start
# In another terminal, start the applications with hot reload
pnpm dev
```
4. **Access Your Applications**
- **Frontend**: http://localhost:3000
- **Backend API**: http://localhost:4000/api
### Development Commands
```bash
# === Daily Development ===
pnpm dev:start # Start PostgreSQL + Redis services
pnpm dev # Start both apps with hot reload
pnpm dev:stop # Stop all services
# === Database Management ===
pnpm dev:migrate # Run database migrations
pnpm db:studio # Open Prisma Studio (database GUI)
pnpm dev:tools # Start admin tools (Adminer + Redis Commander)
# === Utilities ===
pnpm dev:status # Check service status
pnpm dev:logs # View service logs
pnpm dev:reset # Reset development environment
pnpm lint # Run linting across all packages
pnpm type-check # Run TypeScript checks
```
### Build and Export Images (for Plesk upload)
```bash
# Frontend
docker build -t customer-portal-frontend:latest -f apps/portal/Dockerfile .
docker save -o customer-portal-frontend.latest.tar customer-portal-frontend:latest
# Backend
docker build -t customer-portal-backend:latest -f apps/bff/Dockerfile .
docker save -o customer-portal-backend.latest.tar customer-portal-backend:latest
```
Upload the tar files in Plesk → Docker → Images → Upload, then deploy using the appropriate compose stack configuration.
### API Client
The portal uses TanStack Query with a lightweight fetch client that shares request/response contracts from
`@customer-portal/domain` and validates them with Zod:
```ts
import { apiClient } from "@/lib/api-client";
import type { DashboardSummary } from "@customer-portal/domain/dashboard";
const { data: summary } = useQuery({
queryKey: ["dashboard", "summary"],
queryFn: () => apiClient.get<DashboardSummary>("/api/me/summary"),
});
```
Because the schemas and types live in the shared domain package there is no separate
code generation step.
### Environment Configuration
- Local development: configure environment variables (contact team for template)
- Docker services use defaults: PostgreSQL (dev/dev/portal_dev), Redis (no auth)
- Plesk production: use split env files (no secrets under `httpdocs`)
- Frontend: ensure `NEXT_PUBLIC_API_BASE=/api`
- Backend: ensure `TRUST_PROXY=true`, DB uses `database:5432`, Redis uses `cache:6379`
- See deployment documentation for full instructions
#### Key Environment Variables
Required environment variables (contact your team for specific values):
```env
# === Application ===
NODE_ENV=development
BFF_PORT=4000
NEXT_PORT=3000
# === Database & Cache ===
DATABASE_URL=postgresql://dev:dev@localhost:5432/portal_dev?schema=public
REDIS_URL=redis://localhost:6379
# === Frontend (exposed to browser) ===
NEXT_PUBLIC_API_BASE=http://localhost:4000
NEXT_PUBLIC_APP_NAME=Customer Portal (Dev)
NEXT_PUBLIC_ENABLE_DEVTOOLS=true
# === Security ===
JWT_SECRET=<secure_secret_minimum_32_chars>
JWT_REFRESH_SECRET=<different_secure_secret_minimum_32_chars>
BCRYPT_ROUNDS=12
# === External APIs (required for full functionality) ===
WHMCS_BASE_URL=<your_whmcs_url>
WHMCS_API_IDENTIFIER=<your_identifier>
WHMCS_API_SECRET=<your_secret>
SF_LOGIN_URL=<salesforce_instance_url>
SF_CLIENT_ID=<oauth_client_id>
SF_PRIVATE_KEY_PATH=./secrets/sf-private.key
SF_USERNAME=<salesforce_username>
FREEBIT_API_BASE_URL=<freebit_api_url>
FREEBIT_CLIENT_ID=<freebit_client_id>
FREEBIT_CLIENT_SECRET=<freebit_secret>
```
#### Salesforce Pub/Sub (Events)
```env
# Enable Pub/Sub subscription for order provisioning
SF_EVENTS_ENABLED=true
SF_PROVISION_EVENT_CHANNEL=/event/Order_Fulfilment_Requested__e
SF_EVENTS_REPLAY=LATEST # or ALL for retention replay
SF_PUBSUB_ENDPOINT=api.pubsub.salesforce.com:7443
SF_PUBSUB_NUM_REQUESTED=25 # flow control window
```
- Verify subscriber status: `GET /api/health/sf-events`
- `enabled`: whether Pub/Sub is enabled
- `channel`: topic name
- `replay.lastReplayId`: last committed cursor
- `subscriber.status`: connected | disconnected | unknown
Read more about the provisioning flow in `docs/provisioning/RUNBOOK_PROVISIONING.md`.
### Development Tools Access
When running `pnpm dev:tools`, you get access to:
- **Adminer** (Database GUI): http://localhost:8080
- Server: `postgres`, User: `dev`, Password: `dev`, Database: `portal_dev`
- **Redis Commander**: http://localhost:8081
- User: `admin`, Password: `dev`
## Data Model
### Core Tables (PostgreSQL)
- `users` - Portal user accounts with auth credentials
- `id_mappings` - Cross-system user ID mappings
- `invoices_mirror` - Optional WHMCS invoice cache
- `subscriptions_mirror` - Optional WHMCS service cache
- `idempotency_keys` - Prevent duplicate operations
## API Surface (BFF)
### Authentication
- `POST /api/auth/signup` - Create portal user → WHMCS AddClient → SF upsert
- `POST /api/auth/login` - Portal authentication
- `POST /api/auth/migrate` - Account migration from legacy portal
- `POST /api/auth/set-password` - Required after WHMCS link
### User Management
- `GET /api/me` - Current user profile
- `GET /api/me/summary` - Dashboard summary
- `PATCH /api/me` - Update profile
- `PATCH /api/me/address` - Update address fields
### Catalog & Orders
- `GET /api/catalog` - WHMCS GetProducts (cached 5-15m)
- `POST /api/orders` - WHMCS AddOrder with idempotency
### Invoices
- `GET /api/invoices` - Paginated invoice list (cached 60-120s)
- `GET /api/invoices/:id` - Invoice details
- `POST /api/invoices/:id/sso-link` - WHMCS CreateSsoToken
### Subscriptions
- `GET /api/subscriptions` - WHMCS GetClientsProducts
### Support Cases (Salesforce)
- `GET /api/cases` - Cases list (cached 30-60s)
- `GET /api/cases/:id` - Case details with comments
- `POST /api/cases` - Create new case
- `POST /api/cases/:id/comments` - Add comment to case
### Webhooks & Events
- `POST /api/webhooks/whmcs` - WHMCS action hooks → update mirrors + bust cache
- **Salesforce Platform Events** - Real-time order provisioning via gRPC Pub/Sub
## Frontend Pages
### Public Pages
- `/` - Landing page for non-authenticated users
- `/auth/login` - Sign in
- `/auth/signup` - Create account
- `/auth/set-password` - Set password after WHMCS link
### Authenticated Pages
- `/dashboard` - Dashboard (invoices, active subs, orders)
- `/catalog` - Product catalog home
- `/catalog/internet` - Internet plans
- `/catalog/vpn` - VPN products
- `/checkout` - Checkout flow
- `/orders` - Order list
- `/orders/[id]` - Order details
- `/subscriptions` - Active subscriptions
- `/subscriptions/[id]` - Subscription details
- `/billing/invoices` - Invoice list
- `/billing/invoices/[id]` - Invoice details
- `/billing/payments` - Payment methods
- `/support/cases` - Support cases list
- `/support/cases/[id]` - Case details
- `/support/new` - Create new case
- `/account` - User account management
## Development Milestones
### Milestone 1: Identity & Linking
- [x] Portal login/signup with JWT authentication
- [x] One-time WHMCS verification with SSO
- [x] Set new portal password (Bcrypt)
- [x] Store id_mappings (user ↔ WHMCS ↔ Salesforce)
- [x] Refresh token rotation
- [x] Account lockout after failed attempts
- [x] Rate limiting on auth endpoints
### Milestone 2: Catalog & Orders
- [x] Product catalog (GetProducts from WHMCS)
- [x] Catalog caching with CDC invalidation
- [x] Internet plan catalog with address verification
- [x] VPN product catalog
- [x] Checkout flow with cart management
- [x] Order creation (WHMCS AddOrder)
- [x] Order list and details from Salesforce
- [x] Real-time order provisioning via Salesforce Platform Events
### Milestone 3: Billing & Invoices
- [x] Invoice list/detail (GetInvoices from WHMCS)
- [x] Invoice caching with TTL
- [x] WHMCS SSO deep links for payment
- [x] Payment method management
- [x] Payment gateway listing
- [x] Subscription list and details
- [x] WHMCS webhooks → cache bust + mirror updates
### Milestone 4: Support & Cases
- [x] Salesforce case list (cached)
- [x] Case details with comments
- [x] Create new support case
- [x] Add comments to existing cases
- [ ] Case file attachments
- [ ] Email notifications for case updates
### Milestone 5: SIM Management & Provisioning
- [x] Freebit SIM management integration
- [x] Order provisioning workflow
- [x] Real-time event processing via Salesforce Platform Events
- [x] Comprehensive error handling and retry logic
- [ ] Customer-facing SIM management UI
## Security Features
- HTTPS only with HttpOnly/SameSite cookies
- JWT access + refresh tokens with Redis-backed blacklist
- Bcrypt password hashing (configurable rounds)
- Account lockout after failed login attempts
- Rate limiting on auth endpoints and external API calls
- Idempotency keys on all mutating operations
- Row-level security (user must own resources)
- PII minimization with encryption at rest/in transit
- Audit logging for security-critical actions
- No WHMCS/SF credentials exposed to browser
## Caching Strategy
- **Invoices**: TTL-based (90s); bust on WHMCS webhook
- **Catalog**: CDC-driven (no TTL); manual bust on data changes
- **Orders**: CDC-driven (no TTL); real-time invalidation via Salesforce Platform Events
- **WHMCS API responses**: TTL-based caching (configurable per endpoint)
- **Redis-backed** with request coalescing to prevent thundering herd
- **Keys include user_id** to prevent cross-user leakage
- **Metrics tracking** for cache hits, misses, and invalidations
## Troubleshooting
### Common Issues
**Port Already in Use**
```bash
# Check what's using the port
lsof -i :3000 # or :4000, :5432, :6379
# Kill the process or change ports in .env
```
**Database Connection Issues**
```bash
# Check if PostgreSQL is running
pnpm dev:status
# Restart services
pnpm dev:restart
# Reset everything
pnpm dev:reset
```
### Environment Variables Not Loading
- Ensure environment variables are configured (contact team for configuration)
- Restart applications after changing environment variables
- Check for typos in variable names
- Frontend variables must start with `NEXT_PUBLIC_`
**Docker Issues**
```bash
# Clean up Docker resources
docker system prune -f
# Rebuild containers
pnpm dev:stop && pnpm dev:start
```
**pnpm Issues**
```bash
# Clear pnpm cache
pnpm store prune
# Reinstall dependencies
rm -rf node_modules && pnpm install
```
### Getting Help
1. Check the logs: `pnpm dev:logs`
2. Verify service status: `pnpm dev:status`
3. Review environment configuration in `.env`
4. Check the documentation in `docs/` folder
5. Look for similar issues in the project's issue tracker
## Documentation
- **[Getting Started](docs/GETTING_STARTED.md)** - Detailed setup guide
- **[Development Guide](docs/RUN.md)** - Quick reference for daily development
- **[Deployment Guide](docs/DEPLOY.md)** - Production deployment instructions
- **[Architecture](docs/STRUCTURE.md)** - Code organization and conventions
- **[Logging](docs/LOGGING.md)** - Logging configuration and best practices
- **Portal Guides** - High-level flow, data ownership, and error handling (`docs/portal-guides/README.md`)
## Contributing
1. **Setup Development Environment**
```bash
# Configure environment variables (contact team)
pnpm install
pnpm dev:start
```
2. **Follow Code Standards**
- Run `pnpm lint` before committing
- Use `pnpm format` to format code
- Ensure `pnpm type-check` passes
- Write tests for new features
3. **Development Workflow**
- Create feature branches from `main`
- Make small, focused commits
- Update documentation as needed
- Test thoroughly before submitting PRs
4. **Code Quality**
- Follow TypeScript strict mode
- Use proper error handling (no sensitive info exposed)
- Implement clean, minimal UI designs
- Avoid 'V2' suffixes in service names
- Verify API integration against official documentation
## Codebase Coding Standard
1. Have types and validation in the shared domain layer.
2. Keep business logic out of the frontend; use services and APIs instead.
3. Reuse existing types and functions; extend them when additional behavior is needed.
4. Follow the established folder structures documented in `docs/STRUCTURE.md`.
## Documentation
📚 **[Complete Documentation](docs/README.md)** - Full documentation index
### Quick Links
- **[Getting Started](docs/GETTING_STARTED.md)** - Setup and configuration
- **[Development Commands](docs/RUN.md)** - Daily workflow
- **[Address System](docs/ADDRESS_SYSTEM.md)** - Address management
- **[Product Catalog](docs/PRODUCT-CATALOG-ARCHITECTURE.md)** - SKU-based catalog
- **[Deployment Guide](docs/DEPLOY.md)** - Production deployment
### Key Features
- ✅ **Required address at signup** - No incomplete profiles
- ✅ **Order-type specific flows** - Internet orders require verification
- ✅ **Real-time WHMCS sync** - Address updates
- ✅ **Salesforce snapshots** - Point-in-time order addresses
- ✅ **Clean architecture** - Modular, maintainable code
## License
[Your License Here]
See `docs/RUNBOOK_PROVISIONING.md` for the provisioning runbook.
Reference in New Issue View Git Blame Copy Permalink