Assist_Design/docs/operations/customer-data-management.md

Customer Data Management (GDPR)

This document covers procedures for handling customer data in compliance with GDPR and data protection regulations.

---

Data Storage Overview

Customer data is stored across multiple systems:

System	Data Stored	Retention	Notes
Portal (PostgreSQL)	User accounts, ID mappings, audit logs, notifications	Active account lifetime	Auth data only
WHMCS	Billing, invoices, payment methods, addresses	Legal requirement (7 years)	System of record for billing
Salesforce	CRM data, orders, cases, contacts	Business records	System of record for CRM
Redis	Sessions, cache, rate limits	TTL-based (minutes to days)	Temporary data

Portal Database Tables with PII

Table	PII Fields	Purpose
users	email, passwordHash, mfaSecret	Authentication
id_mappings	Links to WHMCS/Salesforce IDs	Identity federation
audit_logs	ipAddress, userAgent, userId	Security audit trail
residence_card_submissions	Document images	ID verification
notifications	User notifications	In-app messaging
sim_call_history_*	Phone numbers, call details	Usage records
sim_sms_history	Phone numbers, SMS details	Usage records

---

Data Subject Rights

Under GDPR, customers have the following rights:

Right	Portal Support	Notes
Right of Access	Manual export	See Data Export section
Right to Rectification	WHMCS self-service	Customer updates in WHMCS
Right to Erasure	Manual process	See Data Deletion section
Right to Portability	Manual export	See Data Export section
Right to Object	Manual process	Opt-out of processing

---

Data Deletion Procedures

Overview

Complete customer data deletion requires coordination across all systems:

1. Portal database deletion
2. WHMCS account handling
3. Salesforce record handling
4. Redis cache clearing
5. Audit trail retention

Pre-Deletion Checklist

• Verify customer identity (authentication or CS verification)
• Check for active subscriptions (must be cancelled first)
• Check for unpaid invoices (must be settled first)
• Check legal retention requirements (invoices, tax records)
• Document the deletion request with timestamp

Step 1: Portal Database Deletion

-- 1. Get user information
SELECT u.id, u.email, im.whmcs_client_id, im.sf_account_id
FROM users u
LEFT JOIN id_mappings im ON u.id = im.user_id
WHERE u.email = 'customer@example.com';

-- 2. Delete notifications
DELETE FROM notifications WHERE user_id = '<user_id>';

-- 3. Delete residence card submissions
DELETE FROM residence_card_submissions WHERE user_id = '<user_id>';

-- 4. Delete SIM usage data (if applicable)
-- Note: Check if SIM account is linked to this user first
DELETE FROM sim_usage_daily WHERE account IN (
  SELECT account FROM sim_voice_options WHERE account = '<sim_account>'
);
DELETE FROM sim_call_history_domestic WHERE account = '<sim_account>';
DELETE FROM sim_call_history_international WHERE account = '<sim_account>';
DELETE FROM sim_sms_history WHERE account = '<sim_account>';
DELETE FROM sim_voice_options WHERE account = '<sim_account>';

-- 5. Delete ID mapping (cascades from user deletion)
-- The id_mappings table has onDelete: Cascade

-- 6. Delete user (cascades audit_logs user reference to NULL, deletes id_mapping)
DELETE FROM users WHERE id = '<user_id>';

Using the Mappings Service:

// Delete mapping programmatically (clears cache too)
await mappingsService.deleteMapping(userId);

Step 2: Audit Log Handling

Audit logs may need to be retained for security compliance. Options:

Option A: Anonymize (Recommended)

-- Anonymize audit logs (keeps security trail, removes PII)
UPDATE audit_logs
SET user_id = NULL,
    ip_address = 'ANONYMIZED',
    user_agent = 'ANONYMIZED',
    details = jsonb_set(
      COALESCE(details, '{}'::jsonb),
      '{anonymized}',
      'true'::jsonb
    )
WHERE user_id = '<user_id>';

Option B: Delete (If Legally Permitted)

DELETE FROM audit_logs WHERE user_id = '<user_id>';

Step 3: Redis Cache Clearing

# Clear user-specific cache keys
redis-cli KEYS "user:*:<user_id>*" | xargs redis-cli DEL
redis-cli KEYS "session:*:<user_id>*" | xargs redis-cli DEL
redis-cli KEYS "mapping:*:<user_id>*" | xargs redis-cli DEL

# Clear refresh token families
redis-cli KEYS "refresh:user:<user_id>*" | xargs redis-cli DEL
redis-cli KEYS "refresh:family:*" | xargs redis-cli DEL  # May need filtering

# Clear rate limit records
redis-cli KEYS "auth-login:*" | xargs redis-cli DEL  # Clears by IP, not user

Step 4: WHMCS Account Handling

WHMCS does not support full account deletion. Options:

Option A: Close Account (Recommended)

1. Cancel all active services
2. Set account status to "Closed"
3. Anonymize personal fields via WHMCS Admin
4. Document closure date

Option B: Anonymize via API

# Update client to anonymized data
curl -X POST "$WHMCS_API_URL" \
  -d "identifier=$WHMCS_API_IDENTIFIER" \
  -d "secret=$WHMCS_API_SECRET" \
  -d "action=UpdateClient" \
  -d "clientid=<whmcs_client_id>" \
  -d "firstname=Deleted" \
  -d "lastname=User" \
  -d "email=deleted_<whmcs_client_id>@deleted.local" \
  -d "address1=Deleted" \
  -d "city=Deleted" \
  -d "state=Deleted" \
  -d "postcode=000-0000" \
  -d "phonenumber=000-0000-0000" \
  -d "status=Closed" \
  -d "responsetype=json"

Step 5: Salesforce Record Handling

Salesforce records often have legal retention requirements:

For Personal Data:

1. Work with Salesforce Admin
2. Consider anonymization vs deletion
3. Check integration impact (linked Orders, Cases)

Anonymization Approach:

• Update Account name to "Deleted Account - [ID]"
• Clear personal fields (phone, address if not needed)
• Keep transactional records with anonymized references

---

Data Export Procedures

Customer Data Export Request

When a customer requests their data:

1. Portal Data Export

-- Export user data
SELECT
  u.id,
  u.email,
  u.email_verified,
  u.created_at,
  u.last_login_at,
  im.whmcs_client_id,
  im.sf_account_id
FROM users u
LEFT JOIN id_mappings im ON u.id = im.user_id
WHERE u.email = 'customer@example.com';

-- Export audit log (security events)
SELECT
  action,
  resource,
  success,
  created_at
FROM audit_logs
WHERE user_id = '<user_id>'
ORDER BY created_at DESC;

-- Export notifications
SELECT
  type,
  title,
  message,
  read,
  created_at
FROM notifications
WHERE user_id = '<user_id>'
ORDER BY created_at DESC;

-- Export SIM usage history (if applicable)
SELECT
  call_date,
  call_time,
  called_to,
  duration_sec,
  charge_yen
FROM sim_call_history_domestic
WHERE account = '<sim_account>'
ORDER BY call_date DESC;

2. WHMCS Data Export

Request via WHMCS Admin:

• Client Details
• Invoices
• Services/Subscriptions
• Tickets/Support History
• Transaction History

3. Salesforce Data Export

Request via Salesforce Admin:

• Account record
• Contact record
• Order history
• Case history
• Opportunities

Export Format

Provide data in machine-readable format:

• JSON for structured data
• CSV for tabular data
• PDF for documents (invoices)

---

PII Handling During Debugging

Safe Logging Practices

The BFF uses Pino with automatic PII redaction. Sensitive fields are sanitized:

{
  "email": "cust***@example.com",
  "password": "[REDACTED]",
  "token": "[REDACTED]",
  "authorization": "[REDACTED]"
}

What NOT to Log

• Full email addresses (use masked version)
• Passwords or password hashes
• JWT tokens
• API keys or secrets
• Credit card numbers
• Full phone numbers
• Full addresses
• ID document contents

Safe Debug Queries

-- Use ID instead of email for lookups
SELECT * FROM users WHERE id = '<uuid>';

-- Mask PII in query results
SELECT
  id,
  CONCAT(LEFT(email, 3), '***', SUBSTRING(email FROM POSITION('@' IN email))) as masked_email,
  created_at
FROM users
WHERE id = '<uuid>';

Production Debugging

When investigating production issues:

1. Use correlation IDs - Search logs by request ID, not user email
2. Access minimal data - Only query what's needed
3. Document access - Note why you accessed customer data
4. Use anonymized exports - When sharing data for analysis

---

Data Retention Policies

Recommended Retention Periods

Data Type	Retention	Justification
Active user accounts	Indefinite	Active service
Closed accounts (portal)	30 days	Grace period
Audit logs	2 years	Security compliance
Session data (Redis)	24 hours	Active sessions
Rate limit data	15 minutes	Operational
Invoices	7 years	Tax/legal requirement
Support cases	5 years	Service history
Call/SMS history	6 months	Billing reconciliation

Automated Cleanup

-- Delete expired notifications (30 days after expiry)
DELETE FROM notifications
WHERE expires_at < NOW() - INTERVAL '30 days';

-- Anonymize old audit logs (over 2 years)
UPDATE audit_logs
SET ip_address = 'EXPIRED',
    user_agent = 'EXPIRED'
WHERE created_at < NOW() - INTERVAL '2 years'
  AND ip_address != 'EXPIRED';

---

Compliance Checklist

Monthly Review

• Review data access logs for unusual patterns
• Verify automated cleanup jobs are running
• Check for pending deletion requests
• Review new data collection points

Quarterly Review

• Audit third-party data sharing
• Review retention policies
• Update data inventory if schema changed
• Staff training on data handling

Annual Review

• Full data protection impact assessment
• Policy review and updates
• Vendor compliance verification
• Documentation updates

---

Emergency Data Breach Response

If a data breach is suspected:

1. Contain - Isolate affected systems
2. Assess - Determine scope and data exposed
3. Notify - Inform DPO/legal within 24 hours
4. Report - GDPR requires notification within 72 hours
5. Remediate - Fix vulnerability and prevent recurrence
6. Document - Full incident report

See Incident Response for general incident procedures.

---

Related Documents

• Incident Response
• Database Operations
• Logging Guide
• Security Monitoring

---

Last Updated: December 2025