Assist_Design/docs/operations/customer-data-management.md

# Customer Data Management (GDPR)

This document covers procedures for handling customer data in compliance with GDPR and data protection regulations.

---

## Data Storage Overview

Customer data is stored across multiple systems:

| System                  | Data Stored                                           | Retention                   | Notes                        |
| ----------------------- | ----------------------------------------------------- | --------------------------- | ---------------------------- |
| **Portal (PostgreSQL)** | User accounts, ID mappings, audit logs, notifications | Active account lifetime     | Auth data only               |
| **WHMCS**               | Billing, invoices, payment methods, addresses         | Legal requirement (7 years) | System of record for billing |
| **Salesforce**          | CRM data, orders, cases, contacts                     | Business records            | System of record for CRM     |
| **Redis**               | Sessions, cache, rate limits                          | TTL-based (minutes to days) | Temporary data               |

### Portal Database Tables with PII

| Table                        | PII Fields                           | Purpose              |
| ---------------------------- | ------------------------------------ | -------------------- |
| `users`                      | `email`, `passwordHash`, `mfaSecret` | Authentication       |
| `id_mappings`                | Links to WHMCS/Salesforce IDs        | Identity federation  |
| `audit_logs`                 | `ipAddress`, `userAgent`, `userId`   | Security audit trail |
| `residence_card_submissions` | Document images                      | ID verification      |
| `notifications`              | User notifications                   | In-app messaging     |
| `sim_call_history_*`         | Phone numbers, call details          | Usage records        |
| `sim_sms_history`            | Phone numbers, SMS details           | Usage records        |

---

## Data Subject Rights

Under GDPR, customers have the following rights:

| Right                  | Portal Support     | Notes                     |
| ---------------------- | ------------------ | ------------------------- |
| Right of Access        | Manual export      | See Data Export section   |
| Right to Rectification | WHMCS self-service | Customer updates in WHMCS |
| Right to Erasure       | Manual process     | See Data Deletion section |
| Right to Portability   | Manual export      | See Data Export section   |
| Right to Object        | Manual process     | Opt-out of processing     |

---

## Data Deletion Procedures

### Overview

Complete customer data deletion requires coordination across all systems:

1. Portal database deletion
2. WHMCS account handling
3. Salesforce record handling
4. Redis cache clearing
5. Audit trail retention

### Pre-Deletion Checklist

- [ ] Verify customer identity (authentication or CS verification)
- [ ] Check for active subscriptions (must be cancelled first)
- [ ] Check for unpaid invoices (must be settled first)
- [ ] Check legal retention requirements (invoices, tax records)
- [ ] Document the deletion request with timestamp

### Step 1: Portal Database Deletion

```sql
-- 1. Get user information
SELECT u.id, u.email, im.whmcs_client_id, im.sf_account_id
FROM users u
LEFT JOIN id_mappings im ON u.id = im.user_id
WHERE u.email = 'customer@example.com';

-- 2. Delete notifications
DELETE FROM notifications WHERE user_id = '<user_id>';

-- 3. Delete residence card submissions
DELETE FROM residence_card_submissions WHERE user_id = '<user_id>';

-- 4. Delete SIM usage data (if applicable)
-- Note: Check if SIM account is linked to this user first
DELETE FROM sim_usage_daily WHERE account IN (
  SELECT account FROM sim_voice_options WHERE account = '<sim_account>'
);
DELETE FROM sim_call_history_domestic WHERE account = '<sim_account>';
DELETE FROM sim_call_history_international WHERE account = '<sim_account>';
DELETE FROM sim_sms_history WHERE account = '<sim_account>';
DELETE FROM sim_voice_options WHERE account = '<sim_account>';

-- 5. Delete ID mapping (cascades from user deletion)
-- The id_mappings table has onDelete: Cascade

-- 6. Delete user (cascades audit_logs user reference to NULL, deletes id_mapping)
DELETE FROM users WHERE id = '<user_id>';
```

**Using the Mappings Service:**

```typescript
// Delete mapping programmatically (clears cache too)
await mappingsService.deleteMapping(userId);
```

### Step 2: Audit Log Handling

Audit logs may need to be retained for security compliance. Options:

**Option A: Anonymize (Recommended)**

```sql
-- Anonymize audit logs (keeps security trail, removes PII)
UPDATE audit_logs
SET user_id = NULL,
    ip_address = 'ANONYMIZED',
    user_agent = 'ANONYMIZED',
    details = jsonb_set(
      COALESCE(details, '{}'::jsonb),
      '{anonymized}',
      'true'::jsonb
    )
WHERE user_id = '<user_id>';
```

**Option B: Delete (If Legally Permitted)**

```sql
DELETE FROM audit_logs WHERE user_id = '<user_id>';
```

### Step 3: Redis Cache Clearing

```bash
# Clear user-specific cache keys
redis-cli KEYS "user:*:<user_id>*" | xargs redis-cli DEL
redis-cli KEYS "session:*:<user_id>*" | xargs redis-cli DEL
redis-cli KEYS "mapping:*:<user_id>*" | xargs redis-cli DEL

# Clear refresh token families
redis-cli KEYS "refresh:user:<user_id>*" | xargs redis-cli DEL
redis-cli KEYS "refresh:family:*" | xargs redis-cli DEL  # May need filtering

# Clear rate limit records
redis-cli KEYS "auth-login:*" | xargs redis-cli DEL  # Clears by IP, not user
```

### Step 4: WHMCS Account Handling

WHMCS does not support full account deletion. Options:

**Option A: Close Account (Recommended)**

1. Cancel all active services
2. Set account status to "Closed"
3. Anonymize personal fields via WHMCS Admin
4. Document closure date

**Option B: Anonymize via API**

```bash
# Update client to anonymized data
curl -X POST "$WHMCS_API_URL" \
  -d "identifier=$WHMCS_API_IDENTIFIER" \
  -d "secret=$WHMCS_API_SECRET" \
  -d "action=UpdateClient" \
  -d "clientid=<whmcs_client_id>" \
  -d "firstname=Deleted" \
  -d "lastname=User" \
  -d "email=deleted_<whmcs_client_id>@deleted.local" \
  -d "address1=Deleted" \
  -d "city=Deleted" \
  -d "state=Deleted" \
  -d "postcode=000-0000" \
  -d "phonenumber=000-0000-0000" \
  -d "status=Closed" \
  -d "responsetype=json"
```

### Step 5: Salesforce Record Handling

Salesforce records often have legal retention requirements:

**For Personal Data:**

1. Work with Salesforce Admin
2. Consider anonymization vs deletion
3. Check integration impact (linked Orders, Cases)

**Anonymization Approach:**

- Update Account name to "Deleted Account - [ID]"
- Clear personal fields (phone, address if not needed)
- Keep transactional records with anonymized references

---

## Data Export Procedures

### Customer Data Export Request

When a customer requests their data:

#### 1. Portal Data Export

```sql
-- Export user data
SELECT
  u.id,
  u.email,
  u.email_verified,
  u.created_at,
  u.last_login_at,
  im.whmcs_client_id,
  im.sf_account_id
FROM users u
LEFT JOIN id_mappings im ON u.id = im.user_id
WHERE u.email = 'customer@example.com';

-- Export audit log (security events)
SELECT
  action,
  resource,
  success,
  created_at
FROM audit_logs
WHERE user_id = '<user_id>'
ORDER BY created_at DESC;

-- Export notifications
SELECT
  type,
  title,
  message,
  read,
  created_at
FROM notifications
WHERE user_id = '<user_id>'
ORDER BY created_at DESC;

-- Export SIM usage history (if applicable)
SELECT
  call_date,
  call_time,
  called_to,
  duration_sec,
  charge_yen
FROM sim_call_history_domestic
WHERE account = '<sim_account>'
ORDER BY call_date DESC;
```

#### 2. WHMCS Data Export

Request via WHMCS Admin:

- Client Details
- Invoices
- Services/Subscriptions
- Tickets/Support History
- Transaction History

#### 3. Salesforce Data Export

Request via Salesforce Admin:

- Account record
- Contact record
- Order history
- Case history
- Opportunities

### Export Format

Provide data in machine-readable format:

- JSON for structured data
- CSV for tabular data
- PDF for documents (invoices)

---

## PII Handling During Debugging

### Safe Logging Practices

The BFF uses Pino with automatic PII redaction. Sensitive fields are sanitized:

```json
{
  "email": "cust***@example.com",
  "password": "[REDACTED]",
  "token": "[REDACTED]",
  "authorization": "[REDACTED]"
}
```

### What NOT to Log

- Full email addresses (use masked version)
- Passwords or password hashes
- JWT tokens
- API keys or secrets
- Credit card numbers
- Full phone numbers
- Full addresses
- ID document contents

### Safe Debug Queries

```sql
-- Use ID instead of email for lookups
SELECT * FROM users WHERE id = '<uuid>';

-- Mask PII in query results
SELECT
  id,
  CONCAT(LEFT(email, 3), '***', SUBSTRING(email FROM POSITION('@' IN email))) as masked_email,
  created_at
FROM users
WHERE id = '<uuid>';
```

### Production Debugging

When investigating production issues:

1. **Use correlation IDs** - Search logs by request ID, not user email
2. **Access minimal data** - Only query what's needed
3. **Document access** - Note why you accessed customer data
4. **Use anonymized exports** - When sharing data for analysis

---

## Data Retention Policies

### Recommended Retention Periods

| Data Type                | Retention  | Justification          |
| ------------------------ | ---------- | ---------------------- |
| Active user accounts     | Indefinite | Active service         |
| Closed accounts (portal) | 30 days    | Grace period           |
| Audit logs               | 2 years    | Security compliance    |
| Session data (Redis)     | 24 hours   | Active sessions        |
| Rate limit data          | 15 minutes | Operational            |
| Invoices                 | 7 years    | Tax/legal requirement  |
| Support cases            | 5 years    | Service history        |
| Call/SMS history         | 6 months   | Billing reconciliation |

### Automated Cleanup

```sql
-- Delete expired notifications (30 days after expiry)
DELETE FROM notifications
WHERE expires_at < NOW() - INTERVAL '30 days';

-- Anonymize old audit logs (over 2 years)
UPDATE audit_logs
SET ip_address = 'EXPIRED',
    user_agent = 'EXPIRED'
WHERE created_at < NOW() - INTERVAL '2 years'
  AND ip_address != 'EXPIRED';
```

---

## Compliance Checklist

### Monthly Review

- [ ] Review data access logs for unusual patterns
- [ ] Verify automated cleanup jobs are running
- [ ] Check for pending deletion requests
- [ ] Review new data collection points

### Quarterly Review

- [ ] Audit third-party data sharing
- [ ] Review retention policies
- [ ] Update data inventory if schema changed
- [ ] Staff training on data handling

### Annual Review

- [ ] Full data protection impact assessment
- [ ] Policy review and updates
- [ ] Vendor compliance verification
- [ ] Documentation updates

---

## Emergency Data Breach Response

If a data breach is suspected:

1. **Contain** - Isolate affected systems
2. **Assess** - Determine scope and data exposed
3. **Notify** - Inform DPO/legal within 24 hours
4. **Report** - GDPR requires notification within 72 hours
5. **Remediate** - Fix vulnerability and prevent recurrence
6. **Document** - Full incident report

See [Incident Response](./incident-response.md) for general incident procedures.

---

## Related Documents

- [Incident Response](./incident-response.md)
- [Database Operations](./database-operations.md)
- [Logging Guide](./logging.md)
- [Security Monitoring](./security-monitoring.md)

---

**Last Updated:** December 2025