Tech Debt Cleanup Plan (TD-006)

Created: 2024-12-24 Updated: 2024-12-24 Status: In Progress (Phase 0 Complete) Priority: P1 (Pre-Launch Blocker)

---

Overview

The tech debt audit identified 35 production blockers - external service stubs that return mock data instead of calling real APIs. These must be implemented before production launch.

Non-API Fixes (Completed 2024-12-24)

Four items were identified that could be fixed without external API credentials:

Item	File	Status
Consent route client lookup	src/api/routes/consent.py	Fixed - Fetches real client data from ClientRepository
AI QA citation resolution	src/workflows/review/ai_qa_workflow.py	Fixed - Resolves document names to actual IDs via DocumentRepository
EFiling ready checks	src/api/routes/efiling.py	Fixed - Checks Form 8879 signatures and document checklist
Placeholder PDF generation	src/workflows/delivery/package_workflow.py	Fixed - Uses ReportLab for proper PDF generation

Additional fixes during implementation: - Fixed ChecklistRepository abstract method implementation (was missing _row_to_entity and _entity_to_row) - Added reportlab>=4.0.0 to requirements.txt

All 1522 tests passing after these changes.

Summary by Service (API Credentials Required)

Service	TODOs	Priority	Dependencies
EmailService	4	P0	AWS SES credentials
SMSService	3	P0	Twilio credentials
PersonaService	5	P1	Persona API key, template ID
SmartVaultService	9	P1	OAuth client credentials
SurePrepService	9	P1	API credentials
GoogleService	4	P1	Service account JSON
Webhook Verification	2	P2	Signing secrets from each provider

---

Phase 1: Communication Services (EmailService + SMSService)

Why First: Every other workflow depends on notifications. Registration, verification, reminders, invoices - all need email/SMS.

EmailService (AWS SES)

File: src/services/email_service.py

Location	Current	Implementation
Line 64	TODO: health check	Verify SES connectivity with ses.get_send_quota()
Line 191	TODO: SES sending	Implement ses.send_email() with template support
registration.py:125	TODO: send verification	Call email_service.send_verification_email()
registration.py:235,327	TODO: send email	Call email service for resend/activation

Implementation Steps: 1. Add aiobotocore SES client initialization 2. Implement _send_email() with retry logic 3. Add email templates (verification, welcome, notification) 4. Wire up registration routes to call email service 5. Add integration tests with SES sandbox

Config Required:

# config.yaml
email:
  # sender_email: Verified SES sender address
  sender_email: ${EMAIL_SENDER:-noreply@taxpractice.ai}

  # sender_name: Display name in From header
  sender_name: "Tax Practice AI"

  # aws_region: SES region (must have verified domain)
  aws_region: ${AWS_REGION:-us-east-1}

  # dry_run: Log instead of sending (for development)
  dry_run: ${EMAIL_DRY_RUN:-true}

Estimated Effort: 3-4 hours

---

SMSService (Twilio)

File: src/services/sms_service.py

Location	Current	Implementation
Line 105	TODO: health check	Verify Twilio credentials with account lookup
Line 313	TODO: SMS sending	Implement twilio.messages.create()
Line 362	TODO: voice calling	Implement twilio.calls.create() for voice codes

Implementation Steps: 1. Add twilio async client initialization 2. Implement send_sms() with rate limiting 3. Implement send_voice_code() for accessibility 4. Add phone number validation/formatting (E.164) 5. Add integration tests with Twilio test credentials

Config Required:

# config.yaml
sms:
  # account_sid: Twilio account SID
  account_sid: ${TWILIO_ACCOUNT_SID}

  # auth_token: Twilio auth token
  auth_token: ${TWILIO_AUTH_TOKEN}

  # from_number: Twilio phone number (E.164 format)
  from_number: ${TWILIO_FROM_NUMBER}

  # dry_run: Log instead of sending (for development)
  dry_run: ${SMS_DRY_RUN:-true}

Estimated Effort: 2-3 hours

---

Phase 2: Identity Verification (PersonaService)

Why Second: Client onboarding requires identity verification (S2-003). Blocks new client registration flow.

File: src/services/persona_service.py

Location	Current	Implementation
Line 91	TODO: health check	Verify Persona API key with GET /api/v1/accounts/me
Line 124	TODO: create inquiry	POST /api/v1/inquiries with template ID
Line 158	TODO: get inquiry	GET /api/v1/inquiries/{id} for status
Line 179	TODO: resume inquiry	POST /api/v1/inquiries/{id}/resume
Line 225	TODO: verify webhook	HMAC-SHA256 signature verification

Implementation Steps: 1. Add httpx client with Persona base URL and API key 2. Implement create_inquiry() - creates verification session 3. Implement get_inquiry_status() - polls for completion 4. Implement verify_webhook_signature() - HMAC validation 5. Connect webhook handler to update verification status 6. Add integration tests with Persona sandbox

Config Required:

# config.yaml
persona:
  # api_key: Persona API key (starts with persona_)
  api_key: ${PERSONA_API_KEY}

  # template_id: Inquiry template ID for tax practice
  template_id: ${PERSONA_TEMPLATE_ID}

  # webhook_secret: HMAC signing secret for webhooks
  webhook_secret: ${PERSONA_WEBHOOK_SECRET}

  # environment: sandbox or production
  environment: ${PERSONA_ENV:-sandbox}

Estimated Effort: 4-5 hours

---

Phase 3: Document Portal (SmartVaultService)

Why Third: Document management (S4) requires SmartVault sync. Clients upload to SmartVault, we fetch and process.

File: src/services/smartvault_service.py

Location	Current	Implementation
Line 280	TODO: token exchange	OAuth 2.0 authorization code → access token
Line 301	TODO: token refresh	Refresh expired access tokens
Line 339	TODO: list documents	GET /nodes/{path}/children
Line 366	TODO: download document	GET /nodes/{id}/content
Line 418	TODO: upload document	PUT /nodes/{path}/content
Line 444	TODO: delete document	DELETE /nodes/{id}
Line 496	TODO: create folder	POST /nodes/{path} with type=folder
Line 522	TODO: list folders	GET /nodes/{path}/children?type=folder
Line 565	TODO: sync client	Full sync of client folder

Implementation Steps: 1. Implement OAuth 2.0 flow (authorization URL, token exchange, refresh) 2. Add token storage (encrypted in database or secrets manager) 3. Implement CRUD operations for documents and folders 4. Implement client folder sync workflow 5. Add webhook handler for real-time document notifications 6. Add integration tests with SmartVault sandbox

Config Required:

# config.yaml
smartvault:
  # base_url: SmartVault API base URL
  base_url: https://rest.smartvault.com

  # client_id: OAuth client ID
  client_id: ${SMARTVAULT_CLIENT_ID}

  # client_secret: OAuth client secret
  client_secret: ${SMARTVAULT_CLIENT_SECRET}

  # account_id: SmartVault account ID
  account_id: ${SMARTVAULT_ACCOUNT_ID}

  # redirect_uri: OAuth callback URL
  redirect_uri: ${SMARTVAULT_REDIRECT_URI}

Estimated Effort: 6-8 hours

---

Phase 4: Document Processing (SurePrepService)

Why Fourth: After documents are in the system, SurePrep handles OCR/extraction. Critical for tax data pipeline.

File: src/services/sureprep_service.py

Location	Current	Implementation
Line 339	TODO: create binder	POST /binders - create processing container
Line 381	TODO: get binder	GET /binders/{id} - status and metadata
Line 412	TODO: list binders	GET /binders with filters
Line 441	TODO: submit binder	POST /binders/{id}/submit - start processing
Line 510	TODO: upload document	POST /binders/{id}/documents
Line 530	TODO: get document	GET /binders/{id}/documents/{doc_id}
Line 549	TODO: get extractions	GET /binders/{id}/extractions
Line 574	TODO: get extraction detail	GET /extractions/{id}
Line 718	TODO: CS Connect sync	Sync data to UltraTax via CS Connect

Implementation Steps: 1. Add HTTP Basic auth + API key authentication 2. Implement binder lifecycle (create, submit, poll status) 3. Implement document upload to binders 4. Implement extraction data retrieval 5. Add webhook handler for processing notifications 6. Implement CS Connect sync for UltraTax (if API available) 7. Add integration tests with SurePrep sandbox

Config Required:

# config.yaml
sureprep:
  # base_url: SurePrep API base URL
  base_url: https://api.sureprep.com

  # username: API username
  username: ${SUREPREP_USERNAME}

  # password: API password
  password: ${SUREPREP_PASSWORD}

  # api_key: API key for additional auth
  api_key: ${SUREPREP_API_KEY}

  # firm_id: SurePrep firm identifier
  firm_id: ${SUREPREP_FIRM_ID}

Estimated Effort: 8-10 hours

---

Phase 5: E-Signatures (GoogleService)

Why Fifth: Client delivery (S9) requires document signing via Google Workspace.

File: src/services/google_service.py

Location	Current	Implementation
Line 152	TODO: health check	Verify service account credentials
Line 194	TODO: create document	Google Docs API - create from template
Line 332	TODO: verify webhook	Verify Google webhook signatures
Line 463	Placeholder PDF	Replace with real Google Docs PDF export

Implementation Steps: 1. Add Google API client with service account auth 2. Implement document creation from templates (engagement letters, 8879) 3. Implement signature request flow 4. Implement PDF export of signed documents 5. Add webhook handler for signature completion 6. Add integration tests with Google Workspace sandbox

Config Required:

# config.yaml
google:
  # service_account_file: Path to service account JSON
  service_account_file: ${GOOGLE_SERVICE_ACCOUNT_FILE}

  # delegated_user: User to impersonate for domain-wide delegation
  delegated_user: ${GOOGLE_DELEGATED_USER}

  # template_folder_id: Google Drive folder with document templates
  template_folder_id: ${GOOGLE_TEMPLATE_FOLDER_ID}

Estimated Effort: 6-8 hours

---

Phase 6: Webhook Security

Why Last: Webhooks work but lack signature verification. Security hardening before production.

File: src/api/routes/webhooks.py

Location	Current	Implementation
Line 355	TODO: Persona webhook	Already in Phase 2
Line 920	TODO: SurePrep signature	HMAC verification with shared secret
Line 931	TODO: verify signature	Generic signature verification helper

Implementation Steps: 1. Create verify_webhook_signature() utility 2. Add HMAC-SHA256 verification for each provider 3. Add request body caching for signature verification 4. Add replay attack prevention (timestamp checking) 5. Add logging for failed verification attempts

Estimated Effort: 2-3 hours

---

Implementation Order Summary

Phase	Service	Effort	Cumulative
1a	EmailService	3-4 hrs	3-4 hrs
1b	SMSService	2-3 hrs	5-7 hrs
2	PersonaService	4-5 hrs	9-12 hrs
3	SmartVaultService	6-8 hrs	15-20 hrs
4	SurePrepService	8-10 hrs	23-30 hrs
5	GoogleService	6-8 hrs	29-38 hrs
6	Webhook Security	2-3 hrs	31-41 hrs

Total Estimated Effort: 31-41 hours (4-5 days)

---

Prerequisites

Before starting implementation:

1. API Credentials - Need accounts and API keys for:
2. AWS SES (verified domain)
3. Twilio (account SID, auth token, phone number)
4. Persona (API key, template ID)
5. SmartVault (OAuth client credentials)
6. SurePrep (username, password, API key)

7. Google Workspace (service account)

8. Sandbox/Test Environments - Each service needs:

9. Development/sandbox credentials
10. Test data and scenarios

11. Webhook endpoints accessible (ngrok or similar for local dev)

12. Secrets Management - Decide on:

13. Local: .env file (development)
14. AWS: Secrets Manager (production)

---

Testing Strategy

Each phase includes: 1. Unit tests - Mock external API responses 2. Integration tests - Real API calls to sandbox 3. E2E tests - Full workflow through API endpoints

Test environment setup:

# Run with sandbox credentials
export PERSONA_ENV=sandbox
export SUREPREP_ENV=sandbox
export EMAIL_DRY_RUN=false  # Use SES sandbox
export SMS_DRY_RUN=false    # Use Twilio test credentials

python -m pytest tests/integration/services/ -v

---

Definition of Done

Each service is complete when: - [ ] All TODO comments removed - [ ] Real API calls implemented (not mocks) - [ ] Error handling for API failures - [ ] Retry logic for transient failures - [ ] Rate limiting respected - [ ] Unit tests passing - [ ] Integration tests passing with sandbox - [ ] Health check endpoint working - [ ] Documented in config.yaml with comments

---

Risk Mitigation

Risk	Mitigation
API credentials delayed	Start with services we have credentials for
Sandbox limitations	Document limitations, test what's possible
Rate limits	Implement exponential backoff, respect limits
Breaking API changes	Pin API versions, monitor changelogs

---

Next Steps

1. Immediate: Gather API credentials for all services
2. Phase 1: Implement Email + SMS (enables all notifications)
3. Phase 2-5: Implement in order based on credential availability
4. Phase 6: Harden webhook security

Ready to begin Phase 1 when credentials are available.