Architecture

Overview

MysticX is a Next.js 16 web application with a multi-process architecture. AI-powered tarot readings are handled asynchronously by background workers to avoid blocking the web server during long-running generative AI calls.

System Diagram

┌──────────────────────────────────────────────────────────────────────────────────┐
│  Browser Client                                                                  │
│  · POST /api/v1/readings          (create reading)                               │
│  · GET  /api/v1/chats/[id]/messages/[id]/stream  (SSE — token stream)            │
└────────────────────────────┬─────────────────────────────────────────────────────┘
                             │ HTTP / SSE
                             ▼
┌────────────────────────────────────────┐       ┌──────────────────────┐
│         Next.js App (App Router)       │◄─────►│   PostgreSQL 18      │
│  · API Routes  · Auth (Better Auth)   │       │   (primary store)    │
│  · Admin Panel · Page rendering       │       └──────────────────────┘
└─────────────────────┬──────────────────┘
                      │ BullMQ jobs / Redis Pub/Sub
                      ▼
         ┌────────────────────────┐
         │         Redis          │
         │  · BullMQ job queues   │
         │  · Pub/Sub streaming   │
         │  · Daily play limits   │
         └──┬──────┬──────┬───┬───┘
            │      │      │   │
     ┌──────┘  ┌───┘  ┌───┘  └──────────────────────┐
     ▼          ▼      ▼                              ▼
┌─────────┐ ┌───────┐ ┌────────────────┐  ┌─────────────────────┐
│  Tarot  │ │Insight│ │ Question       │  │ Translation Worker  │
│ Worker  │ │Worker │ │ Insights Worker│  │ (blog posts +       │
│         │ │       │ │                │  │  blog entities)     │
│ queue:  │ │queue: │ │ queue:         │  │ queues:             │
│ tarot-  │ │guidance│ │ question-      │  │ blog-translation    │
│ reading-│ │-genera-│ │ insights-      │  │ blog-entity-        │
│ genera- │ │tion   │ │ analysis       │  │ translation         │
│ tion    │ │       │ │                │  └──────────┬──────────┘
└────┬────┘ └───┬───┘ └───────┬────────┘             │
     │          │              │                       │
     └──────────┴──────────────┴───────────────────────┘
                               │
                               ▼
          ┌────────────────────────────────────────────┐
          │  Google Vertex AI  —  gemini-3-flash-preview│
          │  (IAM service account auth, standard mode) │
          │  · Tarot readings (streamText)             │
          │  · Follow-ups + card draws (+ Search       │
          │    Grounding via google_search tool)        │
          │  · Weekly Guidance / Soul Journey          │
          │  · Blog translation                        │
          └────────────────────────────────────────────┘

┌──────────────────────────────┐
│  Telegram Bot (grammY)       │  ← separate PM2 process, long-polling
│  Calls internal Next.js API  │    POST /api/v1/internal/readings/create
└──────────────────────────────┘

Processes

MysticX runs six processes in production, managed by PM2:

1. Next.js Web Server (mysticx-web)

• Mode: PM2 cluster (scales to all CPU cores)
• Responsibility: Serves the App Router pages, handles API routes, manages authentication, and serves the admin panel
• Port: 3031 (production default, configurable via PORT env var)
• Memory limit: 1 GB per instance

2. Tarot Worker (mysticx-worker)

• Mode: PM2 fork (single instance)
• Script: scripts/tarot-worker/index.ts
• Queue: tarot-reading-generation
• Job types: generate-reading, generate-followup, generate-draw-card
• Responsibility: Consumes reading jobs from BullMQ, calls gemini-3-flash-preview via @ai-sdk/google-vertex, and publishes streaming tokens to Redis Pub/Sub. Follow-up messages and dynamic card draws also invoke Google Search Grounding (google_search tool) for real-time grounding data
• Memory limit: 512 MB

3. Insights Worker (mysticx-insights-worker)

• Mode: PM2 fork (single instance)
• Script: scripts/insights-worker/index.ts
• Queue: guidance-generation
• Job types (dispatched by job.data.type): weekly, soul-journey
• Responsibility: Generates Weekly Guidance and Soul Journey content asynchronously using gemini-3-flash-preview
• Memory limit: 512 MB
• Concurrency: 1 (lock duration 120s)

4. Telegram Bot (mysticx-telegram-bot)

• Mode: PM2 fork (single instance)
• Script: scripts/telegram-bot/index.ts
• Responsibility: Long-polling Telegram bot using grammY. Makes internal API calls to the Next.js server for reading creation
• Memory limit: 256 MB

5. Question Insights Worker (mysticx-question-insights-worker)

• Mode: PM2 fork (single instance)
• Script: scripts/question-insights-worker/index.ts
• Queue: question-insights-analysis
• Responsibility: Analyzes user question patterns via gemini-3-flash-preview at admin request. Produces category distributions, top keywords, word clouds, and trend data stored as an InsightSnapshot
• Memory limit: 512 MB

6. Translation Worker (mysticx-translation-worker)

• Mode: PM2 fork (single instance)
• Script: scripts/translation-worker/index.ts
• Queues: blog-translation (post fields) and blog-entity-translation (categories, tags, authors)
• Responsibility: Auto-translates blog post fields (title, excerpt, content, SEO metadata) and blog entities to all 12 locales via gemini-3-flash-preview. Processes one field+locale at a time with exponential backoff. Tracks per-field progress in Redis and supports live cancellation
• Memory limit: 512 MB

Data Flow: Tarot Reading

1. User submits question + spread selection
       │
2. POST /api/v1/readings
       │  → Validates credits + daily play limit
       │  → Creates TarotReading + Chat + Message records
       │  → Enqueues job on tarot-reading-generation queue
       │  → Returns { readingId, chatId, messageId } to client
       │
3. Client opens SSE connection:
       GET /api/v1/chats/[chatId]/messages/[messageId]/stream
       │  → Subscribes to Redis Pub/Sub channel
       │  → If message already COMPLETED → returns full content immediately
       │
4. Tarot Worker picks up generate-reading job
       │  → Builds system prompt with reader persona + AI memory facts
       │  → Calls gemini-3-flash-preview via streamText (structured output)
       │  → Publishes streaming tokens to Redis Pub/Sub channel
       │
5. Client receives tokens from SSE and renders incrementally
       │
6. Worker completes
       │  → Updates Message status → COMPLETED, saves full content
       │  → Updates TarotReading with selectedCards + structured output
       │  → Records AiApiCall metrics (tokens, duration, TTFT)
       │  → Triggers extractMemory() fire-and-forget (updates UserMemory)

Data Flow: Insights Generation

1. User clicks "Generate" on Weekly Guidance or Soul Journey
       │
2. POST /api/v1/guidance/weekly (or soul-journey)
       │  → Creates record with status: GENERATING
       │  → Enqueues BullMQ job on guidance-generation queue
       │  → Returns 202 Accepted
       │
3. Insights Worker processes job
       │  → Fetches user's recent readings
       │  → Calls Gemini for content generation
       │  → Updates record: status → READY, fills content
       │  → Creates Notification record
       │
4. On failure:
       │  → Updates status to FAILED
       │  → Refunds credits to user

External Services

Service	Purpose
Google Vertex AI (gemini-3-flash-preview)	AI model for all AI features — readings, follow-ups (+ Search Grounding), guidance, translation, question insights
PostgreSQL 18	Primary data store
Redis 7+	BullMQ job queues, Pub/Sub token streaming, daily play limits
Stripe	Subscription billing, one-time credit pack purchases, webhooks
Cloudflare R2	Asset storage (card skins, reader avatars/covers, feedback screenshots)
Resend	Transactional emails (email verification, password reset)
Mixpanel	Product analytics and event tracking
Google Analytics 4	E-commerce and purchase event reporting (via gtag.js)
Tapfiliate	Affiliate commission tracking
Telegram Bot API	Telegram integration via grammY (long-polling)

Authentication

Authentication is handled by Better Auth with the following providers:

• Email/Password — with email verification via Resend
• Google OAuth — standard OAuth 2.0 flow

Session management uses HTTP-only cookies. The auth middleware lives in proxy.ts (Next.js middleware) and protects routes by checking session validity, redirecting unauthenticated users to /auth/sign-in?redirectTo=....

Note on Vertex AI auth: All workers authenticate to Google Cloud via a service account (GOOGLE_APPLICATION_CREDENTIALS). The standard Vertex AI integration (not Express mode) is required because Express mode does not support Google Search Grounding (vertex.tools.googleSearch) or fine-grained thinkingConfig.

Caching and Real-Time

• Redis Pub/Sub — Worker publishes streaming tokens per-message; the SSE endpoint at GET /api/v1/chats/[chatId]/messages/[messageId]/stream subscribes and forwards them to the browser
• Redis TTL keys — Daily play limits for guest users (resets at local midnight); per-field translation progress and cancellation flags
• BullMQ — Durable job queues with retry logic across all five background workers
• Zustand stores — Client-side state for credits, notifications, card selections, and UI modals