Blog Background Translation Pattern

How blog entity translations are processed via BullMQ background jobs with Redis status tracking. Use this as the reference when adding similar background translation to new modules.

Problem

Translating content directly from the browser (via fetch + await) means the job is lost if the user navigates away before it finishes. Translation must survive page navigation.

Solution

Enqueue a BullMQ job to Redis and return immediately. A standalone worker process picks up the job, translates one locale at a time, and writes progress to Redis. The UI polls for status updates.

Architecture Overview

Browser                    Next.js API              Redis + BullMQ           Worker Process
───────                    ──────────               ──────────────           ──────────────
Click "Translate"  ──►  POST /entity-translate  ──►  Enqueue job      ──►  Pick up job
                         Seed Redis: "queued"        + seed progress        Translate locale by locale
                         Return { jobId }                                   Update Redis after each step
                                                                            Write all to DB when done
Poll every 4s      ──►  GET  /entity-translate/status  ◄── Read Redis
Show progress
  (Queued → 3/11 → Translated)

Two Queue Types

Queue	Const	Use
`blog-translation`	`BLOG_TRANSLATION_QUEUE`	Posts (title, excerpt, content, seoTitle, seoDescription)
`blog-entity-translation`	`BLOG_ENTITY_TRANSLATION_QUEUE`	Tags, categories, authors (single field each)

Posts have more fields and support cancellation. Entities are simpler (one field per entity).

File Map

Queue definition

lib/queue.ts - Queue name constants, job data types, Queue instances (singleton via globalForQueue)

API routes

app/api/admin/blog/[postId]/translate/route.ts - POST: enqueue post translation
app/api/admin/blog/[postId]/translate/status/route.ts - GET: poll post translation progress
app/api/admin/blog/[postId]/translate/cancel/route.ts - POST: cancel post translation
app/api/admin/blog/entity-translate/route.ts - POST: enqueue entity translation (tag/category/author)
app/api/admin/blog/entity-translate/status/route.ts - GET: poll entity translation progress

Worker

scripts/translation-worker/setup.ts - Shared config: Redis connections, AI model, constants, Prisma client
scripts/translation-worker/index.ts - Worker entrypoint: job processors + Worker instances

Admin pages (SSR hydration)

app/[locale]/admin/blog/page.tsx - Posts: fetches initial statuses via redis.mget()
app/[locale]/admin/blog/tags/page.tsx - Tags: same pattern
app/[locale]/admin/blog/categories/page.tsx - Categories: same pattern
app/[locale]/admin/blog/authors/page.tsx - Authors: same pattern

Table components (client-side polling)

app/[locale]/admin/blog/_components/BlogPostsTable.tsx - Reference implementation (most complete)
app/[locale]/admin/blog/tags/_components/BlogTagsTable.tsx
app/[locale]/admin/blog/categories/_components/BlogCategoriesTable.tsx
app/[locale]/admin/blog/authors/_components/BlogAuthorsTable.tsx

Step-by-Step: Adding Background Translation to a New Module

1. Define the queue (if needed)

In lib/queue.ts:

Add a queue name constant
Add a job data type (type TMyJobData = { ... })
Add a Queue instance with removeOnComplete, removeOnFail, attempts: 1
Add to globalForQueue type and dev-mode assignment

2. Create the POST (enqueue) API route

Pattern: app/api/admin/blog/entity-translate/route.ts

// 1. Auth check (session?.user.role !== 'admin')
// 2. Validate input (entityType, entityId)
// 3. Load entity from DB, verify source text exists
// 4. Remove previous job if exists
// 5. Seed Redis with initial "queued" progress
// 6. Enqueue BullMQ job
// 7. Return { jobId }

Redis key pattern: blog-entity-translate:{entityType}:{entityId}

Initial progress shape seeded to Redis:

json

{
  "status": "queued",
  "totalSteps": 11,
  "completedSteps": 0,
  "failedSteps": 0,
  "steps": {}
}

3. Create the GET (status) API route

Pattern: app/api/admin/blog/entity-translate/status/route.ts

// 1. Auth check
// 2. Read Redis key
// 3. Return parsed JSON or { status: 'idle' }

4. Add the worker processor

In scripts/translation-worker/index.ts:

async function processMyJob(data: TMyJobData): Promise<void> {
  // 1. Load entity from DB
  // 2. Build step list (field x locales)
  // 3. Set Redis progress to "running"
  // 4. Loop through steps sequentially:
  //    a. Set step to "running" in Redis
  //    b. Call translateText(sourceLocale, targetLocale, text, context)
  //    c. Set step to "done" or "failed"
  //    d. Update Redis progress
  // 5. Write all translations to DB (raw SQL to avoid @updatedAt)
  // 6. Set terminal status in Redis ("done" or "failed")
}

Important: each Worker needs its own Redis connection (BullMQ requirement). Create a new IORedis instance.

Redis TTL: 3600s for active jobs, 300s for terminal states.

5. Update the page.tsx (SSR hydration)

Fetch initial statuses from Redis so the UI shows correct state on page load:

const keys = entities.map((e) => `blog-entity-translate:mytype:${e.id}`);
const values = await redis.mget(...keys);
// Parse each value, filter to queued/running/done/failed
// Pass as initialTranslationStatuses prop to the table

6. Update the table component (client-side polling)

Key pieces:

statusMap state: Map<string, TStatusEntry> initialized from SSR data
pollActive callback: polls GET status endpoint every 4s for queued/running entries
dismissTimers ref: auto-dismiss terminal states after 15s
handleTranslate function: POST to enqueue, optimistically set "queued" in statusMap
TranslationCell component with 5 states:
- queued - Clock icon, "Queued"
- running - Spinner + "3/11" counter
- failed - "Retry" button (calls handleTranslate again)
- done - Check icon, "Translated" (ephemeral, auto-dismisses to DB flag)
- default - "Translate" button (checks entity.isTranslated for persistent state)

7. Locale tab green checkmark pattern (forms)

When a locale has content filled, show a green tab with checkmark in form editors:

tsx

const hasContent = !!formData.fieldName[locale]?.trim();
// ...
className={cn(
  'rounded-full border px-3 py-1 text-xs font-medium transition-colors',
  activeLocale === locale
    ? 'border-primary bg-primary text-primary-foreground'
    : hasContent
      ? 'border-emerald-200 bg-emerald-50/50 text-emerald-700'
      : 'border-border bg-background text-muted-foreground hover:bg-muted hover:text-foreground',
)}
// Show checkmark for non-active filled tabs:
{hasContent && activeLocale !== locale ? (
  <Check className="ml-1 inline size-3 flex-none" />
) : null}

Key Design Decisions

Concurrency: 1 - One translation at a time to respect Vertex AI rate limits
Retries: handled inside the processor (5 retries per step with exponential backoff), not by BullMQ (attempts: 1)
Raw SQL for DB writes - prisma.$executeRawUnsafe to avoid @updatedAt auto-update on translations
12 locales, 11 targets - en is source, translate to: zh_CN, zh_TW, ja, ko, pt, es, fr, de, ar, id, nl
Worker runs standalone via PM2: npx tsx scripts/translation-worker/index.ts

Blog Background Translation Pattern ​

Problem ​

Solution ​

Architecture Overview ​

Two Queue Types ​

File Map ​

Queue definition ​

API routes ​

Worker ​

Admin pages (SSR hydration) ​

Table components (client-side polling) ​

Step-by-Step: Adding Background Translation to a New Module ​

1. Define the queue (if needed) ​

2. Create the POST (enqueue) API route ​

3. Create the GET (status) API route ​

4. Add the worker processor ​

5. Update the page.tsx (SSR hydration) ​

6. Update the table component (client-side polling) ​

7. Locale tab green checkmark pattern (forms) ​

Key Design Decisions ​