Skip to main content
Status: Accepted Date: 2026-02-23 Deciders: Reflections Maintainers

​
Context

Loti provides daily JSONL feeds of content-monitoring results (face-match detections across platforms). Their Lambda endpoint is IP-whitelisted — only pre-registered static IPs can fetch data. Railway (the hosting platform) assigns ephemeral IPs to containers. This means the API cannot call Loti directly; a relay with a static IP is needed. Requirements:
  • Fetch daily JSONL files from Loti’s IP-whitelisted endpoint.
  • Forward results to the API in idempotent chunks.
  • Minimize cost and operational surface area.

​
Decision

Use an EC2 t3.micro relay with a daily cron job that fetches JSONL from Loti and POSTs it in 500-line chunks to POST /internal/loti-feed. Data flow: 
Loti Lambda --(IP-whitelisted)--> EC2 relay (cron, static EIP)
    --(chunked POST, x-internal-secret)--> Railway API /internal/loti-feed
    --(upsert ON CONFLICT DO NOTHING)--> Postgres loti_results table
The API endpoint validates JSONL, parses each line against a Zod schema, and upserts into loti_results deduped by result_id.

​
Alternatives considered

AlternativeWhy rejected
Direct S3 cross-account accessLoti uses IP whitelisting, not IAM cross-account roles. Would require Loti to change their auth model.
Lambda-to-Lambda pushRequires Loti to implement a push model; they only support pull.
Serverless NAT Gateway~$32/mo for a NAT Gateway + Lambda just to get a static IP. Overkill for one daily cron.
Railway static IP add-onNot available on the current plan. Even if available, would couple relay concern to app infrastructure.

​
Consequences

Benefits:
  • Low cost (~$7.50/mo for t3.micro + EIP).
  • Minimal surface: single bash script, single cron entry, single API route.
  • Fully idempotent: ON CONFLICT (result_id) DO NOTHING means re-running is safe.
  • Chunked delivery (500 lines/request) keeps request sizes manageable.
Costs:
  • EC2 instance to maintain (patching, monitoring).
  • Additional infrastructure outside Railway.
Risks:
  • Silent cron failure if EC2 goes down or Loti endpoint changes.
Mitigations:
  • Monitor daily row count in loti_results — alert if zero new rows for 48 hours.
  • EC2 instance tagged and documented; relay script is simple enough to recreate in minutes.

​
Implementation notes

  • The migration creates the loti_results table with appropriate indexes.
  • DB query module provides upsertLotiResults re-exported via the write entrypoint.
  • API route validates and upserts JSONL chunks, authenticated via internal secret header.
  • Body limit override is configured to accommodate larger JSONL payloads (10 MB).