Components

PULSE system components and modules

Components

PULSE is organized into modular components, each with a specific responsibility.

Event Ingestion (/src/workers/ingestion)

Accepts and validates incoming events.

Key Files:

• index.ts — Main HTTP handler
• validation.ts — Event schema validation

Responsibilities:

• Accept POST requests to /collect
• Validate event format and required fields
• Rate limiting per API key
• Enrich events with server timestamp
• Batch insert into D1

Performance:

• <50ms latency per event
• Handles 1000+ events/sec

Query Engine (/src/workers/query)

Executes analytics queries and aggregations.

Key Files:

• index.ts — HTTP handler
• stats-query.ts — Metric aggregations
• funnel-query.ts — Funnel analysis
• timeseriesquery.ts — Time-series data

Responsibilities:

• Parse query parameters (metrics, filters, grouping)
• Execute D1 queries with proper indexing
• Cache results in KV
• Format responses

Performance:

• Cache hit: <10ms
• Cache miss: <100ms

Cohort Analysis (/src/workers/cohorts)

Manages cohorts and retention analytics.

Key Files:

• handler.ts — HTTP endpoints
• index.ts — Core cohort logic

Responsibilities:

• CRUD operations for cohorts
• Cohort member management
• Retention calculation
• Snapshot pre-computation

Snapshots:

• Generated nightly at 2 AM UTC
• 13 weeks of retention data
• ~50x faster than live calculation

Alert Rules (/src/workers/alerts)

Monitors metrics and triggers alerts.

Key Files:

• handler.ts — HTTP endpoints
• index.ts — Alert logic
• evaluator.ts — Condition evaluation

Responsibilities:

• Define alert conditions (threshold, anomaly)
• Evaluate conditions periodically
• Dispatch webhooks on trigger
• Track alert history

Authentication (/src/workers/auth)

API key validation and session management.

Key Files:

• index.ts — Auth middleware
• crypto.ts — Key derivation

Responsibilities:

• Validate API keys
• Cache keys in KV (60s TTL)
• Issue session tokens
• Rate limit auth attempts

Security:

• HMAC-SHA256 key hashing
• 60s cache with TTL
• Rate limit: 10 requests/min per IP

WebSocket Streaming (/src/workers/streaming)

Real-time event streaming to connected clients.

Key Files:

• index.ts — HTTP upgrade handler
• aggregator.ts — Event aggregation

Durable Objects:

• StreamManagerDO — Connection management
• AggregatorDO — Event aggregation

Responsibilities:

• Handle WebSocket upgrades
• Maintain client connections
• Broadcast events in real-time
• Aggregate and send metrics every 10s

Webhooks (/src/workers/webhooks)

External system integration via webhooks.

Key Files:

• handler.ts — HTTP endpoints
• dispatcher.ts — Event delivery

Responsibilities:

• Manage webhook endpoints
• Dispatch events asynchronously
• Retry with exponential backoff
• HMAC signing for security
• Track delivery status

Imports (/src/workers/imports)

Bulk data upload and processing.

Key Files:

• handler.ts — HTTP endpoints
• parser.ts — CSV/JSON parsing
• processor.ts — Batch processing

Responsibilities:

• Accept file uploads
• Parse CSV and JSON formats
• Validate event/user data
• Batch insert (1000 records at a time)
• Track progress and errors

Performance:

• Batch processing: 20-25x faster
• 1000 records in <1-2 seconds

Exports (/src/workers/exports)

Scheduled report generation and storage.

Key Files:

• handler.ts — HTTP endpoints
• generator.ts — Data generation
• uploader.ts — R2 storage

Responsibilities:

• Generate reports and exports
• Store in R2 (Cloudflare storage)
• Schedule recurring exports
• Provide download URLs

Settings (/src/workers/settings)

Site and API key management.

Key Files:

• index.ts — HTTP endpoints

Responsibilities:

• Create and list API keys
• Rotate keys safely
• Configure per-site settings
• Track key usage

Repository Layer (/src/repositories)

Data access layer using repository pattern.

Key Repositories:

• CohortsRepository — Cohort CRUD
• AlertRulesRepository — Alert CRUD
• ImportsRepository — Import jobs
• ExportsRepository — Export jobs
• WebhooksRepository — Webhook endpoints

Responsibilities:

• Encapsulate database queries
• Provide type-safe interfaces
• Handle transactions
• Cache invalidation

Utilities

Validation (/src/utils/validation.ts)

• ReDoS prevention patterns
• Email validation
• URL validation

Response Helpers (/src/utils/handlers.ts)

• Standardized JSON responses
• Error formatting
• Pagination responses

CRUD Helpers (/src/utils/crud-helpers.ts)

• Pagination parsing
• Resource ID extraction
• Dynamic update builders

Durable Objects

StreamManagerDO

Real-time analytics streaming.

• Manages WebSocket connections
• Broadcasts aggregated events
• Tracks active subscriptions
• Handles reconnections

AggregatorDO

Event aggregation for metrics.

• Aggregates events per minute
• Computes metrics (counts, rates)
• Sends to connected clients
• Per-site isolation

Database Migrations

Located in /src/migrations/:

001_initial_schema.sql         -- Core tables
002_sessions.sql               -- Session tracking
003_indexes.sql                -- Performance indexes
004_cohorts.sql                -- Cohort tables
...
021_performance_indexes.sql    -- Performance optimizations

Type System (/src/types)

TypeScript interfaces for type safety:

// API requests
interface EventIngestionRequest { ... }
interface QueryRequest { ... }

// API responses
interface ApiResponse<T> { ... }

// Database models
interface Event { ... }
interface CohortDefinition { ... }
interface AlertRule { ... }

Configuration

• wrangler.toml — Worker configuration and bindings
• .dev.vars — Local development environment
• tsconfig.json — TypeScript configuration
• vitest.config.ts — Testing configuration

Component Interactions

Event Flow

Event → Ingestion → Validation → D1 Insert → KV Invalidate → Webhook Dispatch

Query Flow

Query → Auth → Validation → KV Check → D1 Query → Serialize → Response

Alert Flow

Alert Rule Triggers → Condition Evaluation → Webhook Dispatch → Retry Loop

WebSocket Flow

Connect → StreamManager DO → Subscribe → Aggregator DO → Real-Time Broadcasts

Next Steps

• Concurrency & Locking — Distributed concurrency patterns
• Performance — Optimization techniques
• Security — Security architecture