berrypod/docs/plans/products-context.md
Jamey Greenwood c5c06d9979 feat: add Products context with provider integration (Phase 1)
Implement the schema foundation for syncing products from POD providers
like Printify. This includes encrypted credential storage, product/variant
schemas, and an Oban worker for background sync.

New modules:
- Vault: AES-256-GCM encryption for API keys
- Products context: CRUD and sync operations for products
- Provider behaviour: abstraction for POD provider implementations
- ProductSyncWorker: Oban job for async product sync

Schemas: ProviderConnection, Product, ProductImage, ProductVariant

Also reorganizes Printify client to lib/simpleshop_theme/clients/ and
mockup generator to lib/simpleshop_theme/mockups/ for better structure.

134 tests added covering all new functionality.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 20:32:20 +00:00

1184 lines
58 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Plan: Products Context with Provider Integration
## Goal
Build a Products context that syncs products from external POD providers (Printify first), stores them locally, and enables order submission for fulfillment.
## Core Flow
1. **Connect** - Authenticate with provider (API key or OAuth)
2. **Sync** - Fetch products from provider, upsert locally
3. **Order** - Submit orders to provider for fulfillment
---
## Current Domain Analysis
SimpleShop has **6 well-defined domains** with clear boundaries:
| Domain | Purpose | Schemas | Public Functions |
|--------|---------|---------|------------------|
| **Accounts** | User authentication & sessions | User, UserToken | 15 |
| **Settings** | Theme configuration persistence | Setting, ThemeSettings | 6 |
| **Theme** | CSS generation, caching, presets | (structs only) | N/A |
| **Media** | Image upload/storage | Image | 8 |
| **Images** | Image optimization pipeline | (Oban workers) | N/A |
| **Printify** | External API integration | (no schemas) | API client |
The new **Products** context will be a new top-level domain that:
- Owns product/variant schemas
- Coordinates with Printify (and future providers) for sync
- Provides data to shop LiveViews (replacing PreviewData)
---
## Architecture Diagram
```
┌─────────────────────────────────────────────────────────────────────────────┐
│ SimpleshopTheme │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌──────────────────────────────────────────────────────────────────────┐ │
│ │ WEB LAYER │ │
│ │ SimpleshopThemeWeb │ │
│ │ ┌────────────────┐ ┌────────────────┐ ┌─────────────────────────┐ │ │
│ │ │ Shop LiveViews │ │ Admin LiveViews│ │ Theme Editor LiveView │ │ │
│ │ │ - ProductShow │ │ - UserLogin │ │ - ThemeLive.Index │ │ │
│ │ │ - Collection │ │ - UserSettings │ │ - Preview iframe │ │ │
│ │ │ - Cart │ │ │ │ │ │ │
│ │ └───────┬─────────┘ └────────────────┘ └────────┬───────────────┘ │ │
│ └──────────┼────────────────────────────────────────┼──────────────────┘ │
│ │ │ │
│ ▼ ▼ │
│ ┌──────────────────────────────────────────────────────────────────────┐ │
│ │ DOMAIN LAYER │ │
│ │ │ │
│ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │
│ │ │ Accounts │ │ Settings │ │ Theme │ │ │
│ │ │ │ │ │ │ │ │ │
│ │ │ • User │ │ • Setting │ │ • CSSCache │ │ │
│ │ │ • UserToken │ │ • Theme │ │ • CSSGen │ │ │
│ │ │ │ │ Settings │ │ • Presets │ │ │
│ │ │ │ │ │ │ • Fonts │ │ │
│ │ │ │ │ │ │ • Preview │ │ │
│ │ └─────────────┘ └──────┬──────┘ │ Data (*) │ │ │
│ │ │ └─────────────┘ │ │
│ │ │ │ │
│ │ ┌─────────────┐ ┌──────▼──────┐ ┌─────────────┐ │ │
│ │ │ Media │ │ Images │ │ Printify │ │ │
│ │ │ │ │ │ │ │ │ │
│ │ │ • Image │◄──│ • Optimizer │ │ • Client │ │ │
│ │ │ (upload) │ │ • Worker │ │ • Catalog │ (NEW) │ │
│ │ │ │ │ • Cache │ │ • Mockup │ │ │
│ │ └─────────────┘ └─────────────┘ └──────┬──────┘ │ │
│ │ │ │ │
│ │ ┌───────────────────────────────────────────▼───────────────────┐ │ │
│ │ │ Products (NEW) │ │ │
│ │ │ │ │ │
│ │ │ ┌─────────────────────┐ ┌────────────────────────────┐ │ │ │
│ │ │ │ Provider Layer │ │ Product Layer │ │ │ │
│ │ │ │ │ │ │ │ │ │
│ │ │ │ • ProviderConnection│───▶│ • Product │ │ │ │
│ │ │ │ • Provider behaviour│ │ • ProductImage │ │ │ │
│ │ │ │ • PrintifyProvider │ │ • ProductVariant │ │ │ │
│ │ │ │ • (future providers)│ │ │ │ │ │
│ │ │ └─────────────────────┘ └────────────────────────────┘ │ │ │
│ │ │ │ │ │
│ │ │ ┌─────────────────────────────────────────────────────────┐ │ │ │
│ │ │ │ Sync Layer │ │ │ │
│ │ │ │ • ProductSyncWorker (Oban) │ │ │ │
│ │ │ │ • Upsert logic │ │ │ │
│ │ │ └─────────────────────────────────────────────────────────┘ │ │ │
│ │ └────────────────────────────────────────────────────────────────┘ │ │
│ │ │ │
│ │ ┌────────────────────────────────────────────────────────────────┐ │ │
│ │ │ Orders (NEW) │ │ │
│ │ │ │ │ │
│ │ │ • Order • submit_to_provider/1 │ │ │
│ │ │ • OrderLineItem • track_status/1 │ │ │
│ │ └────────────────────────────────────────────────────────────────┘ │ │
│ │ │ │
│ └───────────────────────────────────────────────────────────────────────┘ │
│ │
│ ┌───────────────────────────────────────────────────────────────────────┐ │
│ │ INFRASTRUCTURE │ │
│ │ Repo (SQLite) • Vault (encryption) • Oban (background jobs) │ │
│ └───────────────────────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
(*) PreviewData will be replaced by Products context queries
```
---
## Database ERD
```
┌─────────────────────────────────────────────────────────────────────────────┐
│ CURRENT SCHEMA │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌──────────────────┐ ┌──────────────────┐ │
│ │ users │ │ users_tokens │ │
│ ├──────────────────┤ ├──────────────────┤ │
│ │ id (binary_id) │◄────────│ user_id (FK) │ │
│ │ email │ │ token │ │
│ │ hashed_password │ │ context │ │
│ │ confirmed_at │ │ sent_to │ │
│ │ inserted_at │ │ inserted_at │ │
│ │ updated_at │ └──────────────────┘ │
│ └──────────────────┘ │
│ │
│ ┌──────────────────┐ ┌──────────────────┐ │
│ │ settings │ │ images │ │
│ ├──────────────────┤ ├──────────────────┤ │
│ │ id (binary_id) │ │ id (binary_id) │ │
│ │ key │ │ filename │ │
│ │ value (JSON) │ │ content_type │ │
│ │ inserted_at │ │ byte_size │ │
│ │ updated_at │ │ width │ │
│ └──────────────────┘ │ height │ │
│ │ storage_key │ │
│ │ inserted_at │ │
│ │ updated_at │ │
│ └──────────────────┘ │
├─────────────────────────────────────────────────────────────────────────────┤
│ NEW SCHEMA (Products) │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌────────────────────────┐ │
│ │ provider_connections │ │
│ ├────────────────────────┤ │
│ │ id (binary_id) │ │
│ │ provider_type │──┐ "printify" | "prodigi" | "gelato" │
│ │ name │ │ │
│ │ enabled │ │ │
│ │ api_key_encrypted │ │ (Vault encrypted) │
│ │ config (JSON) │ │ {"shop_id": "123"} │
│ │ last_synced_at │ │ │
│ │ sync_status │ │ │
│ │ inserted_at │ │ │
│ │ updated_at │ │ │
│ └───────────┬────────────┘ │ │
│ │ │ │
│ │ 1 │ │
│ ▼ * │ │
│ ┌────────────────────────┐ │ │
│ │ products │ │ │
│ ├────────────────────────┤ │ │
│ │ id (binary_id) │ │ │
│ │ provider_connection_id │◄─┘ │
│ │ provider_product_id │ External ID (unique per connection) │
│ │ title │ │
│ │ description │ │
│ │ slug │ URL-friendly (unique globally) │
│ │ status │ "active" | "draft" | "archived" │
│ │ visible │ │
│ │ category │ │
│ │ option_types (JSON) │ ["size", "color"] - display order │
│ │ provider_data (JSON) │ Provider-specific fields (blueprint_id etc) │
│ │ inserted_at │ │
│ │ updated_at │ │
│ └───────────┬────────────┘ │
│ │ │
│ │ 1 │
│ ┌─────────┴─────────┐ │
│ ▼ * ▼ * │
│ ┌──────────────────┐ ┌──────────────────────┐ │
│ │ product_images │ │ product_variants │ │
│ ├──────────────────┤ ├──────────────────────┤ │
│ │ id (binary_id) │ │ id (binary_id) │ │
│ │ product_id (FK) │ │ product_id (FK) │ │
│ │ src │ │ provider_variant_id │ │
│ │ position │ │ title │ "Large / Black" │
│ │ alt │ │ sku │ │
│ │ inserted_at │ │ price │ Selling price (pence) │
│ │ updated_at │ │ compare_at_price │ For sale display │
│ └──────────────────┘ │ cost │ Provider cost (profit) │
│ │ options (JSON) │ {"size": "L", "color": ""} │
│ │ is_enabled │ │
│ │ is_available │ │
│ │ inserted_at │ │
│ │ updated_at │ │
│ └──────────────────────┘ │
│ │
├─────────────────────────────────────────────────────────────────────────────┤
│ NEW SCHEMA (Orders) │
│ │
│ Customer sees ONE order. Fulfillments are internal (one per provider). │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌────────────────────────┐ │
│ │ orders │ Customer-facing order │
│ ├────────────────────────┤ │
│ │ id (binary_id) │ │
│ │ order_number │ "SS-250125-A1B2" (customer sees this) │
│ │ status │ pending → processing → shipped → delivered │
│ │ payment_status │ pending | paid | refunded │
│ │ customer_email │ │
│ │ shipping_address (JSON)│ │
│ │ subtotal │ (Money - ex_money) │
│ │ shipping_cost │ (Money - ex_money) │
│ │ total │ (Money - ex_money) │
│ │ total_cost │ Provider costs (for profit) │
│ │ inserted_at │ │
│ │ updated_at │ │
│ └───────────┬────────────┘ │
│ │ │
│ │ 1 │
│ ┌─────────┴─────────────────────────┐ │
│ ▼ * ▼ * │
│ ┌────────────────────────┐ ┌─────────────────────────┐ │
│ │ order_fulfillments │ │ order_line_items │ │
│ ├────────────────────────┤ ├─────────────────────────┤ │
│ │ id (binary_id) │ │ id (binary_id) │ │
│ │ order_id (FK) │ │ order_id (FK) │ │
│ │ provider_connection_id │ │ order_fulfillment_id(FK)│ Which provider │
│ │ provider_order_id │ │ product_variant_id (FK) │ │
│ │ status │ │ quantity │ │
│ │ tracking_number │ │ unit_price │ │
│ │ tracking_url │ │ unit_cost │ │
│ │ shipped_at │ │ inserted_at │ │
│ │ inserted_at │ │ updated_at │ │
│ │ updated_at │ └─────────────────────────┘ │
│ └────────────────────────┘ │
│ │
│ ┌────────────────────────┐ │
│ │ order_events │ Timeline visible to customer │
│ ├────────────────────────┤ │
│ │ id (binary_id) │ │
│ │ order_id (FK) │ │
│ │ type │ "placed" | "paid" | "shipped" | "delivered" │
│ │ message │ "Your T-Shirt has shipped via Royal Mail" │
│ │ metadata (JSON) │ {tracking_number, carrier, etc} │
│ │ inserted_at │ │
│ └────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
```
---
## Printify Code Reuse
### Existing Client (Highly Reusable)
The existing `Printify.Client` is directly reusable:
```elixir
# HTTP infrastructure (no changes needed)
Client.get/2, Client.post/3, Client.delete/2
# Shop management
Client.get_shops/0, Client.get_shop_id/0
# Product operations (already implemented)
Client.get_product/2, Client.create_product/2, Client.delete_product/2
```
**To add:**
```elixir
Client.list_products/1 # GET /shops/:shop_id/products.json
Client.create_order/2 # POST /shops/:shop_id/orders.json
Client.get_order/2 # GET /shops/:shop_id/orders/:id.json
```
### MockupGenerator Refactoring
Extract reusable helpers into new `Printify.Catalog` module:
| Function | Current | Target |
|----------|---------|--------|
| `find_blueprint/1` | MockupGenerator | `Printify.Catalog` |
| `find_print_provider/2` | MockupGenerator | `Printify.Catalog` |
| `get_variant_info/1` | MockupGenerator | `Printify.Catalog` |
| `calculate_cover_scale/2` | MockupGenerator | Keep (mockup-specific) |
| `product_definitions/0` | MockupGenerator | Keep (sample data) |
**Module structure:**
```
lib/simpleshop_theme/clients/
├── printify.ex # Printify HTTP client (moved from printify/client.ex)
├── gelato.ex # Gelato HTTP client
└── prodigi.ex # Prodigi HTTP client
lib/simpleshop_theme/providers/
├── provider.ex # Behaviour definition
├── printify.ex # Printify implementation (uses Clients.Printify)
├── gelato.ex # Gelato implementation (uses Clients.Gelato)
└── prodigi.ex # Prodigi implementation (uses Clients.Prodigi)
lib/simpleshop_theme/mockups/
└── generator.ex # Mockup generation (currently uses Clients.Printify)
# Provider-agnostic location for future flexibility
lib/simpleshop_theme/printify/
└── catalog.ex # Blueprint/variant discovery helpers (Printify-specific)
```
Each provider module uses its corresponding client. The mockup generator is in a provider-agnostic location.
---
## Database Schema
### Tables Overview
| Table | Purpose |
|-------|---------|
| `provider_connections` | Provider credentials + config (one per type) |
| `products` | Core product data synced from provider |
| `product_images` | Product images (multiple per product) |
| `product_variants` | Size/color variants with pricing |
| `orders` | Customer orders |
| `order_line_items` | Items within orders |
### Key Design Decisions
1. **Provider IDs on product directly** (not a separate linking table)
- `products.provider_product_id` stores external ID
- `product_variants.provider_variant_id` stores external variant ID
- Unique constraint: `[:provider_connection_id, :provider_product_id]`
2. **Credentials encrypted in database**
- Use `SimpleshopTheme.Vault` for at-rest encryption
- `api_key_encrypted`, `oauth_access_token_encrypted` fields
3. **Cost tracking for profit calculation**
- `product_variants.cost` - cost from provider
- `order_line_items.unit_cost` - snapshot at order time
- `orders.total_cost` - sum for profit reporting
4. **Provider type as enum** (not separate table)
- Hardcoded: `~w(printify prodigi gelato)`
- Keeps type safety for provider-specific logic
---
## Schema Details
### provider_connections
```elixir
field :provider_type, :string # "printify", "prodigi", "gelato"
field :name, :string # "My Printify Account"
field :enabled, :boolean
field :api_key_encrypted, :binary # Encrypted API key
field :config, :map # {"shop_id": "12345"}
field :last_synced_at, :utc_datetime
field :sync_status, :string # pending, syncing, completed, failed
```
### products
```elixir
belongs_to :provider_connection
field :provider_product_id, :string # External ID from provider
field :title, :string
field :description, :text
field :slug, :string # URL-friendly, unique
field :status, :string # active, draft, archived
field :visible, :boolean
field :category, :string
field :option_types, {:array, :string} # ["Colors", "Sizes"] - option names in display order
# Extracted from provider's product.options[].name on sync
# Used to order the variant selector UI
field :provider_data, :map # Provider-specific fields stored here
# Printify: %{"blueprint_id" => 145, "print_provider_id" => 29, ...}
# Keeps products table generic across providers
```
### product_variants
```elixir
belongs_to :product
field :provider_variant_id, :string # CRITICAL: The ID we send back to provider when ordering
# Printify: variant_id (e.g., "12345")
# Printful: sync_variant_id (e.g., "4752058849")
# Gelato: productUid (e.g., "apparel_product_gca_t-shirt_gsi_m_gco_white")
# Prodigi: sku (e.g., "GLOBAL-CAN-10x10")
field :title, :string # "Large / Black"
field :sku, :string # Shop's internal SKU (optional)
field :price, :integer # Selling price (pence)
field :compare_at_price, :integer # Original price (for sales)
field :cost, :integer # Provider cost (for profit)
field :options, :map # {"size": "Large", "color": "Black"}
# JSON is appropriate here because:
# - Options always loaded with variant (no join needed)
# - No need to query "find all Large variants"
# - Matches provider data format (easy sync)
# - Simpler than a separate options table
field :is_enabled, :boolean
field :is_available, :boolean # In stock at provider
```
### orders (customer-facing)
```elixir
# Customer sees ONE order, even if fulfilled by multiple providers
field :order_number, :string # "SS-250125-A1B2" (customer-visible)
field :status, :string # Aggregate: pending → processing → shipped → delivered
field :payment_status, :string # pending, paid, refunded
field :customer_email, :string
field :shipping_address, :map
field :subtotal, Money.Ecto.Map.Type # Using ex_money for currency safety
field :shipping_cost, Money.Ecto.Map.Type
field :total, Money.Ecto.Map.Type
field :total_cost, Money.Ecto.Map.Type # Sum of provider costs
has_many :order_fulfillments # One per provider
has_many :order_line_items
has_many :order_events # Timeline for customer
```
### order_fulfillments (one per provider)
```elixir
belongs_to :order
belongs_to :provider_connection
field :provider_order_id, :string # External ID after submission
field :status, :string # pending → submitted → processing → shipped
field :tracking_number, :string
field :tracking_url, :string
field :shipped_at, :utc_datetime
has_many :order_line_items # Items fulfilled by this provider
```
### order_line_items
```elixir
belongs_to :order
belongs_to :order_fulfillment # Links to which provider ships this item
belongs_to :product_variant
field :quantity, :integer
field :unit_price, Money.Ecto.Map.Type
field :unit_cost, Money.Ecto.Map.Type
# Per-item tracking (provider may ship items separately)
field :status, :string # pending → shipped → delivered
field :tracking_number, :string
field :tracking_url, :string
field :shipped_at, :utc_datetime
```
### order_events (customer timeline)
```elixir
belongs_to :order
field :type, :string # Event types (see below)
field :message, :string # "Your T-Shirt and Mug have shipped!"
field :metadata, :map # {tracking_number, carrier, items, etc}
# Event types:
# - "placed" → Order placed
# - "paid" → Payment confirmed
# - "fulfillment_shipped" → Some items shipped (granular per-fulfillment)
# - "shipped" → ALL items now shipped
# - "delivered" → ALL items delivered
# Example metadata for fulfillment_shipped:
# %{fulfillment_id: "uuid", items: ["T-Shirt", "Mug"],
# carrier: "Royal Mail", tracking_number: "RM123", tracking_url: "..."}
```
---
## Variant Selector UX Pattern
Products with multiple option types (e.g., size × color × sleeves) can have many variants:
- 4 sizes × 4 colors × 2 sleeves = **32 variants**
### Data Model
Each variant stores its specific combination as JSON with **human-readable labels**:
```elixir
%{id: "v1", options: %{"size" => "M", "color" => "Black", "sleeves" => "Short"}, price: 2500}
%{id: "v2", options: %{"size" => "M", "color" => "Black", "sleeves" => "Long"}, price: 2700}
# ... etc
```
### Provider Option Structure (Important)
Providers like Printify use a **two-tier option system**:
**Product level** - defines options with IDs and labels:
```json
{
"options": [
{ "name": "Colors", "type": "color", "values": [
{ "id": 751, "title": "Solid White" },
{ "id": 752, "title": "Black" }
]},
{ "name": "Sizes", "type": "size", "values": [
{ "id": 2, "title": "S" },
{ "id": 3, "title": "M" }
]}
]
}
```
**Variant level** - only stores option IDs:
```json
{ "id": 12345, "options": [751, 2], "title": "Solid White / S" }
```
### Sync Strategy: Denormalize to Labels
During sync, we **resolve IDs to labels** and store the human-readable version:
```elixir
def normalize_variant(provider_variant, product_options) do
# Build ID -> label lookup from product options
option_lookup = build_option_lookup(product_options)
# => %{751 => {"Colors", "Solid White"}, 2 => {"Sizes", "S"}}
# Convert variant's [751, 2] to %{"Colors" => "Solid White", "Sizes" => "S"}
options =
provider_variant["options"]
|> Enum.map(&Map.get(option_lookup, &1))
|> Enum.reject(&is_nil/1)
|> Map.new()
%{
provider_variant_id: to_string(provider_variant["id"]),
title: provider_variant["title"],
options: options,
# ... other fields
}
end
```
**Why denormalize?**
- No need to join to an options table on every product page load
- Labels rarely change (and if they do, re-sync updates them)
- Simpler queries, faster rendering
- Keeps our schema provider-agnostic (we just store labels, not provider-specific IDs)
### Provider Compatibility
All major POD providers can be normalized to our `options: %{"Size" => "M", "Color" => "Black"}` schema:
| Provider | Raw Variant Structure | Normalization |
|----------|----------------------|---------------|
| **Printify** | `options: [751, 2]` (IDs only, labels at product level) | Resolve IDs → labels during sync |
| **Printful** | `color: "Black", size: "M"` (flat attributes) | Map attribute keys to standard names |
| **Gelato** | `variantOptions: [{name: "Size", value: "M"}]` (array) | Convert array to map |
| **Prodigi** | `attributes: {"wrap": "Black"}` (object) | Direct mapping |
Each provider's `normalize_variant/2` function handles the transformation:
```elixir
# Printify - resolve IDs to labels
%{"options" => [751, 2]} %{"Colors" => "Solid White", "Sizes" => "S"}
# Printful - rename keys
%{"color" => "Black", "size" => "M"} %{"Color" => "Black", "Size" => "M"}
# Gelato - array to map
[%{"name" => "Size", "value" => "M"}] %{"Size" => "M"}
# Prodigi - direct (already a map)
%{"wrap" => "Black"} %{"Wrap" => "Black"}
```
Sources:
- [Printful API Documentation](https://developers.printful.com/docs/)
- [Gelato API - Get Template](https://dashboard.gelato.com/docs/ecommerce/templates/get/)
- [Prodigi API Documentation](https://www.prodigi.com/print-api/docs/reference/)
### Why JSON Options Work Well
| Use Case | Query Pattern | Efficient? |
|----------|---------------|------------|
| Product page - list variants | `where(product_id: ^id)` | Yes - no JSON query |
| Cart - get variant by ID | `Repo.get(Variant, id)` | Yes - no JSON query |
| Display options on page | Access `variant.options` | Yes - in-memory map |
| Find matching variant | `Enum.find` on loaded list | Yes - all variants loaded anyway |
A separate `variant_options` table would add joins and complexity without benefit.
### LiveView Implementation
**On mount**, load product with all variants and extract option types:
```elixir
def mount(_params, _session, socket) do
product = Products.get_product_with_variants(slug)
# Extract unique option names and values from variants
option_types = extract_option_types(product.variants)
# => %{
# "size" => ["S", "M", "L", "XL"],
# "color" => ["Black", "Green", "Blue", "Grey"],
# "sleeves" => ["Short", "Long"]
# }
{:ok, assign(socket, product: product, option_types: option_types, selected: %{})}
end
defp extract_option_types(variants) do
variants
|> Enum.flat_map(fn v -> Map.to_list(v.options) end)
|> Enum.group_by(&elem(&1, 0), &elem(&1, 1))
|> Map.new(fn {k, v} -> {k, Enum.uniq(v)} end)
end
```
### Cascading Selector UI
Standard e-commerce pattern with availability filtering:
```
┌─────────────────────────────────────┐
│ Size: [S] [M] [L] [XL] │ ← User selects "M"
├─────────────────────────────────────┤
│ Color: [Black] [Green] [Blue] │ ← Grey disabled (no M+Grey combo)
├─────────────────────────────────────┤
│ Sleeves: [Short] [Long] │ ← User selects "Long"
├─────────────────────────────────────┤
│ Price: £27.00 │
│ [Add to Cart] │ ← Adds variant "M/Black/Long"
└─────────────────────────────────────┘
```
**Key behaviors:**
1. User selects options one at a time
2. Unavailable combinations are disabled/greyed out
3. Once all options selected, show matching variant's price
4. Add to cart uses the specific variant ID
### Selection Handler
```elixir
def handle_event("select_option", %{"option" => option, "value" => value}, socket) do
selected = Map.put(socket.assigns.selected, option, value)
# Find matching variant (if all options selected)
current_variant = find_matching_variant(socket.assigns.product.variants, selected)
# Determine which options are still available given current selection
available_options = compute_available_options(socket.assigns.product.variants, selected)
{:noreply, assign(socket,
selected: selected,
current_variant: current_variant,
available_options: available_options
)}
end
defp find_matching_variant(variants, selected) do
Enum.find(variants, fn v -> Map.equal?(v.options, selected) end)
end
defp compute_available_options(variants, selected) do
# For each unselected option type, find values that have
# at least one variant matching current selection
# (filters out impossible combinations)
end
```
### Optional: Store Option Order on Product
For consistent selector ordering, optionally store on product:
```elixir
field :option_types, {:array, :string} # ["size", "color", "sleeves"]
```
This defines display order. Can be extracted from provider API on first sync.
---
## Provider Behaviour
```elixir
@callback provider_type() :: String.t()
@callback test_connection(conn) :: {:ok, map()} | {:error, term()}
@callback fetch_products(conn) :: {:ok, [map()]} | {:error, term()}
@callback normalize_product(raw_product) :: map() # Transform to our schema
@callback normalize_variant(raw_variant, product) :: map() # Transform + resolve options
@callback build_order_item(variant, quantity) :: map() # Build provider-specific order item
@callback submit_order(conn, order) :: {:ok, %{provider_order_id: String.t()}} | {:error, term()}
@callback get_order_status(conn, provider_order_id) :: {:ok, map()} | {:error, term()}
```
Each provider (Printify, Prodigi, Gelato) implements this behaviour with provider-specific API calls and data normalization.
---
## Order Submission Flow
When submitting an order, we must send **provider-specific variant identifiers** back to each provider.
### What Each Provider Needs
| Provider | Order Item Fields Required | Source in Our Schema |
|----------|---------------------------|---------------------|
| **Printify** | `product_id`, `variant_id`, `print_provider_id` | `product.provider_product_id`, `variant.provider_variant_id`, `product.provider_data["print_provider_id"]` |
| **Printful** | `sync_variant_id` | `variant.provider_variant_id` |
| **Gelato** | `productUid`, `itemReferenceId` | `variant.provider_variant_id`, our `order_line_item.id` |
| **Prodigi** | `sku`, `attributes` | `variant.provider_variant_id`, `variant.options` |
### Provider `build_order_item/2` Examples
```elixir
# Printify
def build_order_item(variant, quantity) do
product = variant.product
%{
"product_id" => product.provider_product_id,
"variant_id" => String.to_integer(variant.provider_variant_id),
"print_provider_id" => product.provider_data["print_provider_id"],
"quantity" => quantity
}
end
# Printful
def build_order_item(variant, quantity) do
%{
"sync_variant_id" => String.to_integer(variant.provider_variant_id),
"quantity" => quantity
}
end
# Gelato
def build_order_item(variant, quantity) do
%{
"productUid" => variant.provider_variant_id,
"quantity" => quantity,
"itemReferenceId" => Ecto.UUID.generate()
}
end
# Prodigi
def build_order_item(variant, quantity) do
%{
"sku" => variant.provider_variant_id,
"copies" => quantity,
"attributes" => variant.options # {"wrap" => "Black"} etc.
}
end
```
### Order Submission Implementation
```elixir
def submit_to_provider(%Order{} = order) do
order = Repo.preload(order, [:provider_connection, line_items: [product_variant: :product]])
provider = Provider.for_type(order.provider_connection.provider_type)
# Build provider-specific line items
items = Enum.map(order.line_items, fn item ->
provider.build_order_item(item.product_variant, item.quantity)
end)
# Submit via provider behaviour
case provider.submit_order(order.provider_connection, %{
line_items: items,
shipping_address: order.shipping_address,
external_id: order.order_number
}) do
{:ok, %{provider_order_id: provider_id}} ->
order
|> Order.changeset(%{provider_order_id: provider_id, status: "submitted"})
|> Repo.update()
{:error, reason} ->
{:error, reason}
end
end
```
### Key Insight
The `provider_variant_id` field serves a **dual purpose**:
1. **Sync**: We receive it from the provider and store it
2. **Order**: We send it back to the provider when fulfilling
This is why it's stored as a string - different providers use different formats (integers, UUIDs, compound UIDs), but we treat them all as opaque identifiers.
Sources:
- [Printify API Reference](https://developers.printify.com/)
- [Printful API Documentation](https://developers.printful.com/docs/)
- [Gelato API - Create Order](https://dashboard.gelato.com/docs/orders/v4/create/)
- [Prodigi API Documentation](https://www.prodigi.com/print-api/docs/reference/)
---
## Sync Strategy
1. Fetch all products from provider via API
2. For each product:
- Find existing by `[provider_connection_id, provider_product_id]`
- Upsert product with changeset
- Delete removed images, insert new ones
- Upsert variants (delete removed, update existing, insert new)
3. Update `provider_connection.last_synced_at`
Sync runs via Oban worker for async processing with retry on failure.
---
## Files to Create
### Dependencies
Add to `mix.exs`:
- `{:ex_money, "~> 5.0"}` - Currency handling
- `{:ex_money_sql, "~> 1.0"}` - Ecto types for Money
### Migrations
- `*_create_provider_connections.exs`
- `*_create_products.exs`
- `*_create_product_images.exs`
- `*_create_product_variants.exs`
- `*_create_orders.exs`
- `*_create_order_fulfillments.exs`
- `*_create_order_line_items.exs`
- `*_create_order_events.exs`
- `*_create_admin_notifications.exs`
### Schemas
- `lib/simpleshop_theme/products/provider_connection.ex`
- `lib/simpleshop_theme/products/product.ex`
- `lib/simpleshop_theme/products/product_image.ex`
- `lib/simpleshop_theme/products/product_variant.ex`
- `lib/simpleshop_theme/orders/order.ex`
- `lib/simpleshop_theme/orders/order_fulfillment.ex`
- `lib/simpleshop_theme/orders/order_line_item.ex`
- `lib/simpleshop_theme/orders/order_event.ex`
- `lib/simpleshop_theme/admin_notifications/notification.ex`
### Contexts
- `lib/simpleshop_theme/products.ex` - Product queries, sync logic
- `lib/simpleshop_theme/orders.ex` - Order creation, submission
- `lib/simpleshop_theme/admin_notifications.ex` - Admin notification management
### Providers
- `lib/simpleshop_theme/providers/provider.ex` - Behaviour definition
- `lib/simpleshop_theme/providers/printify.ex` - Printify implementation
### Workers
- `lib/simpleshop_theme/sync/product_sync_worker.ex` - Oban worker
### Webhooks
- `lib/simpleshop_theme_web/controllers/webhook_controller.ex`
- `lib/simpleshop_theme/webhooks/printify_handler.ex`
### Notifiers
- `lib/simpleshop_theme_web/notifiers/customer_notifier.ex` - Customer emails
### Support
- `lib/simpleshop_theme/vault.ex` - Credential encryption
---
## Files to Modify
- `lib/simpleshop_theme/printify/client.ex` → Move to `lib/simpleshop_theme/clients/printify.ex`
- `lib/simpleshop_theme/printify/mockup_generator.ex` → Move to `lib/simpleshop_theme/mockups/generator.ex`
- `lib/simpleshop_theme/theme/preview_data.ex` - Query real products when available
- `lib/simpleshop_theme_web/live/shop_live/*.ex` - Use Products context instead of PreviewData
---
## Implementation Phases
### Phase 1: Schema Foundation
1. Create Vault module for credential encryption
2. Create all migrations
3. Create all schema modules
4. Run migrations
### Phase 2: Products Context
1. Create Products context with CRUD
2. Implement sync/upsert logic
3. Create ProductSyncWorker (Oban)
4. Update Printify.Client with get_products endpoint
### Phase 3: Provider Abstraction
1. Create Provider behaviour
2. Implement Printify provider
3. Connect admin UI for provider setup
4. Test sync flow end-to-end
### Phase 4: Orders
1. Create Orders context
2. Implement order submission to provider
3. Add order status tracking
### Phase 5: Integration
1. Update PreviewData to use real products
2. Update shop LiveViews
3. Update cart to use real variant IDs
---
## Notifications Strategy
### Admin Notifications (Internal)
Stored in database, shown in admin dashboard:
```elixir
schema "admin_notifications" do
field :type, :string # "sync_error", "order_failed", "webhook_error"
field :severity, :string # "info", "warning", "error"
field :title, :string
field :message, :string
field :metadata, :map
field :read_at, :utc_datetime
timestamps()
end
```
**Use cases:** Sync failures, order submission errors, provider API issues.
### Customer Notifications
Two mechanisms:
1. **Transactional emails** via `CustomerNotifier` (order confirmation, shipping updates)
2. **Order events** table for timeline display on order page
```elixir
# order_events - visible to customer on their order page
schema "order_events" do
belongs_to :order
field :type, :string # "placed", "paid", "shipped", "delivered"
field :message, :string # Human-readable status
field :metadata, :map # {tracking_number, carrier, fulfillment_id}
timestamps()
end
```
### Files to Create
- `lib/simpleshop_theme/admin_notifications.ex` - Admin notification context
- `lib/simpleshop_theme/admin_notifications/notification.ex` - Schema
- `lib/simpleshop_theme/orders/order_event.ex` - Customer-facing event schema
- `lib/simpleshop_theme_web/notifiers/customer_notifier.ex` - Emails
---
## Data Integrity: Checksum Approach
Use checksums to detect changes during sync (more robust than `updated_at` across providers):
```elixir
# On product schema
field :provider_checksum, :string
# Computing checksum
def compute_checksum(provider_data) do
provider_data
|> normalize_for_checksum()
|> Jason.encode!(pretty: false)
|> then(&:crypto.hash(:sha256, &1))
|> Base.encode16(case: :lower)
end
defp normalize_for_checksum(data) do
data
|> Map.drop(["created_at", "updated_at"]) # Ignore volatile fields
|> sort_keys_recursively()
end
# During sync
def needs_update?(existing_product, provider_data) do
new_checksum = compute_checksum(provider_data)
existing_product.provider_checksum != new_checksum
end
```
**Benefits:**
- Works regardless of provider timestamp support
- Catches any field change
- Avoids unnecessary database writes
- Stable across JSON key ordering variations
---
## Edge Cases (Follow-up Implementation Notes)
| Edge Case | Expected Behavior |
|-----------|-------------------|
| **Provider API down during sync** | Oban retries with exponential backoff (30s → 5min). After max_attempts (3), mark sync as `failed`. Create admin notification. |
| **Product deleted on provider** | Compare local products with provider list during sync. Mark missing products as `status: "archived"` (don't hard delete - preserves order history). |
| **Variant prices change mid-cart** | Cart stores `cached_price` at add-time. On checkout, re-validate current prices. Show warning to customer if price increased, allow proceed if decreased. |
| **`provider_variant_id` format changes** | Already stored as string. Provider behaviour's `normalize_variant/2` handles format. If API version changes, update provider module. |
| **Provider returns malformed data** | Validate in `normalize_product/1`. Log error, skip malformed product, continue batch. Don't fail entire sync. |
| **Rate limiting from provider** | Single sync worker (queue concurrency: 1). Add configurable delay between paginated requests. Respect Retry-After headers. |
| **Sync interrupted mid-batch** | Track sync progress in `provider_connection.sync_metadata`. On resume, continue from last successful page. |
| **Duplicate slugs across providers** | Generate slug with provider suffix on collision: `awesome-shirt-abc123` where `abc123` is short provider product ID. |
| **Webhook signature validation** | Each provider has different signing. Validate HMAC signature before processing. Reject invalid webhooks with 401. |
| **Order submission fails** | Keep order as `status: "pending"`. Retry via Oban worker. After max retries, mark as `status: "failed"`, notify admin. |
---
## Testing Strategy
### Test Setup
**Mocking External APIs (Mox pattern):**
```elixir
# test/support/mocks.ex
Mox.defmock(SimpleshopTheme.Clients.MockPrintify, for: SimpleshopTheme.Clients.PrintifyBehaviour)
# config/test.exs
config :simpleshop_theme, :printify_client, SimpleshopTheme.Clients.MockPrintify
```
**Oban testing:**
```elixir
use Oban.Testing, repo: SimpleshopTheme.Repo
# Jobs run synchronously in tests via perform_job/2
```
### Unit Tests
| Module | Test Coverage |
|--------|---------------|
| **Products.Product** | Changeset validation, slug generation, status transitions |
| **Products.ProductVariant** | Options validation, price/cost integrity |
| **Products.ProviderConnection** | Credential encryption/decryption, config validation |
| **Providers.Printify** | `normalize_product/1`, `normalize_variant/2`, `build_order_item/2` |
| **Orders.Order** | Status aggregation from fulfillments, total calculation |
| **Orders.OrderFulfillment** | Status transitions, tracking updates |
| **AdminNotifications** | Create, list_unread, mark_read |
### Integration Tests
| Flow | Test Scenarios |
|------|----------------|
| **Product Sync** | Successful sync, partial failures, pagination, checksum skip, archived products |
| **Order Creation** | Single provider, multi-provider split into fulfillments |
| **Order Submission** | Success, API failure with retry, partial fulfillment submission |
| **Webhook Handling** | Valid signature, invalid signature rejection, status updates |
| **Variant Selector** | Option extraction, availability filtering, variant matching |
### Example Test Cases
```elixir
# test/simpleshop_theme/products_test.exs
describe "sync_products/1" do
test "syncs products from provider" do
conn = provider_connection_fixture()
expect(MockPrintify, :list_products, fn _shop_id ->
{:ok, [printify_product_fixture()]}
end)
assert {:ok, %{synced: 1, failed: 0}} = Products.sync_products(conn)
assert [product] = Products.list_products()
assert product.title == "Test Product"
end
test "continues on single product failure" do
expect(MockPrintify, :list_products, fn _ ->
{:ok, [valid_product(), malformed_product()]}
end)
assert {:ok, %{synced: 1, failed: 1}} = Products.sync_products(conn)
end
test "skips unchanged products via checksum" do
existing = product_fixture(provider_checksum: "abc123")
expect(MockPrintify, :list_products, fn _ ->
{:ok, [product_with_same_checksum()]}
end)
assert {:ok, %{synced: 0, skipped: 1}} = Products.sync_products(conn)
end
end
# test/simpleshop_theme/orders_test.exs
describe "create_order_from_cart/1" do
test "splits cart into fulfillments by provider" do
printify_variant = variant_fixture(provider: :printify)
gelato_variant = variant_fixture(provider: :gelato)
cart = [
%{variant_id: printify_variant.id, quantity: 1},
%{variant_id: gelato_variant.id, quantity: 2}
]
assert {:ok, order} = Orders.create_order_from_cart(cart, customer_params())
assert length(order.order_fulfillments) == 2
end
end
# test/simpleshop_theme/sync/product_sync_worker_test.exs
describe "perform/1" do
test "retries on API failure" do
expect(MockPrintify, :list_products, fn _ -> {:error, :timeout} end)
assert {:error, :timeout} = perform_job(ProductSyncWorker, %{conn_id: conn.id})
# Oban will retry automatically
end
test "creates admin notification on final failure" do
# After max_attempts...
assert [notification] = AdminNotifications.list_unread()
assert notification.type == "sync_error"
end
end
```
### Test Fixtures
```elixir
# test/support/fixtures/products_fixtures.ex
def provider_connection_fixture(attrs \\ %{}) do
{:ok, conn} = Products.create_provider_connection(%{
provider_type: "printify",
name: "Test Connection",
api_key: "test_key",
config: %{"shop_id" => "123"}
} |> Map.merge(attrs))
conn
end
def product_fixture(attrs \\ %{}) do
conn = attrs[:provider_connection] || provider_connection_fixture()
{:ok, product} = Products.create_product(%{
provider_connection_id: conn.id,
provider_product_id: "ext_#{System.unique_integer()}",
title: "Test Product",
slug: "test-product-#{System.unique_integer()}",
status: "active"
} |> Map.merge(attrs))
product
end
def printify_product_fixture do
%{
"id" => "12345",
"title" => "Test Product",
"description" => "A test product",
"options" => [%{"name" => "Size", "values" => [%{"id" => 1, "title" => "M"}]}],
"variants" => [%{"id" => 100, "options" => [1], "price" => 2500}],
"images" => [%{"src" => "https://example.com/img.jpg", "position" => 0}]
}
end
```
### Coverage Goals
| Area | Target |
|------|--------|
| Schema modules | 100% (validation, associations) |
| Context functions | 90%+ (all public functions) |
| Provider implementations | 90%+ (normalize, build functions) |
| Oban workers | 80%+ (happy path, error handling) |
| LiveView interactions | 70%+ (critical user flows) |
---
## Verification (Manual)
- Create Printify connection with API key
- `Products.test_provider_connection(conn)` returns shop info
2. **Product Sync**
- Run `ProductSyncWorker.enqueue(conn.id)`
- Products appear in database with images and variants
- Re-sync updates existing, doesn't duplicate
3. **Order Submission**
- Create order from cart
- `Orders.submit_to_provider(order)` returns provider_order_id
- Order status updates from provider
4. **Shop Pages**
- `/collections/all` shows real products
- `/products/:slug` shows product details
- Add to cart works with real variant IDs