d97918d66a
- Create PROGRESS.md as single source of truth for status - Slim ROADMAP.md to vision only (~100 lines, down from ~500) - Expand CLAUDE.md with streams, auth routing, forms, workflow - Convert AGENTS.md to stub pointing to CLAUDE.md - Update plan files with status headers, remove progress trackers Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
58 KiB
58 KiB
Plan: Products Context with Provider Integration
Status: Phase 1 Complete (
c5c06d9) - See PROGRESS.md for current status.
Goal
Build a Products context that syncs products from external POD providers (Printify first), stores them locally, and enables order submission for fulfillment.
Core Flow
- Connect - Authenticate with provider (API key or OAuth)
- Sync - Fetch products from provider, upsert locally
- Order - Submit orders to provider for fulfillment
Current Domain Analysis
SimpleShop has 6 well-defined domains with clear boundaries:
| Domain | Purpose | Schemas | Public Functions |
|---|---|---|---|
| Accounts | User authentication & sessions | User, UserToken | 15 |
| Settings | Theme configuration persistence | Setting, ThemeSettings | 6 |
| Theme | CSS generation, caching, presets | (structs only) | N/A |
| Media | Image upload/storage | Image | 8 |
| Images | Image optimization pipeline | (Oban workers) | N/A |
| Printify | External API integration | (no schemas) | API client |
The new Products context will be a new top-level domain that:
- Owns product/variant schemas
- Coordinates with Printify (and future providers) for sync
- Provides data to shop LiveViews (replacing PreviewData)
Architecture Diagram
┌─────────────────────────────────────────────────────────────────────────────┐
│ SimpleshopTheme │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌──────────────────────────────────────────────────────────────────────┐ │
│ │ WEB LAYER │ │
│ │ SimpleshopThemeWeb │ │
│ │ ┌────────────────┐ ┌────────────────┐ ┌─────────────────────────┐ │ │
│ │ │ Shop LiveViews │ │ Admin LiveViews│ │ Theme Editor LiveView │ │ │
│ │ │ - ProductShow │ │ - UserLogin │ │ - ThemeLive.Index │ │ │
│ │ │ - Collection │ │ - UserSettings │ │ - Preview iframe │ │ │
│ │ │ - Cart │ │ │ │ │ │ │
│ │ └───────┬─────────┘ └────────────────┘ └────────┬───────────────┘ │ │
│ └──────────┼────────────────────────────────────────┼──────────────────┘ │
│ │ │ │
│ ▼ ▼ │
│ ┌──────────────────────────────────────────────────────────────────────┐ │
│ │ DOMAIN LAYER │ │
│ │ │ │
│ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │
│ │ │ Accounts │ │ Settings │ │ Theme │ │ │
│ │ │ │ │ │ │ │ │ │
│ │ │ • User │ │ • Setting │ │ • CSSCache │ │ │
│ │ │ • UserToken │ │ • Theme │ │ • CSSGen │ │ │
│ │ │ │ │ Settings │ │ • Presets │ │ │
│ │ │ │ │ │ │ • Fonts │ │ │
│ │ │ │ │ │ │ • Preview │ │ │
│ │ └─────────────┘ └──────┬──────┘ │ Data (*) │ │ │
│ │ │ └─────────────┘ │ │
│ │ │ │ │
│ │ ┌─────────────┐ ┌──────▼──────┐ ┌─────────────┐ │ │
│ │ │ Media │ │ Images │ │ Printify │ │ │
│ │ │ │ │ │ │ │ │ │
│ │ │ • Image │◄──│ • Optimizer │ │ • Client │ │ │
│ │ │ (upload) │ │ • Worker │ │ • Catalog │ (NEW) │ │
│ │ │ │ │ • Cache │ │ • Mockup │ │ │
│ │ └─────────────┘ └─────────────┘ └──────┬──────┘ │ │
│ │ │ │ │
│ │ ┌───────────────────────────────────────────▼───────────────────┐ │ │
│ │ │ Products (NEW) │ │ │
│ │ │ │ │ │
│ │ │ ┌─────────────────────┐ ┌────────────────────────────┐ │ │ │
│ │ │ │ Provider Layer │ │ Product Layer │ │ │ │
│ │ │ │ │ │ │ │ │ │
│ │ │ │ • ProviderConnection│───▶│ • Product │ │ │ │
│ │ │ │ • Provider behaviour│ │ • ProductImage │ │ │ │
│ │ │ │ • PrintifyProvider │ │ • ProductVariant │ │ │ │
│ │ │ │ • (future providers)│ │ │ │ │ │
│ │ │ └─────────────────────┘ └────────────────────────────┘ │ │ │
│ │ │ │ │ │
│ │ │ ┌─────────────────────────────────────────────────────────┐ │ │ │
│ │ │ │ Sync Layer │ │ │ │
│ │ │ │ • ProductSyncWorker (Oban) │ │ │ │
│ │ │ │ • Upsert logic │ │ │ │
│ │ │ └─────────────────────────────────────────────────────────┘ │ │ │
│ │ └────────────────────────────────────────────────────────────────┘ │ │
│ │ │ │
│ │ ┌────────────────────────────────────────────────────────────────┐ │ │
│ │ │ Orders (NEW) │ │ │
│ │ │ │ │ │
│ │ │ • Order • submit_to_provider/1 │ │ │
│ │ │ • OrderLineItem • track_status/1 │ │ │
│ │ └────────────────────────────────────────────────────────────────┘ │ │
│ │ │ │
│ └───────────────────────────────────────────────────────────────────────┘ │
│ │
│ ┌───────────────────────────────────────────────────────────────────────┐ │
│ │ INFRASTRUCTURE │ │
│ │ Repo (SQLite) • Vault (encryption) • Oban (background jobs) │ │
│ └───────────────────────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
(*) PreviewData will be replaced by Products context queries
Database ERD
┌─────────────────────────────────────────────────────────────────────────────┐
│ CURRENT SCHEMA │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌──────────────────┐ ┌──────────────────┐ │
│ │ users │ │ users_tokens │ │
│ ├──────────────────┤ ├──────────────────┤ │
│ │ id (binary_id) │◄────────│ user_id (FK) │ │
│ │ email │ │ token │ │
│ │ hashed_password │ │ context │ │
│ │ confirmed_at │ │ sent_to │ │
│ │ inserted_at │ │ inserted_at │ │
│ │ updated_at │ └──────────────────┘ │
│ └──────────────────┘ │
│ │
│ ┌──────────────────┐ ┌──────────────────┐ │
│ │ settings │ │ images │ │
│ ├──────────────────┤ ├──────────────────┤ │
│ │ id (binary_id) │ │ id (binary_id) │ │
│ │ key │ │ filename │ │
│ │ value (JSON) │ │ content_type │ │
│ │ inserted_at │ │ byte_size │ │
│ │ updated_at │ │ width │ │
│ └──────────────────┘ │ height │ │
│ │ storage_key │ │
│ │ inserted_at │ │
│ │ updated_at │ │
│ └──────────────────┘ │
├─────────────────────────────────────────────────────────────────────────────┤
│ NEW SCHEMA (Products) │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌────────────────────────┐ │
│ │ provider_connections │ │
│ ├────────────────────────┤ │
│ │ id (binary_id) │ │
│ │ provider_type │──┐ "printify" | "prodigi" | "gelato" │
│ │ name │ │ │
│ │ enabled │ │ │
│ │ api_key_encrypted │ │ (Vault encrypted) │
│ │ config (JSON) │ │ {"shop_id": "123"} │
│ │ last_synced_at │ │ │
│ │ sync_status │ │ │
│ │ inserted_at │ │ │
│ │ updated_at │ │ │
│ └───────────┬────────────┘ │ │
│ │ │ │
│ │ 1 │ │
│ ▼ * │ │
│ ┌────────────────────────┐ │ │
│ │ products │ │ │
│ ├────────────────────────┤ │ │
│ │ id (binary_id) │ │ │
│ │ provider_connection_id │◄─┘ │
│ │ provider_product_id │ External ID (unique per connection) │
│ │ title │ │
│ │ description │ │
│ │ slug │ URL-friendly (unique globally) │
│ │ status │ "active" | "draft" | "archived" │
│ │ visible │ │
│ │ category │ │
│ │ option_types (JSON) │ ["size", "color"] - display order │
│ │ provider_data (JSON) │ Provider-specific fields (blueprint_id etc) │
│ │ inserted_at │ │
│ │ updated_at │ │
│ └───────────┬────────────┘ │
│ │ │
│ │ 1 │
│ ┌─────────┴─────────┐ │
│ ▼ * ▼ * │
│ ┌──────────────────┐ ┌──────────────────────┐ │
│ │ product_images │ │ product_variants │ │
│ ├──────────────────┤ ├──────────────────────┤ │
│ │ id (binary_id) │ │ id (binary_id) │ │
│ │ product_id (FK) │ │ product_id (FK) │ │
│ │ src │ │ provider_variant_id │ │
│ │ position │ │ title │ "Large / Black" │
│ │ alt │ │ sku │ │
│ │ inserted_at │ │ price │ Selling price (pence) │
│ │ updated_at │ │ compare_at_price │ For sale display │
│ └──────────────────┘ │ cost │ Provider cost (profit) │
│ │ options (JSON) │ {"size": "L", "color": ""} │
│ │ is_enabled │ │
│ │ is_available │ │
│ │ inserted_at │ │
│ │ updated_at │ │
│ └──────────────────────┘ │
│ │
├─────────────────────────────────────────────────────────────────────────────┤
│ NEW SCHEMA (Orders) │
│ │
│ Customer sees ONE order. Fulfillments are internal (one per provider). │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌────────────────────────┐ │
│ │ orders │ Customer-facing order │
│ ├────────────────────────┤ │
│ │ id (binary_id) │ │
│ │ order_number │ "SS-250125-A1B2" (customer sees this) │
│ │ status │ pending → processing → shipped → delivered │
│ │ payment_status │ pending | paid | refunded │
│ │ customer_email │ │
│ │ shipping_address (JSON)│ │
│ │ subtotal │ (Money - ex_money) │
│ │ shipping_cost │ (Money - ex_money) │
│ │ total │ (Money - ex_money) │
│ │ total_cost │ Provider costs (for profit) │
│ │ inserted_at │ │
│ │ updated_at │ │
│ └───────────┬────────────┘ │
│ │ │
│ │ 1 │
│ ┌─────────┴─────────────────────────┐ │
│ ▼ * ▼ * │
│ ┌────────────────────────┐ ┌─────────────────────────┐ │
│ │ order_fulfillments │ │ order_line_items │ │
│ ├────────────────────────┤ ├─────────────────────────┤ │
│ │ id (binary_id) │ │ id (binary_id) │ │
│ │ order_id (FK) │ │ order_id (FK) │ │
│ │ provider_connection_id │ │ order_fulfillment_id(FK)│ Which provider │
│ │ provider_order_id │ │ product_variant_id (FK) │ │
│ │ status │ │ quantity │ │
│ │ tracking_number │ │ unit_price │ │
│ │ tracking_url │ │ unit_cost │ │
│ │ shipped_at │ │ inserted_at │ │
│ │ inserted_at │ │ updated_at │ │
│ │ updated_at │ └─────────────────────────┘ │
│ └────────────────────────┘ │
│ │
│ ┌────────────────────────┐ │
│ │ order_events │ Timeline visible to customer │
│ ├────────────────────────┤ │
│ │ id (binary_id) │ │
│ │ order_id (FK) │ │
│ │ type │ "placed" | "paid" | "shipped" | "delivered" │
│ │ message │ "Your T-Shirt has shipped via Royal Mail" │
│ │ metadata (JSON) │ {tracking_number, carrier, etc} │
│ │ inserted_at │ │
│ └────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
Printify Code Reuse
Existing Client (Highly Reusable)
The existing Printify.Client is directly reusable:
# HTTP infrastructure (no changes needed)
Client.get/2, Client.post/3, Client.delete/2
# Shop management
Client.get_shops/0, Client.get_shop_id/0
# Product operations (already implemented)
Client.get_product/2, Client.create_product/2, Client.delete_product/2
To add:
Client.list_products/1 # GET /shops/:shop_id/products.json
Client.create_order/2 # POST /shops/:shop_id/orders.json
Client.get_order/2 # GET /shops/:shop_id/orders/:id.json
MockupGenerator Refactoring
Extract reusable helpers into new Printify.Catalog module:
| Function | Current | Target |
|---|---|---|
find_blueprint/1 |
MockupGenerator | Printify.Catalog |
find_print_provider/2 |
MockupGenerator | Printify.Catalog |
get_variant_info/1 |
MockupGenerator | Printify.Catalog |
calculate_cover_scale/2 |
MockupGenerator | Keep (mockup-specific) |
product_definitions/0 |
MockupGenerator | Keep (sample data) |
Module structure:
lib/simpleshop_theme/clients/
├── printify.ex # Printify HTTP client (moved from printify/client.ex)
├── gelato.ex # Gelato HTTP client
└── prodigi.ex # Prodigi HTTP client
lib/simpleshop_theme/providers/
├── provider.ex # Behaviour definition
├── printify.ex # Printify implementation (uses Clients.Printify)
├── gelato.ex # Gelato implementation (uses Clients.Gelato)
└── prodigi.ex # Prodigi implementation (uses Clients.Prodigi)
lib/simpleshop_theme/mockups/
└── generator.ex # Mockup generation (currently uses Clients.Printify)
# Provider-agnostic location for future flexibility
lib/simpleshop_theme/printify/
└── catalog.ex # Blueprint/variant discovery helpers (Printify-specific)
Each provider module uses its corresponding client. The mockup generator is in a provider-agnostic location.
Database Schema
Tables Overview
| Table | Purpose |
|---|---|
provider_connections |
Provider credentials + config (one per type) |
products |
Core product data synced from provider |
product_images |
Product images (multiple per product) |
product_variants |
Size/color variants with pricing |
orders |
Customer orders |
order_line_items |
Items within orders |
Key Design Decisions
-
Provider IDs on product directly (not a separate linking table)
products.provider_product_idstores external IDproduct_variants.provider_variant_idstores external variant ID- Unique constraint:
[:provider_connection_id, :provider_product_id]
-
Credentials encrypted in database
- Use
SimpleshopTheme.Vaultfor at-rest encryption api_key_encrypted,oauth_access_token_encryptedfields
- Use
-
Cost tracking for profit calculation
product_variants.cost- cost from providerorder_line_items.unit_cost- snapshot at order timeorders.total_cost- sum for profit reporting
-
Provider type as enum (not separate table)
- Hardcoded:
~w(printify prodigi gelato) - Keeps type safety for provider-specific logic
- Hardcoded:
Schema Details
provider_connections
field :provider_type, :string # "printify", "prodigi", "gelato"
field :name, :string # "My Printify Account"
field :enabled, :boolean
field :api_key_encrypted, :binary # Encrypted API key
field :config, :map # {"shop_id": "12345"}
field :last_synced_at, :utc_datetime
field :sync_status, :string # pending, syncing, completed, failed
products
belongs_to :provider_connection
field :provider_product_id, :string # External ID from provider
field :title, :string
field :description, :text
field :slug, :string # URL-friendly, unique
field :status, :string # active, draft, archived
field :visible, :boolean
field :category, :string
field :option_types, {:array, :string} # ["Colors", "Sizes"] - option names in display order
# Extracted from provider's product.options[].name on sync
# Used to order the variant selector UI
field :provider_data, :map # Provider-specific fields stored here
# Printify: %{"blueprint_id" => 145, "print_provider_id" => 29, ...}
# Keeps products table generic across providers
product_variants
belongs_to :product
field :provider_variant_id, :string # CRITICAL: The ID we send back to provider when ordering
# Printify: variant_id (e.g., "12345")
# Printful: sync_variant_id (e.g., "4752058849")
# Gelato: productUid (e.g., "apparel_product_gca_t-shirt_gsi_m_gco_white")
# Prodigi: sku (e.g., "GLOBAL-CAN-10x10")
field :title, :string # "Large / Black"
field :sku, :string # Shop's internal SKU (optional)
field :price, :integer # Selling price (pence)
field :compare_at_price, :integer # Original price (for sales)
field :cost, :integer # Provider cost (for profit)
field :options, :map # {"size": "Large", "color": "Black"}
# JSON is appropriate here because:
# - Options always loaded with variant (no join needed)
# - No need to query "find all Large variants"
# - Matches provider data format (easy sync)
# - Simpler than a separate options table
field :is_enabled, :boolean
field :is_available, :boolean # In stock at provider
orders (customer-facing)
# Customer sees ONE order, even if fulfilled by multiple providers
field :order_number, :string # "SS-250125-A1B2" (customer-visible)
field :status, :string # Aggregate: pending → processing → shipped → delivered
field :payment_status, :string # pending, paid, refunded
field :customer_email, :string
field :shipping_address, :map
field :subtotal, Money.Ecto.Map.Type # Using ex_money for currency safety
field :shipping_cost, Money.Ecto.Map.Type
field :total, Money.Ecto.Map.Type
field :total_cost, Money.Ecto.Map.Type # Sum of provider costs
has_many :order_fulfillments # One per provider
has_many :order_line_items
has_many :order_events # Timeline for customer
order_fulfillments (one per provider)
belongs_to :order
belongs_to :provider_connection
field :provider_order_id, :string # External ID after submission
field :status, :string # pending → submitted → processing → shipped
field :tracking_number, :string
field :tracking_url, :string
field :shipped_at, :utc_datetime
has_many :order_line_items # Items fulfilled by this provider
order_line_items
belongs_to :order
belongs_to :order_fulfillment # Links to which provider ships this item
belongs_to :product_variant
field :quantity, :integer
field :unit_price, Money.Ecto.Map.Type
field :unit_cost, Money.Ecto.Map.Type
# Per-item tracking (provider may ship items separately)
field :status, :string # pending → shipped → delivered
field :tracking_number, :string
field :tracking_url, :string
field :shipped_at, :utc_datetime
order_events (customer timeline)
belongs_to :order
field :type, :string # Event types (see below)
field :message, :string # "Your T-Shirt and Mug have shipped!"
field :metadata, :map # {tracking_number, carrier, items, etc}
# Event types:
# - "placed" → Order placed
# - "paid" → Payment confirmed
# - "fulfillment_shipped" → Some items shipped (granular per-fulfillment)
# - "shipped" → ALL items now shipped
# - "delivered" → ALL items delivered
# Example metadata for fulfillment_shipped:
# %{fulfillment_id: "uuid", items: ["T-Shirt", "Mug"],
# carrier: "Royal Mail", tracking_number: "RM123", tracking_url: "..."}
Variant Selector UX Pattern
Products with multiple option types (e.g., size × color × sleeves) can have many variants:
- 4 sizes × 4 colors × 2 sleeves = 32 variants
Data Model
Each variant stores its specific combination as JSON with human-readable labels:
%{id: "v1", options: %{"size" => "M", "color" => "Black", "sleeves" => "Short"}, price: 2500}
%{id: "v2", options: %{"size" => "M", "color" => "Black", "sleeves" => "Long"}, price: 2700}
# ... etc
Provider Option Structure (Important)
Providers like Printify use a two-tier option system:
Product level - defines options with IDs and labels:
{
"options": [
{ "name": "Colors", "type": "color", "values": [
{ "id": 751, "title": "Solid White" },
{ "id": 752, "title": "Black" }
]},
{ "name": "Sizes", "type": "size", "values": [
{ "id": 2, "title": "S" },
{ "id": 3, "title": "M" }
]}
]
}
Variant level - only stores option IDs:
{ "id": 12345, "options": [751, 2], "title": "Solid White / S" }
Sync Strategy: Denormalize to Labels
During sync, we resolve IDs to labels and store the human-readable version:
def normalize_variant(provider_variant, product_options) do
# Build ID -> label lookup from product options
option_lookup = build_option_lookup(product_options)
# => %{751 => {"Colors", "Solid White"}, 2 => {"Sizes", "S"}}
# Convert variant's [751, 2] to %{"Colors" => "Solid White", "Sizes" => "S"}
options =
provider_variant["options"]
|> Enum.map(&Map.get(option_lookup, &1))
|> Enum.reject(&is_nil/1)
|> Map.new()
%{
provider_variant_id: to_string(provider_variant["id"]),
title: provider_variant["title"],
options: options,
# ... other fields
}
end
Why denormalize?
- No need to join to an options table on every product page load
- Labels rarely change (and if they do, re-sync updates them)
- Simpler queries, faster rendering
- Keeps our schema provider-agnostic (we just store labels, not provider-specific IDs)
Provider Compatibility
All major POD providers can be normalized to our options: %{"Size" => "M", "Color" => "Black"} schema:
| Provider | Raw Variant Structure | Normalization |
|---|---|---|
| Printify | options: [751, 2] (IDs only, labels at product level) |
Resolve IDs → labels during sync |
| Printful | color: "Black", size: "M" (flat attributes) |
Map attribute keys to standard names |
| Gelato | variantOptions: [{name: "Size", value: "M"}] (array) |
Convert array to map |
| Prodigi | attributes: {"wrap": "Black"} (object) |
Direct mapping |
Each provider's normalize_variant/2 function handles the transformation:
# Printify - resolve IDs to labels
%{"options" => [751, 2]} → %{"Colors" => "Solid White", "Sizes" => "S"}
# Printful - rename keys
%{"color" => "Black", "size" => "M"} → %{"Color" => "Black", "Size" => "M"}
# Gelato - array to map
[%{"name" => "Size", "value" => "M"}] → %{"Size" => "M"}
# Prodigi - direct (already a map)
%{"wrap" => "Black"} → %{"Wrap" => "Black"}
Sources:
Why JSON Options Work Well
| Use Case | Query Pattern | Efficient? |
|---|---|---|
| Product page - list variants | where(product_id: ^id) |
Yes - no JSON query |
| Cart - get variant by ID | Repo.get(Variant, id) |
Yes - no JSON query |
| Display options on page | Access variant.options |
Yes - in-memory map |
| Find matching variant | Enum.find on loaded list |
Yes - all variants loaded anyway |
A separate variant_options table would add joins and complexity without benefit.
LiveView Implementation
On mount, load product with all variants and extract option types:
def mount(_params, _session, socket) do
product = Products.get_product_with_variants(slug)
# Extract unique option names and values from variants
option_types = extract_option_types(product.variants)
# => %{
# "size" => ["S", "M", "L", "XL"],
# "color" => ["Black", "Green", "Blue", "Grey"],
# "sleeves" => ["Short", "Long"]
# }
{:ok, assign(socket, product: product, option_types: option_types, selected: %{})}
end
defp extract_option_types(variants) do
variants
|> Enum.flat_map(fn v -> Map.to_list(v.options) end)
|> Enum.group_by(&elem(&1, 0), &elem(&1, 1))
|> Map.new(fn {k, v} -> {k, Enum.uniq(v)} end)
end
Cascading Selector UI
Standard e-commerce pattern with availability filtering:
┌─────────────────────────────────────┐
│ Size: [S] [M] [L] [XL] │ ← User selects "M"
├─────────────────────────────────────┤
│ Color: [Black] [Green] [Blue] │ ← Grey disabled (no M+Grey combo)
├─────────────────────────────────────┤
│ Sleeves: [Short] [Long] │ ← User selects "Long"
├─────────────────────────────────────┤
│ Price: £27.00 │
│ [Add to Cart] │ ← Adds variant "M/Black/Long"
└─────────────────────────────────────┘
Key behaviors:
- User selects options one at a time
- Unavailable combinations are disabled/greyed out
- Once all options selected, show matching variant's price
- Add to cart uses the specific variant ID
Selection Handler
def handle_event("select_option", %{"option" => option, "value" => value}, socket) do
selected = Map.put(socket.assigns.selected, option, value)
# Find matching variant (if all options selected)
current_variant = find_matching_variant(socket.assigns.product.variants, selected)
# Determine which options are still available given current selection
available_options = compute_available_options(socket.assigns.product.variants, selected)
{:noreply, assign(socket,
selected: selected,
current_variant: current_variant,
available_options: available_options
)}
end
defp find_matching_variant(variants, selected) do
Enum.find(variants, fn v -> Map.equal?(v.options, selected) end)
end
defp compute_available_options(variants, selected) do
# For each unselected option type, find values that have
# at least one variant matching current selection
# (filters out impossible combinations)
end
Optional: Store Option Order on Product
For consistent selector ordering, optionally store on product:
field :option_types, {:array, :string} # ["size", "color", "sleeves"]
This defines display order. Can be extracted from provider API on first sync.
Provider Behaviour
@callback provider_type() :: String.t()
@callback test_connection(conn) :: {:ok, map()} | {:error, term()}
@callback fetch_products(conn) :: {:ok, [map()]} | {:error, term()}
@callback normalize_product(raw_product) :: map() # Transform to our schema
@callback normalize_variant(raw_variant, product) :: map() # Transform + resolve options
@callback build_order_item(variant, quantity) :: map() # Build provider-specific order item
@callback submit_order(conn, order) :: {:ok, %{provider_order_id: String.t()}} | {:error, term()}
@callback get_order_status(conn, provider_order_id) :: {:ok, map()} | {:error, term()}
Each provider (Printify, Prodigi, Gelato) implements this behaviour with provider-specific API calls and data normalization.
Order Submission Flow
When submitting an order, we must send provider-specific variant identifiers back to each provider.
What Each Provider Needs
| Provider | Order Item Fields Required | Source in Our Schema |
|---|---|---|
| Printify | product_id, variant_id, print_provider_id |
product.provider_product_id, variant.provider_variant_id, product.provider_data["print_provider_id"] |
| Printful | sync_variant_id |
variant.provider_variant_id |
| Gelato | productUid, itemReferenceId |
variant.provider_variant_id, our order_line_item.id |
| Prodigi | sku, attributes |
variant.provider_variant_id, variant.options |
Provider build_order_item/2 Examples
# Printify
def build_order_item(variant, quantity) do
product = variant.product
%{
"product_id" => product.provider_product_id,
"variant_id" => String.to_integer(variant.provider_variant_id),
"print_provider_id" => product.provider_data["print_provider_id"],
"quantity" => quantity
}
end
# Printful
def build_order_item(variant, quantity) do
%{
"sync_variant_id" => String.to_integer(variant.provider_variant_id),
"quantity" => quantity
}
end
# Gelato
def build_order_item(variant, quantity) do
%{
"productUid" => variant.provider_variant_id,
"quantity" => quantity,
"itemReferenceId" => Ecto.UUID.generate()
}
end
# Prodigi
def build_order_item(variant, quantity) do
%{
"sku" => variant.provider_variant_id,
"copies" => quantity,
"attributes" => variant.options # {"wrap" => "Black"} etc.
}
end
Order Submission Implementation
def submit_to_provider(%Order{} = order) do
order = Repo.preload(order, [:provider_connection, line_items: [product_variant: :product]])
provider = Provider.for_type(order.provider_connection.provider_type)
# Build provider-specific line items
items = Enum.map(order.line_items, fn item ->
provider.build_order_item(item.product_variant, item.quantity)
end)
# Submit via provider behaviour
case provider.submit_order(order.provider_connection, %{
line_items: items,
shipping_address: order.shipping_address,
external_id: order.order_number
}) do
{:ok, %{provider_order_id: provider_id}} ->
order
|> Order.changeset(%{provider_order_id: provider_id, status: "submitted"})
|> Repo.update()
{:error, reason} ->
{:error, reason}
end
end
Key Insight
The provider_variant_id field serves a dual purpose:
- Sync: We receive it from the provider and store it
- Order: We send it back to the provider when fulfilling
This is why it's stored as a string - different providers use different formats (integers, UUIDs, compound UIDs), but we treat them all as opaque identifiers.
Sources:
- Printify API Reference
- Printful API Documentation
- Gelato API - Create Order
- Prodigi API Documentation
Sync Strategy
- Fetch all products from provider via API
- For each product:
- Find existing by
[provider_connection_id, provider_product_id] - Upsert product with changeset
- Delete removed images, insert new ones
- Upsert variants (delete removed, update existing, insert new)
- Find existing by
- Update
provider_connection.last_synced_at
Sync runs via Oban worker for async processing with retry on failure.
Files to Create
Dependencies
Add to mix.exs:
{:ex_money, "~> 5.0"}- Currency handling{:ex_money_sql, "~> 1.0"}- Ecto types for Money
Migrations
*_create_provider_connections.exs*_create_products.exs*_create_product_images.exs*_create_product_variants.exs*_create_orders.exs*_create_order_fulfillments.exs*_create_order_line_items.exs*_create_order_events.exs*_create_admin_notifications.exs
Schemas
lib/simpleshop_theme/products/provider_connection.exlib/simpleshop_theme/products/product.exlib/simpleshop_theme/products/product_image.exlib/simpleshop_theme/products/product_variant.exlib/simpleshop_theme/orders/order.exlib/simpleshop_theme/orders/order_fulfillment.exlib/simpleshop_theme/orders/order_line_item.exlib/simpleshop_theme/orders/order_event.exlib/simpleshop_theme/admin_notifications/notification.ex
Contexts
lib/simpleshop_theme/products.ex- Product queries, sync logiclib/simpleshop_theme/orders.ex- Order creation, submissionlib/simpleshop_theme/admin_notifications.ex- Admin notification management
Providers
lib/simpleshop_theme/providers/provider.ex- Behaviour definitionlib/simpleshop_theme/providers/printify.ex- Printify implementation
Workers
lib/simpleshop_theme/sync/product_sync_worker.ex- Oban worker
Webhooks
lib/simpleshop_theme_web/controllers/webhook_controller.exlib/simpleshop_theme/webhooks/printify_handler.ex
Notifiers
lib/simpleshop_theme_web/notifiers/customer_notifier.ex- Customer emails
Support
lib/simpleshop_theme/vault.ex- Credential encryption
Files to Modify
lib/simpleshop_theme/printify/client.ex→ Move tolib/simpleshop_theme/clients/printify.exlib/simpleshop_theme/printify/mockup_generator.ex→ Move tolib/simpleshop_theme/mockups/generator.exlib/simpleshop_theme/theme/preview_data.ex- Query real products when availablelib/simpleshop_theme_web/live/shop_live/*.ex- Use Products context instead of PreviewData
Implementation Phases
Phase 1: Schema Foundation
- Create Vault module for credential encryption
- Create all migrations
- Create all schema modules
- Run migrations
Phase 2: Products Context
- Create Products context with CRUD
- Implement sync/upsert logic
- Create ProductSyncWorker (Oban)
- Update Printify.Client with get_products endpoint
Phase 3: Provider Abstraction
- Create Provider behaviour
- Implement Printify provider
- Connect admin UI for provider setup
- Test sync flow end-to-end
Phase 4: Orders
- Create Orders context
- Implement order submission to provider
- Add order status tracking
Phase 5: Integration
- Update PreviewData to use real products
- Update shop LiveViews
- Update cart to use real variant IDs
Notifications Strategy
Admin Notifications (Internal)
Stored in database, shown in admin dashboard:
schema "admin_notifications" do
field :type, :string # "sync_error", "order_failed", "webhook_error"
field :severity, :string # "info", "warning", "error"
field :title, :string
field :message, :string
field :metadata, :map
field :read_at, :utc_datetime
timestamps()
end
Use cases: Sync failures, order submission errors, provider API issues.
Customer Notifications
Two mechanisms:
- Transactional emails via
CustomerNotifier(order confirmation, shipping updates) - Order events table for timeline display on order page
# order_events - visible to customer on their order page
schema "order_events" do
belongs_to :order
field :type, :string # "placed", "paid", "shipped", "delivered"
field :message, :string # Human-readable status
field :metadata, :map # {tracking_number, carrier, fulfillment_id}
timestamps()
end
Files to Create
lib/simpleshop_theme/admin_notifications.ex- Admin notification contextlib/simpleshop_theme/admin_notifications/notification.ex- Schemalib/simpleshop_theme/orders/order_event.ex- Customer-facing event schemalib/simpleshop_theme_web/notifiers/customer_notifier.ex- Emails
Data Integrity: Checksum Approach
Use checksums to detect changes during sync (more robust than updated_at across providers):
# On product schema
field :provider_checksum, :string
# Computing checksum
def compute_checksum(provider_data) do
provider_data
|> normalize_for_checksum()
|> Jason.encode!(pretty: false)
|> then(&:crypto.hash(:sha256, &1))
|> Base.encode16(case: :lower)
end
defp normalize_for_checksum(data) do
data
|> Map.drop(["created_at", "updated_at"]) # Ignore volatile fields
|> sort_keys_recursively()
end
# During sync
def needs_update?(existing_product, provider_data) do
new_checksum = compute_checksum(provider_data)
existing_product.provider_checksum != new_checksum
end
Benefits:
- Works regardless of provider timestamp support
- Catches any field change
- Avoids unnecessary database writes
- Stable across JSON key ordering variations
Edge Cases (Follow-up Implementation Notes)
| Edge Case | Expected Behavior |
|---|---|
| Provider API down during sync | Oban retries with exponential backoff (30s → 5min). After max_attempts (3), mark sync as failed. Create admin notification. |
| Product deleted on provider | Compare local products with provider list during sync. Mark missing products as status: "archived" (don't hard delete - preserves order history). |
| Variant prices change mid-cart | Cart stores cached_price at add-time. On checkout, re-validate current prices. Show warning to customer if price increased, allow proceed if decreased. |
provider_variant_id format changes |
Already stored as string. Provider behaviour's normalize_variant/2 handles format. If API version changes, update provider module. |
| Provider returns malformed data | Validate in normalize_product/1. Log error, skip malformed product, continue batch. Don't fail entire sync. |
| Rate limiting from provider | Single sync worker (queue concurrency: 1). Add configurable delay between paginated requests. Respect Retry-After headers. |
| Sync interrupted mid-batch | Track sync progress in provider_connection.sync_metadata. On resume, continue from last successful page. |
| Duplicate slugs across providers | Generate slug with provider suffix on collision: awesome-shirt-abc123 where abc123 is short provider product ID. |
| Webhook signature validation | Each provider has different signing. Validate HMAC signature before processing. Reject invalid webhooks with 401. |
| Order submission fails | Keep order as status: "pending". Retry via Oban worker. After max retries, mark as status: "failed", notify admin. |
Testing Strategy
Test Setup
Mocking External APIs (Mox pattern):
# test/support/mocks.ex
Mox.defmock(SimpleshopTheme.Clients.MockPrintify, for: SimpleshopTheme.Clients.PrintifyBehaviour)
# config/test.exs
config :simpleshop_theme, :printify_client, SimpleshopTheme.Clients.MockPrintify
Oban testing:
use Oban.Testing, repo: SimpleshopTheme.Repo
# Jobs run synchronously in tests via perform_job/2
Unit Tests
| Module | Test Coverage |
|---|---|
| Products.Product | Changeset validation, slug generation, status transitions |
| Products.ProductVariant | Options validation, price/cost integrity |
| Products.ProviderConnection | Credential encryption/decryption, config validation |
| Providers.Printify | normalize_product/1, normalize_variant/2, build_order_item/2 |
| Orders.Order | Status aggregation from fulfillments, total calculation |
| Orders.OrderFulfillment | Status transitions, tracking updates |
| AdminNotifications | Create, list_unread, mark_read |
Integration Tests
| Flow | Test Scenarios |
|---|---|
| Product Sync | Successful sync, partial failures, pagination, checksum skip, archived products |
| Order Creation | Single provider, multi-provider split into fulfillments |
| Order Submission | Success, API failure with retry, partial fulfillment submission |
| Webhook Handling | Valid signature, invalid signature rejection, status updates |
| Variant Selector | Option extraction, availability filtering, variant matching |
Example Test Cases
# test/simpleshop_theme/products_test.exs
describe "sync_products/1" do
test "syncs products from provider" do
conn = provider_connection_fixture()
expect(MockPrintify, :list_products, fn _shop_id ->
{:ok, [printify_product_fixture()]}
end)
assert {:ok, %{synced: 1, failed: 0}} = Products.sync_products(conn)
assert [product] = Products.list_products()
assert product.title == "Test Product"
end
test "continues on single product failure" do
expect(MockPrintify, :list_products, fn _ ->
{:ok, [valid_product(), malformed_product()]}
end)
assert {:ok, %{synced: 1, failed: 1}} = Products.sync_products(conn)
end
test "skips unchanged products via checksum" do
existing = product_fixture(provider_checksum: "abc123")
expect(MockPrintify, :list_products, fn _ ->
{:ok, [product_with_same_checksum()]}
end)
assert {:ok, %{synced: 0, skipped: 1}} = Products.sync_products(conn)
end
end
# test/simpleshop_theme/orders_test.exs
describe "create_order_from_cart/1" do
test "splits cart into fulfillments by provider" do
printify_variant = variant_fixture(provider: :printify)
gelato_variant = variant_fixture(provider: :gelato)
cart = [
%{variant_id: printify_variant.id, quantity: 1},
%{variant_id: gelato_variant.id, quantity: 2}
]
assert {:ok, order} = Orders.create_order_from_cart(cart, customer_params())
assert length(order.order_fulfillments) == 2
end
end
# test/simpleshop_theme/sync/product_sync_worker_test.exs
describe "perform/1" do
test "retries on API failure" do
expect(MockPrintify, :list_products, fn _ -> {:error, :timeout} end)
assert {:error, :timeout} = perform_job(ProductSyncWorker, %{conn_id: conn.id})
# Oban will retry automatically
end
test "creates admin notification on final failure" do
# After max_attempts...
assert [notification] = AdminNotifications.list_unread()
assert notification.type == "sync_error"
end
end
Test Fixtures
# test/support/fixtures/products_fixtures.ex
def provider_connection_fixture(attrs \\ %{}) do
{:ok, conn} = Products.create_provider_connection(%{
provider_type: "printify",
name: "Test Connection",
api_key: "test_key",
config: %{"shop_id" => "123"}
} |> Map.merge(attrs))
conn
end
def product_fixture(attrs \\ %{}) do
conn = attrs[:provider_connection] || provider_connection_fixture()
{:ok, product} = Products.create_product(%{
provider_connection_id: conn.id,
provider_product_id: "ext_#{System.unique_integer()}",
title: "Test Product",
slug: "test-product-#{System.unique_integer()}",
status: "active"
} |> Map.merge(attrs))
product
end
def printify_product_fixture do
%{
"id" => "12345",
"title" => "Test Product",
"description" => "A test product",
"options" => [%{"name" => "Size", "values" => [%{"id" => 1, "title" => "M"}]}],
"variants" => [%{"id" => 100, "options" => [1], "price" => 2500}],
"images" => [%{"src" => "https://example.com/img.jpg", "position" => 0}]
}
end
Coverage Goals
| Area | Target |
|---|---|
| Schema modules | 100% (validation, associations) |
| Context functions | 90%+ (all public functions) |
| Provider implementations | 90%+ (normalize, build functions) |
| Oban workers | 80%+ (happy path, error handling) |
| LiveView interactions | 70%+ (critical user flows) |
Verification (Manual)
- Create Printify connection with API key
Products.test_provider_connection(conn)returns shop info
-
Product Sync
- Run
ProductSyncWorker.enqueue(conn.id) - Products appear in database with images and variants
- Re-sync updates existing, doesn't duplicate
- Run
-
Order Submission
- Create order from cart
Orders.submit_to_provider(order)returns provider_order_id- Order status updates from provider
-
Shop Pages
/collections/allshows real products/products/:slugshows product details- Add to cart works with real variant IDs