# Plan: Products Context with Provider Integration > **Status:** Complete (c5c06d9, 037cd16, 57c3ba0) - See [PROGRESS.md](../../PROGRESS.md) for current status. ## Goal Build a Products context that syncs products from external POD providers (Printify first), stores them locally, and enables order submission for fulfillment. ## Core Flow 1. **Connect** - Authenticate with provider (API key or OAuth) 2. **Sync** - Fetch products from provider, upsert locally 3. **Order** - Submit orders to provider for fulfillment --- ## Current Domain Analysis Berrypod has **6 well-defined domains** with clear boundaries: | Domain | Purpose | Schemas | Public Functions | |--------|---------|---------|------------------| | **Accounts** | User authentication & sessions | User, UserToken | 15 | | **Settings** | Theme configuration persistence | Setting, ThemeSettings | 6 | | **Theme** | CSS generation, caching, presets | (structs only) | N/A | | **Media** | Image upload/storage | Image | 8 | | **Images** | Image optimization pipeline | (Oban workers) | N/A | | **Printify** | External API integration | (no schemas) | API client | The new **Products** context will be a new top-level domain that: - Owns product/variant schemas - Coordinates with Printify (and future providers) for sync - Provides data to shop LiveViews (replacing PreviewData) --- ## Architecture Diagram ``` ┌─────────────────────────────────────────────────────────────────────────────┐ │ Berrypod │ ├─────────────────────────────────────────────────────────────────────────────┤ │ │ │ ┌──────────────────────────────────────────────────────────────────────┐ │ │ │ WEB LAYER │ │ │ │ BerrypodWeb │ │ │ │ ┌────────────────┐ ┌────────────────┐ ┌─────────────────────────┐ │ │ │ │ │ Shop LiveViews │ │ Admin LiveViews│ │ Theme Editor LiveView │ │ │ │ │ │ - ProductShow │ │ - UserLogin │ │ - ThemeLive.Index │ │ │ │ │ │ - Collection │ │ - UserSettings │ │ - Preview iframe │ │ │ │ │ │ - Cart │ │ │ │ │ │ │ │ │ └───────┬─────────┘ └────────────────┘ └────────┬───────────────┘ │ │ │ └──────────┼────────────────────────────────────────┼──────────────────┘ │ │ │ │ │ │ ▼ ▼ │ │ ┌──────────────────────────────────────────────────────────────────────┐ │ │ │ DOMAIN LAYER │ │ │ │ │ │ │ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │ │ │ │ Accounts │ │ Settings │ │ Theme │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ • User │ │ • Setting │ │ • CSSCache │ │ │ │ │ │ • UserToken │ │ • Theme │ │ • CSSGen │ │ │ │ │ │ │ │ Settings │ │ • Presets │ │ │ │ │ │ │ │ │ │ • Fonts │ │ │ │ │ │ │ │ │ │ • Preview │ │ │ │ │ └─────────────┘ └──────┬──────┘ │ Data (*) │ │ │ │ │ │ └─────────────┘ │ │ │ │ │ │ │ │ │ ┌─────────────┐ ┌──────▼──────┐ ┌─────────────┐ │ │ │ │ │ Media │ │ Images │ │ Printify │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ • Image │◄──│ • Optimizer │ │ • Client │ │ │ │ │ │ (upload) │ │ • Worker │ │ • Catalog │ (NEW) │ │ │ │ │ │ │ • Cache │ │ • Mockup │ │ │ │ │ └─────────────┘ └─────────────┘ └──────┬──────┘ │ │ │ │ │ │ │ │ │ ┌───────────────────────────────────────────▼───────────────────┐ │ │ │ │ │ Products (NEW) │ │ │ │ │ │ │ │ │ │ │ │ ┌─────────────────────┐ ┌────────────────────────────┐ │ │ │ │ │ │ │ Provider Layer │ │ Product Layer │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ • ProviderConnection│───▶│ • Product │ │ │ │ │ │ │ │ • Provider behaviour│ │ • ProductImage │ │ │ │ │ │ │ │ • PrintifyProvider │ │ • ProductVariant │ │ │ │ │ │ │ │ • (future providers)│ │ │ │ │ │ │ │ │ └─────────────────────┘ └────────────────────────────┘ │ │ │ │ │ │ │ │ │ │ │ │ ┌─────────────────────────────────────────────────────────┐ │ │ │ │ │ │ │ Sync Layer │ │ │ │ │ │ │ │ • ProductSyncWorker (Oban) │ │ │ │ │ │ │ │ • Upsert logic │ │ │ │ │ │ │ └─────────────────────────────────────────────────────────┘ │ │ │ │ │ └────────────────────────────────────────────────────────────────┘ │ │ │ │ │ │ │ │ ┌────────────────────────────────────────────────────────────────┐ │ │ │ │ │ Orders (NEW) │ │ │ │ │ │ │ │ │ │ │ │ • Order • submit_to_provider/1 │ │ │ │ │ │ • OrderLineItem • track_status/1 │ │ │ │ │ └────────────────────────────────────────────────────────────────┘ │ │ │ │ │ │ │ └───────────────────────────────────────────────────────────────────────┘ │ │ │ │ ┌───────────────────────────────────────────────────────────────────────┐ │ │ │ INFRASTRUCTURE │ │ │ │ Repo (SQLite) • Vault (encryption) • Oban (background jobs) │ │ │ └───────────────────────────────────────────────────────────────────────┘ │ │ │ └─────────────────────────────────────────────────────────────────────────────┘ (*) PreviewData will be replaced by Products context queries ``` --- ## Database ERD ``` ┌─────────────────────────────────────────────────────────────────────────────┐ │ CURRENT SCHEMA │ ├─────────────────────────────────────────────────────────────────────────────┤ │ │ │ ┌──────────────────┐ ┌──────────────────┐ │ │ │ users │ │ users_tokens │ │ │ ├──────────────────┤ ├──────────────────┤ │ │ │ id (binary_id) │◄────────│ user_id (FK) │ │ │ │ email │ │ token │ │ │ │ hashed_password │ │ context │ │ │ │ confirmed_at │ │ sent_to │ │ │ │ inserted_at │ │ inserted_at │ │ │ │ updated_at │ └──────────────────┘ │ │ └──────────────────┘ │ │ │ │ ┌──────────────────┐ ┌──────────────────┐ │ │ │ settings │ │ images │ │ │ ├──────────────────┤ ├──────────────────┤ │ │ │ id (binary_id) │ │ id (binary_id) │ │ │ │ key │ │ filename │ │ │ │ value (JSON) │ │ content_type │ │ │ │ inserted_at │ │ byte_size │ │ │ │ updated_at │ │ width │ │ │ └──────────────────┘ │ height │ │ │ │ storage_key │ │ │ │ inserted_at │ │ │ │ updated_at │ │ │ └──────────────────┘ │ ├─────────────────────────────────────────────────────────────────────────────┤ │ NEW SCHEMA (Products) │ ├─────────────────────────────────────────────────────────────────────────────┤ │ │ │ ┌────────────────────────┐ │ │ │ provider_connections │ │ │ ├────────────────────────┤ │ │ │ id (binary_id) │ │ │ │ provider_type │──┐ "printify" | "prodigi" | "gelato" │ │ │ name │ │ │ │ │ enabled │ │ │ │ │ api_key_encrypted │ │ (Vault encrypted) │ │ │ config (JSON) │ │ {"shop_id": "123"} │ │ │ last_synced_at │ │ │ │ │ sync_status │ │ │ │ │ inserted_at │ │ │ │ │ updated_at │ │ │ │ └───────────┬────────────┘ │ │ │ │ │ │ │ │ 1 │ │ │ ▼ * │ │ │ ┌────────────────────────┐ │ │ │ │ products │ │ │ │ ├────────────────────────┤ │ │ │ │ id (binary_id) │ │ │ │ │ provider_connection_id │◄─┘ │ │ │ provider_product_id │ External ID (unique per connection) │ │ │ title │ │ │ │ description │ │ │ │ slug │ URL-friendly (unique globally) │ │ │ status │ "active" | "draft" | "archived" │ │ │ visible │ │ │ │ category │ │ │ │ option_types (JSON) │ ["size", "color"] - display order │ │ │ provider_data (JSON) │ Provider-specific fields (blueprint_id etc) │ │ │ inserted_at │ │ │ │ updated_at │ │ │ └───────────┬────────────┘ │ │ │ │ │ │ 1 │ │ ┌─────────┴─────────┐ │ │ ▼ * ▼ * │ │ ┌──────────────────┐ ┌──────────────────────┐ │ │ │ product_images │ │ product_variants │ │ │ ├──────────────────┤ ├──────────────────────┤ │ │ │ id (binary_id) │ │ id (binary_id) │ │ │ │ product_id (FK) │ │ product_id (FK) │ │ │ │ src │ │ provider_variant_id │ │ │ │ position │ │ title │ "Large / Black" │ │ │ alt │ │ sku │ │ │ │ inserted_at │ │ price │ Selling price (pence) │ │ │ updated_at │ │ compare_at_price │ For sale display │ │ └──────────────────┘ │ cost │ Provider cost (profit) │ │ │ options (JSON) │ {"size": "L", "color": ""} │ │ │ is_enabled │ │ │ │ is_available │ │ │ │ inserted_at │ │ │ │ updated_at │ │ │ └──────────────────────┘ │ │ │ ├─────────────────────────────────────────────────────────────────────────────┤ │ NEW SCHEMA (Orders) │ │ │ │ Customer sees ONE order. Fulfillments are internal (one per provider). │ ├─────────────────────────────────────────────────────────────────────────────┤ │ │ │ ┌────────────────────────┐ │ │ │ orders │ Customer-facing order │ │ ├────────────────────────┤ │ │ │ id (binary_id) │ │ │ │ order_number │ "SS-250125-A1B2" (customer sees this) │ │ │ status │ pending → processing → shipped → delivered │ │ │ payment_status │ pending | paid | refunded │ │ │ customer_email │ │ │ │ shipping_address (JSON)│ │ │ │ subtotal │ (Money - ex_money) │ │ │ shipping_cost │ (Money - ex_money) │ │ │ total │ (Money - ex_money) │ │ │ total_cost │ Provider costs (for profit) │ │ │ inserted_at │ │ │ │ updated_at │ │ │ └───────────┬────────────┘ │ │ │ │ │ │ 1 │ │ ┌─────────┴─────────────────────────┐ │ │ ▼ * ▼ * │ │ ┌────────────────────────┐ ┌─────────────────────────┐ │ │ │ order_fulfillments │ │ order_line_items │ │ │ ├────────────────────────┤ ├─────────────────────────┤ │ │ │ id (binary_id) │ │ id (binary_id) │ │ │ │ order_id (FK) │ │ order_id (FK) │ │ │ │ provider_connection_id │ │ order_fulfillment_id(FK)│ Which provider │ │ │ provider_order_id │ │ product_variant_id (FK) │ │ │ │ status │ │ quantity │ │ │ │ tracking_number │ │ unit_price │ │ │ │ tracking_url │ │ unit_cost │ │ │ │ shipped_at │ │ inserted_at │ │ │ │ inserted_at │ │ updated_at │ │ │ │ updated_at │ └─────────────────────────┘ │ │ └────────────────────────┘ │ │ │ │ ┌────────────────────────┐ │ │ │ order_events │ Timeline visible to customer │ │ ├────────────────────────┤ │ │ │ id (binary_id) │ │ │ │ order_id (FK) │ │ │ │ type │ "placed" | "paid" | "shipped" | "delivered" │ │ │ message │ "Your T-Shirt has shipped via Royal Mail" │ │ │ metadata (JSON) │ {tracking_number, carrier, etc} │ │ │ inserted_at │ │ │ └────────────────────────┘ │ │ │ └─────────────────────────────────────────────────────────────────────────────┘ ``` --- ## Printify Code Reuse ### Existing Client (Highly Reusable) The existing `Printify.Client` is directly reusable: ```elixir # HTTP infrastructure (no changes needed) Client.get/2, Client.post/3, Client.delete/2 # Shop management Client.get_shops/0, Client.get_shop_id/0 # Product operations (already implemented) Client.get_product/2, Client.create_product/2, Client.delete_product/2 ``` **To add:** ```elixir Client.list_products/1 # GET /shops/:shop_id/products.json Client.create_order/2 # POST /shops/:shop_id/orders.json Client.get_order/2 # GET /shops/:shop_id/orders/:id.json ``` ### MockupGenerator Refactoring Extract reusable helpers into new `Printify.Catalog` module: | Function | Current | Target | |----------|---------|--------| | `find_blueprint/1` | MockupGenerator | `Printify.Catalog` | | `find_print_provider/2` | MockupGenerator | `Printify.Catalog` | | `get_variant_info/1` | MockupGenerator | `Printify.Catalog` | | `calculate_cover_scale/2` | MockupGenerator | Keep (mockup-specific) | | `product_definitions/0` | MockupGenerator | Keep (sample data) | **Module structure:** ``` lib/berrypod/clients/ ├── printify.ex # Printify HTTP client (moved from printify/client.ex) ├── gelato.ex # Gelato HTTP client └── prodigi.ex # Prodigi HTTP client lib/berrypod/providers/ ├── provider.ex # Behaviour definition ├── printify.ex # Printify implementation (uses Clients.Printify) ├── gelato.ex # Gelato implementation (uses Clients.Gelato) └── prodigi.ex # Prodigi implementation (uses Clients.Prodigi) lib/berrypod/mockups/ └── generator.ex # Mockup generation (currently uses Clients.Printify) # Provider-agnostic location for future flexibility lib/berrypod/printify/ └── catalog.ex # Blueprint/variant discovery helpers (Printify-specific) ``` Each provider module uses its corresponding client. The mockup generator is in a provider-agnostic location. --- ## Database Schema ### Tables Overview | Table | Purpose | |-------|---------| | `provider_connections` | Provider credentials + config (one per type) | | `products` | Core product data synced from provider | | `product_images` | Product images (multiple per product) | | `product_variants` | Size/color variants with pricing | | `orders` | Customer orders | | `order_line_items` | Items within orders | ### Key Design Decisions 1. **Provider IDs on product directly** (not a separate linking table) - `products.provider_product_id` stores external ID - `product_variants.provider_variant_id` stores external variant ID - Unique constraint: `[:provider_connection_id, :provider_product_id]` 2. **Credentials encrypted in database** - Use `Berrypod.Vault` for at-rest encryption - `api_key_encrypted`, `oauth_access_token_encrypted` fields 3. **Cost tracking for profit calculation** - `product_variants.cost` - cost from provider - `order_line_items.unit_cost` - snapshot at order time - `orders.total_cost` - sum for profit reporting 4. **Provider type as enum** (not separate table) - Hardcoded: `~w(printify prodigi gelato)` - Keeps type safety for provider-specific logic --- ## Schema Details ### provider_connections ```elixir field :provider_type, :string # "printify", "prodigi", "gelato" field :name, :string # "My Printify Account" field :enabled, :boolean field :api_key_encrypted, :binary # Encrypted API key field :config, :map # {"shop_id": "12345"} field :last_synced_at, :utc_datetime field :sync_status, :string # pending, syncing, completed, failed ``` ### products ```elixir belongs_to :provider_connection field :provider_product_id, :string # External ID from provider field :title, :string field :description, :text field :slug, :string # URL-friendly, unique field :status, :string # active, draft, archived field :visible, :boolean field :category, :string field :option_types, {:array, :string} # ["Colors", "Sizes"] - option names in display order # Extracted from provider's product.options[].name on sync # Used to order the variant selector UI field :provider_data, :map # Provider-specific fields stored here # Printify: %{"blueprint_id" => 145, "print_provider_id" => 29, ...} # Keeps products table generic across providers ``` ### product_variants ```elixir belongs_to :product field :provider_variant_id, :string # CRITICAL: The ID we send back to provider when ordering # Printify: variant_id (e.g., "12345") # Printful: sync_variant_id (e.g., "4752058849") # Gelato: productUid (e.g., "apparel_product_gca_t-shirt_gsi_m_gco_white") # Prodigi: sku (e.g., "GLOBAL-CAN-10x10") field :title, :string # "Large / Black" field :sku, :string # Shop's internal SKU (optional) field :price, :integer # Selling price (pence) field :compare_at_price, :integer # Original price (for sales) field :cost, :integer # Provider cost (for profit) field :options, :map # {"size": "Large", "color": "Black"} # JSON is appropriate here because: # - Options always loaded with variant (no join needed) # - No need to query "find all Large variants" # - Matches provider data format (easy sync) # - Simpler than a separate options table field :is_enabled, :boolean field :is_available, :boolean # In stock at provider ``` ### orders (customer-facing) ```elixir # Customer sees ONE order, even if fulfilled by multiple providers field :order_number, :string # "SS-250125-A1B2" (customer-visible) field :status, :string # Aggregate: pending → processing → shipped → delivered field :payment_status, :string # pending, paid, refunded field :customer_email, :string field :shipping_address, :map field :subtotal, Money.Ecto.Map.Type # Using ex_money for currency safety field :shipping_cost, Money.Ecto.Map.Type field :total, Money.Ecto.Map.Type field :total_cost, Money.Ecto.Map.Type # Sum of provider costs has_many :order_fulfillments # One per provider has_many :order_line_items has_many :order_events # Timeline for customer ``` ### order_fulfillments (one per provider) ```elixir belongs_to :order belongs_to :provider_connection field :provider_order_id, :string # External ID after submission field :status, :string # pending → submitted → processing → shipped field :tracking_number, :string field :tracking_url, :string field :shipped_at, :utc_datetime has_many :order_line_items # Items fulfilled by this provider ``` ### order_line_items ```elixir belongs_to :order belongs_to :order_fulfillment # Links to which provider ships this item belongs_to :product_variant field :quantity, :integer field :unit_price, Money.Ecto.Map.Type field :unit_cost, Money.Ecto.Map.Type # Per-item tracking (provider may ship items separately) field :status, :string # pending → shipped → delivered field :tracking_number, :string field :tracking_url, :string field :shipped_at, :utc_datetime ``` ### order_events (customer timeline) ```elixir belongs_to :order field :type, :string # Event types (see below) field :message, :string # "Your T-Shirt and Mug have shipped!" field :metadata, :map # {tracking_number, carrier, items, etc} # Event types: # - "placed" → Order placed # - "paid" → Payment confirmed # - "fulfillment_shipped" → Some items shipped (granular per-fulfillment) # - "shipped" → ALL items now shipped # - "delivered" → ALL items delivered # Example metadata for fulfillment_shipped: # %{fulfillment_id: "uuid", items: ["T-Shirt", "Mug"], # carrier: "Royal Mail", tracking_number: "RM123", tracking_url: "..."} ``` --- ## Variant Selector UX Pattern Products with multiple option types (e.g., size × color × sleeves) can have many variants: - 4 sizes × 4 colors × 2 sleeves = **32 variants** ### Data Model Each variant stores its specific combination as JSON with **human-readable labels**: ```elixir %{id: "v1", options: %{"size" => "M", "color" => "Black", "sleeves" => "Short"}, price: 2500} %{id: "v2", options: %{"size" => "M", "color" => "Black", "sleeves" => "Long"}, price: 2700} # ... etc ``` ### Provider Option Structure (Important) Providers like Printify use a **two-tier option system**: **Product level** - defines options with IDs and labels: ```json { "options": [ { "name": "Colors", "type": "color", "values": [ { "id": 751, "title": "Solid White" }, { "id": 752, "title": "Black" } ]}, { "name": "Sizes", "type": "size", "values": [ { "id": 2, "title": "S" }, { "id": 3, "title": "M" } ]} ] } ``` **Variant level** - only stores option IDs: ```json { "id": 12345, "options": [751, 2], "title": "Solid White / S" } ``` ### Sync Strategy: Denormalize to Labels During sync, we **resolve IDs to labels** and store the human-readable version: ```elixir def normalize_variant(provider_variant, product_options) do # Build ID -> label lookup from product options option_lookup = build_option_lookup(product_options) # => %{751 => {"Colors", "Solid White"}, 2 => {"Sizes", "S"}} # Convert variant's [751, 2] to %{"Colors" => "Solid White", "Sizes" => "S"} options = provider_variant["options"] |> Enum.map(&Map.get(option_lookup, &1)) |> Enum.reject(&is_nil/1) |> Map.new() %{ provider_variant_id: to_string(provider_variant["id"]), title: provider_variant["title"], options: options, # ... other fields } end ``` **Why denormalize?** - No need to join to an options table on every product page load - Labels rarely change (and if they do, re-sync updates them) - Simpler queries, faster rendering - Keeps our schema provider-agnostic (we just store labels, not provider-specific IDs) ### Provider Compatibility All major POD providers can be normalized to our `options: %{"Size" => "M", "Color" => "Black"}` schema: | Provider | Raw Variant Structure | Normalization | |----------|----------------------|---------------| | **Printify** | `options: [751, 2]` (IDs only, labels at product level) | Resolve IDs → labels during sync | | **Printful** | `color: "Black", size: "M"` (flat attributes) | Map attribute keys to standard names | | **Gelato** | `variantOptions: [{name: "Size", value: "M"}]` (array) | Convert array to map | | **Prodigi** | `attributes: {"wrap": "Black"}` (object) | Direct mapping | Each provider's `normalize_variant/2` function handles the transformation: ```elixir # Printify - resolve IDs to labels %{"options" => [751, 2]} → %{"Colors" => "Solid White", "Sizes" => "S"} # Printful - rename keys %{"color" => "Black", "size" => "M"} → %{"Color" => "Black", "Size" => "M"} # Gelato - array to map [%{"name" => "Size", "value" => "M"}] → %{"Size" => "M"} # Prodigi - direct (already a map) %{"wrap" => "Black"} → %{"Wrap" => "Black"} ``` Sources: - [Printful API Documentation](https://developers.printful.com/docs/) - [Gelato API - Get Template](https://dashboard.gelato.com/docs/ecommerce/templates/get/) - [Prodigi API Documentation](https://www.prodigi.com/print-api/docs/reference/) ### Why JSON Options Work Well | Use Case | Query Pattern | Efficient? | |----------|---------------|------------| | Product page - list variants | `where(product_id: ^id)` | Yes - no JSON query | | Cart - get variant by ID | `Repo.get(Variant, id)` | Yes - no JSON query | | Display options on page | Access `variant.options` | Yes - in-memory map | | Find matching variant | `Enum.find` on loaded list | Yes - all variants loaded anyway | A separate `variant_options` table would add joins and complexity without benefit. ### LiveView Implementation **On mount**, load product with all variants and extract option types: ```elixir def mount(_params, _session, socket) do product = Products.get_product_with_variants(slug) # Extract unique option names and values from variants option_types = extract_option_types(product.variants) # => %{ # "size" => ["S", "M", "L", "XL"], # "color" => ["Black", "Green", "Blue", "Grey"], # "sleeves" => ["Short", "Long"] # } {:ok, assign(socket, product: product, option_types: option_types, selected: %{})} end defp extract_option_types(variants) do variants |> Enum.flat_map(fn v -> Map.to_list(v.options) end) |> Enum.group_by(&elem(&1, 0), &elem(&1, 1)) |> Map.new(fn {k, v} -> {k, Enum.uniq(v)} end) end ``` ### Cascading Selector UI Standard e-commerce pattern with availability filtering: ``` ┌─────────────────────────────────────┐ │ Size: [S] [M] [L] [XL] │ ← User selects "M" ├─────────────────────────────────────┤ │ Color: [Black] [Green] [Blue] │ ← Grey disabled (no M+Grey combo) ├─────────────────────────────────────┤ │ Sleeves: [Short] [Long] │ ← User selects "Long" ├─────────────────────────────────────┤ │ Price: £27.00 │ │ [Add to Cart] │ ← Adds variant "M/Black/Long" └─────────────────────────────────────┘ ``` **Key behaviors:** 1. User selects options one at a time 2. Unavailable combinations are disabled/greyed out 3. Once all options selected, show matching variant's price 4. Add to cart uses the specific variant ID ### Selection Handler ```elixir def handle_event("select_option", %{"option" => option, "value" => value}, socket) do selected = Map.put(socket.assigns.selected, option, value) # Find matching variant (if all options selected) current_variant = find_matching_variant(socket.assigns.product.variants, selected) # Determine which options are still available given current selection available_options = compute_available_options(socket.assigns.product.variants, selected) {:noreply, assign(socket, selected: selected, current_variant: current_variant, available_options: available_options )} end defp find_matching_variant(variants, selected) do Enum.find(variants, fn v -> Map.equal?(v.options, selected) end) end defp compute_available_options(variants, selected) do # For each unselected option type, find values that have # at least one variant matching current selection # (filters out impossible combinations) end ``` ### Optional: Store Option Order on Product For consistent selector ordering, optionally store on product: ```elixir field :option_types, {:array, :string} # ["size", "color", "sleeves"] ``` This defines display order. Can be extracted from provider API on first sync. --- ## Provider Behaviour ```elixir @callback provider_type() :: String.t() @callback test_connection(conn) :: {:ok, map()} | {:error, term()} @callback fetch_products(conn) :: {:ok, [map()]} | {:error, term()} @callback normalize_product(raw_product) :: map() # Transform to our schema @callback normalize_variant(raw_variant, product) :: map() # Transform + resolve options @callback build_order_item(variant, quantity) :: map() # Build provider-specific order item @callback submit_order(conn, order) :: {:ok, %{provider_order_id: String.t()}} | {:error, term()} @callback get_order_status(conn, provider_order_id) :: {:ok, map()} | {:error, term()} ``` Each provider (Printify, Prodigi, Gelato) implements this behaviour with provider-specific API calls and data normalization. --- ## Order Submission Flow When submitting an order, we must send **provider-specific variant identifiers** back to each provider. ### What Each Provider Needs | Provider | Order Item Fields Required | Source in Our Schema | |----------|---------------------------|---------------------| | **Printify** | `product_id`, `variant_id`, `print_provider_id` | `product.provider_product_id`, `variant.provider_variant_id`, `product.provider_data["print_provider_id"]` | | **Printful** | `sync_variant_id` | `variant.provider_variant_id` | | **Gelato** | `productUid`, `itemReferenceId` | `variant.provider_variant_id`, our `order_line_item.id` | | **Prodigi** | `sku`, `attributes` | `variant.provider_variant_id`, `variant.options` | ### Provider `build_order_item/2` Examples ```elixir # Printify def build_order_item(variant, quantity) do product = variant.product %{ "product_id" => product.provider_product_id, "variant_id" => String.to_integer(variant.provider_variant_id), "print_provider_id" => product.provider_data["print_provider_id"], "quantity" => quantity } end # Printful def build_order_item(variant, quantity) do %{ "sync_variant_id" => String.to_integer(variant.provider_variant_id), "quantity" => quantity } end # Gelato def build_order_item(variant, quantity) do %{ "productUid" => variant.provider_variant_id, "quantity" => quantity, "itemReferenceId" => Ecto.UUID.generate() } end # Prodigi def build_order_item(variant, quantity) do %{ "sku" => variant.provider_variant_id, "copies" => quantity, "attributes" => variant.options # {"wrap" => "Black"} etc. } end ``` ### Order Submission Implementation ```elixir def submit_to_provider(%Order{} = order) do order = Repo.preload(order, [:provider_connection, line_items: [product_variant: :product]]) provider = Provider.for_type(order.provider_connection.provider_type) # Build provider-specific line items items = Enum.map(order.line_items, fn item -> provider.build_order_item(item.product_variant, item.quantity) end) # Submit via provider behaviour case provider.submit_order(order.provider_connection, %{ line_items: items, shipping_address: order.shipping_address, external_id: order.order_number }) do {:ok, %{provider_order_id: provider_id}} -> order |> Order.changeset(%{provider_order_id: provider_id, status: "submitted"}) |> Repo.update() {:error, reason} -> {:error, reason} end end ``` ### Key Insight The `provider_variant_id` field serves a **dual purpose**: 1. **Sync**: We receive it from the provider and store it 2. **Order**: We send it back to the provider when fulfilling This is why it's stored as a string - different providers use different formats (integers, UUIDs, compound UIDs), but we treat them all as opaque identifiers. Sources: - [Printify API Reference](https://developers.printify.com/) - [Printful API Documentation](https://developers.printful.com/docs/) - [Gelato API - Create Order](https://dashboard.gelato.com/docs/orders/v4/create/) - [Prodigi API Documentation](https://www.prodigi.com/print-api/docs/reference/) --- ## Sync Strategy 1. Fetch all products from provider via API 2. For each product: - Find existing by `[provider_connection_id, provider_product_id]` - Upsert product with changeset - Delete removed images, insert new ones - Upsert variants (delete removed, update existing, insert new) 3. Update `provider_connection.last_synced_at` Sync runs via Oban worker for async processing with retry on failure. --- ## Files to Create ### Dependencies Add to `mix.exs`: - `{:ex_money, "~> 5.0"}` - Currency handling - `{:ex_money_sql, "~> 1.0"}` - Ecto types for Money ### Migrations - `*_create_provider_connections.exs` - `*_create_products.exs` - `*_create_product_images.exs` - `*_create_product_variants.exs` - `*_create_orders.exs` - `*_create_order_fulfillments.exs` - `*_create_order_line_items.exs` - `*_create_order_events.exs` - `*_create_admin_notifications.exs` ### Schemas - `lib/berrypod/products/provider_connection.ex` - `lib/berrypod/products/product.ex` - `lib/berrypod/products/product_image.ex` - `lib/berrypod/products/product_variant.ex` - `lib/berrypod/orders/order.ex` - `lib/berrypod/orders/order_fulfillment.ex` - `lib/berrypod/orders/order_line_item.ex` - `lib/berrypod/orders/order_event.ex` - `lib/berrypod/admin_notifications/notification.ex` ### Contexts - `lib/berrypod/products.ex` - Product queries, sync logic - `lib/berrypod/orders.ex` - Order creation, submission - `lib/berrypod/admin_notifications.ex` - Admin notification management ### Providers - `lib/berrypod/providers/provider.ex` - Behaviour definition - `lib/berrypod/providers/printify.ex` - Printify implementation ### Workers - `lib/berrypod/sync/product_sync_worker.ex` - Oban worker ### Webhooks - `lib/berrypod_web/controllers/webhook_controller.ex` - `lib/berrypod/webhooks/printify_handler.ex` ### Notifiers - `lib/berrypod_web/notifiers/customer_notifier.ex` - Customer emails ### Support - `lib/berrypod/vault.ex` - Credential encryption --- ## Files to Modify - `lib/berrypod/printify/client.ex` → Move to `lib/berrypod/clients/printify.ex` - `lib/berrypod/printify/mockup_generator.ex` → Move to `lib/berrypod/mockups/generator.ex` - `lib/berrypod/theme/preview_data.ex` - Query real products when available - `lib/berrypod_web/live/shop_live/*.ex` - Use Products context instead of PreviewData --- ## Implementation Phases ### Phase 1: Schema Foundation 1. Create Vault module for credential encryption 2. Create all migrations 3. Create all schema modules 4. Run migrations ### Phase 2: Products Context 1. Create Products context with CRUD 2. Implement sync/upsert logic 3. Create ProductSyncWorker (Oban) 4. Update Printify.Client with get_products endpoint ### Phase 3: Provider Abstraction 1. Create Provider behaviour 2. Implement Printify provider 3. Connect admin UI for provider setup 4. Test sync flow end-to-end ### Phase 4: Orders 1. Create Orders context 2. Implement order submission to provider 3. Add order status tracking ### Phase 5: Integration 1. Update PreviewData to use real products 2. Update shop LiveViews 3. Update cart to use real variant IDs --- ## Notifications Strategy ### Admin Notifications (Internal) Stored in database, shown in admin dashboard: ```elixir schema "admin_notifications" do field :type, :string # "sync_error", "order_failed", "webhook_error" field :severity, :string # "info", "warning", "error" field :title, :string field :message, :string field :metadata, :map field :read_at, :utc_datetime timestamps() end ``` **Use cases:** Sync failures, order submission errors, provider API issues. ### Customer Notifications Two mechanisms: 1. **Transactional emails** via `CustomerNotifier` (order confirmation, shipping updates) 2. **Order events** table for timeline display on order page ```elixir # order_events - visible to customer on their order page schema "order_events" do belongs_to :order field :type, :string # "placed", "paid", "shipped", "delivered" field :message, :string # Human-readable status field :metadata, :map # {tracking_number, carrier, fulfillment_id} timestamps() end ``` ### Files to Create - `lib/berrypod/admin_notifications.ex` - Admin notification context - `lib/berrypod/admin_notifications/notification.ex` - Schema - `lib/berrypod/orders/order_event.ex` - Customer-facing event schema - `lib/berrypod_web/notifiers/customer_notifier.ex` - Emails --- ## Data Integrity: Checksum Approach Use checksums to detect changes during sync (more robust than `updated_at` across providers): ```elixir # On product schema field :provider_checksum, :string # Computing checksum def compute_checksum(provider_data) do provider_data |> normalize_for_checksum() |> Jason.encode!(pretty: false) |> then(&:crypto.hash(:sha256, &1)) |> Base.encode16(case: :lower) end defp normalize_for_checksum(data) do data |> Map.drop(["created_at", "updated_at"]) # Ignore volatile fields |> sort_keys_recursively() end # During sync def needs_update?(existing_product, provider_data) do new_checksum = compute_checksum(provider_data) existing_product.provider_checksum != new_checksum end ``` **Benefits:** - Works regardless of provider timestamp support - Catches any field change - Avoids unnecessary database writes - Stable across JSON key ordering variations --- ## Edge Cases (Follow-up Implementation Notes) | Edge Case | Expected Behavior | |-----------|-------------------| | **Provider API down during sync** | Oban retries with exponential backoff (30s → 5min). After max_attempts (3), mark sync as `failed`. Create admin notification. | | **Product deleted on provider** | Compare local products with provider list during sync. Mark missing products as `status: "archived"` (don't hard delete - preserves order history). | | **Variant prices change mid-cart** | Cart stores `cached_price` at add-time. On checkout, re-validate current prices. Show warning to customer if price increased, allow proceed if decreased. | | **`provider_variant_id` format changes** | Already stored as string. Provider behaviour's `normalize_variant/2` handles format. If API version changes, update provider module. | | **Provider returns malformed data** | Validate in `normalize_product/1`. Log error, skip malformed product, continue batch. Don't fail entire sync. | | **Rate limiting from provider** | Single sync worker (queue concurrency: 1). Add configurable delay between paginated requests. Respect Retry-After headers. | | **Sync interrupted mid-batch** | Track sync progress in `provider_connection.sync_metadata`. On resume, continue from last successful page. | | **Duplicate slugs across providers** | Generate slug with provider suffix on collision: `awesome-shirt-abc123` where `abc123` is short provider product ID. | | **Webhook signature validation** | Each provider has different signing. Validate HMAC signature before processing. Reject invalid webhooks with 401. | | **Order submission fails** | Keep order as `status: "pending"`. Retry via Oban worker. After max retries, mark as `status: "failed"`, notify admin. | --- ## Testing Strategy ### Test Setup **Mocking External APIs (Mox pattern):** ```elixir # test/support/mocks.ex Mox.defmock(Berrypod.Clients.MockPrintify, for: Berrypod.Clients.PrintifyBehaviour) # config/test.exs config :berrypod, :printify_client, Berrypod.Clients.MockPrintify ``` **Oban testing:** ```elixir use Oban.Testing, repo: Berrypod.Repo # Jobs run synchronously in tests via perform_job/2 ``` ### Unit Tests | Module | Test Coverage | |--------|---------------| | **Products.Product** | Changeset validation, slug generation, status transitions | | **Products.ProductVariant** | Options validation, price/cost integrity | | **Products.ProviderConnection** | Credential encryption/decryption, config validation | | **Providers.Printify** | `normalize_product/1`, `normalize_variant/2`, `build_order_item/2` | | **Orders.Order** | Status aggregation from fulfillments, total calculation | | **Orders.OrderFulfillment** | Status transitions, tracking updates | | **AdminNotifications** | Create, list_unread, mark_read | ### Integration Tests | Flow | Test Scenarios | |------|----------------| | **Product Sync** | Successful sync, partial failures, pagination, checksum skip, archived products | | **Order Creation** | Single provider, multi-provider split into fulfillments | | **Order Submission** | Success, API failure with retry, partial fulfillment submission | | **Webhook Handling** | Valid signature, invalid signature rejection, status updates | | **Variant Selector** | Option extraction, availability filtering, variant matching | ### Example Test Cases ```elixir # test/berrypod/products_test.exs describe "sync_products/1" do test "syncs products from provider" do conn = provider_connection_fixture() expect(MockPrintify, :list_products, fn _shop_id -> {:ok, [printify_product_fixture()]} end) assert {:ok, %{synced: 1, failed: 0}} = Products.sync_products(conn) assert [product] = Products.list_products() assert product.title == "Test Product" end test "continues on single product failure" do expect(MockPrintify, :list_products, fn _ -> {:ok, [valid_product(), malformed_product()]} end) assert {:ok, %{synced: 1, failed: 1}} = Products.sync_products(conn) end test "skips unchanged products via checksum" do existing = product_fixture(provider_checksum: "abc123") expect(MockPrintify, :list_products, fn _ -> {:ok, [product_with_same_checksum()]} end) assert {:ok, %{synced: 0, skipped: 1}} = Products.sync_products(conn) end end # test/berrypod/orders_test.exs describe "create_order_from_cart/1" do test "splits cart into fulfillments by provider" do printify_variant = variant_fixture(provider: :printify) gelato_variant = variant_fixture(provider: :gelato) cart = [ %{variant_id: printify_variant.id, quantity: 1}, %{variant_id: gelato_variant.id, quantity: 2} ] assert {:ok, order} = Orders.create_order_from_cart(cart, customer_params()) assert length(order.order_fulfillments) == 2 end end # test/berrypod/sync/product_sync_worker_test.exs describe "perform/1" do test "retries on API failure" do expect(MockPrintify, :list_products, fn _ -> {:error, :timeout} end) assert {:error, :timeout} = perform_job(ProductSyncWorker, %{conn_id: conn.id}) # Oban will retry automatically end test "creates admin notification on final failure" do # After max_attempts... assert [notification] = AdminNotifications.list_unread() assert notification.type == "sync_error" end end ``` ### Test Fixtures ```elixir # test/support/fixtures/products_fixtures.ex def provider_connection_fixture(attrs \\ %{}) do {:ok, conn} = Products.create_provider_connection(%{ provider_type: "printify", name: "Test Connection", api_key: "test_key", config: %{"shop_id" => "123"} } |> Map.merge(attrs)) conn end def product_fixture(attrs \\ %{}) do conn = attrs[:provider_connection] || provider_connection_fixture() {:ok, product} = Products.create_product(%{ provider_connection_id: conn.id, provider_product_id: "ext_#{System.unique_integer()}", title: "Test Product", slug: "test-product-#{System.unique_integer()}", status: "active" } |> Map.merge(attrs)) product end def printify_product_fixture do %{ "id" => "12345", "title" => "Test Product", "description" => "A test product", "options" => [%{"name" => "Size", "values" => [%{"id" => 1, "title" => "M"}]}], "variants" => [%{"id" => 100, "options" => [1], "price" => 2500}], "images" => [%{"src" => "https://example.com/img.jpg", "position" => 0}] } end ``` ### Coverage Goals | Area | Target | |------|--------| | Schema modules | 100% (validation, associations) | | Context functions | 90%+ (all public functions) | | Provider implementations | 90%+ (normalize, build functions) | | Oban workers | 80%+ (happy path, error handling) | | LiveView interactions | 70%+ (critical user flows) | --- --- ## Task: Admin Provider Setup UI > **Status:** Next up > **Estimate:** ~2 hours > **Prerequisite:** Phase 1 complete (schemas, context, Printify provider) ### Goal Add an admin UI at `/admin/providers` for managing POD provider connections. This enables shop owners to connect their Printify account and trigger product syncs without using IEx. ### User Stories 1. **Connect provider:** Enter Printify API key, test connection, save 2. **View status:** See connected providers, last sync time, product count 3. **Trigger sync:** Manually sync products from provider 4. **Disconnect:** Remove a provider connection ### Routes ```elixir # In router.ex, within :require_authenticated_user live_session live "/admin/providers", ProviderLive.Index, :index live "/admin/providers/new", ProviderLive.Index, :new live "/admin/providers/:id/edit", ProviderLive.Index, :edit ``` ### Files to Create/Modify | File | Purpose | |------|---------| | `lib/berrypod_web/live/provider_live/index.ex` | LiveView for provider list + modal forms | | `lib/berrypod_web/live/provider_live/index.html.heex` | Template | | `lib/berrypod_web/live/provider_live/form_component.ex` | Form component for new/edit | | `lib/berrypod/providers/printify.ex` | Add `register_webhooks/1`, `unregister_webhooks/1` | | `lib/berrypod/workers/product_sync_worker.ex` | Stub for "Sync Now" (full impl in next task) | | `test/berrypod_web/live/provider_live_test.exs` | LiveView tests | ### UI Design Single-page admin with modal for add/edit (follows Phoenix generator pattern): ``` ┌─────────────────────────────────────────────────────────────┐ │ Admin › Provider Connections [+ Add] │ ├─────────────────────────────────────────────────────────────┤ │ │ │ ┌─────────────────────────────────────────────────────┐ │ │ │ 🟢 Printify [Edit] [×] │ │ │ │ "My Printify Shop" │ │ │ │ Shop: Acme Store • 24 products │ │ │ │ Last synced: 5 minutes ago │ │ │ │ [Sync Now] │ │ │ └─────────────────────────────────────────────────────┘ │ │ │ │ ┌─────────────────────────────────────────────────────┐ │ │ │ ⚪ Gelato [Edit] [×] │ │ │ │ "Gelato Account" │ │ │ │ Not connected (invalid API key) │ │ │ │ [Sync Now] │ │ │ └─────────────────────────────────────────────────────┘ │ │ │ └─────────────────────────────────────────────────────────────┘ ``` **Add/Edit Modal:** ``` ┌─────────────────────────────────────────────────────────────┐ │ Connect Printify [×] │ ├─────────────────────────────────────────────────────────────┤ │ │ │ Name │ │ ┌─────────────────────────────────────────────────────┐ │ │ │ My Printify Shop │ │ │ └─────────────────────────────────────────────────────┘ │ │ │ │ API Key │ │ ┌─────────────────────────────────────────────────────┐ │ │ │ •••••••••••••••••••• │ │ │ └─────────────────────────────────────────────────────┘ │ │ Get your API key from Printify Settings → Connections │ │ │ │ ┌──────────────────┐ │ │ │ Test Connection │ ✓ Connected to "Acme Store" │ │ └──────────────────┘ │ │ │ │ □ Enable automatic sync │ │ │ │ [Cancel] [Save Connection] │ └─────────────────────────────────────────────────────────────┘ ``` ### LiveView Implementation **Index LiveView (`provider_live/index.ex`):** ```elixir defmodule BerrypodWeb.ProviderLive.Index do use BerrypodWeb, :live_view alias Berrypod.Products alias Berrypod.Products.ProviderConnection @impl true def mount(_params, _session, socket) do connections = Products.list_provider_connections() {:ok, stream(socket, :connections, connections)} end @impl true def handle_params(params, _url, socket) do {:noreply, apply_action(socket, socket.assigns.live_action, params)} end defp apply_action(socket, :edit, %{"id" => id}) do socket |> assign(:page_title, "Edit Provider") |> assign(:connection, Products.get_provider_connection!(id)) end defp apply_action(socket, :new, _params) do socket |> assign(:page_title, "Add Provider") |> assign(:connection, %ProviderConnection{}) end defp apply_action(socket, :index, _params) do socket |> assign(:page_title, "Provider Connections") |> assign(:connection, nil) end @impl true def handle_info({BerrypodWeb.ProviderLive.FormComponent, {:saved, connection}}, socket) do {:noreply, stream_insert(socket, :connections, connection)} end @impl true def handle_event("delete", %{"id" => id}, socket) do connection = Products.get_provider_connection!(id) {:ok, _} = Products.delete_provider_connection(connection) {:noreply, stream_delete(socket, :connections, connection)} end @impl true def handle_event("sync", %{"id" => id}, socket) do connection = Products.get_provider_connection!(id) # Enqueue sync job (Oban worker) {:ok, _job} = Products.enqueue_sync(connection) {:noreply, socket |> put_flash(:info, "Sync started for #{connection.name}") |> stream_insert(:connections, %{connection | sync_status: "syncing"})} end end ``` **Form Component (`provider_live/form_component.ex`):** Key features: - Provider type dropdown (Printify only initially, extensible) - API key field (password type, shows masked on edit) - "Test Connection" button with async feedback - Validation before save - Auto-registers webhooks on successful save ```elixir def handle_event("test_connection", _params, socket) do form_data = socket.assigns.form.source.changes api_key = form_data[:api_key] || socket.assigns.connection.api_key case test_provider_connection(socket.assigns.provider_type, api_key) do {:ok, %{shop_name: name}} -> {:noreply, assign(socket, test_result: {:ok, name})} {:error, reason} -> {:noreply, assign(socket, test_result: {:error, reason})} end end defp test_provider_connection("printify", api_key) do # Build temporary connection struct for testing conn = %ProviderConnection{provider_type: "printify", api_key: api_key} Berrypod.Providers.Printify.test_connection(conn) end ``` ### Events | Event | Handler | Action | |-------|---------|--------| | `"validate"` | FormComponent | Live validation of form fields | | `"test_connection"` | FormComponent | Call provider's `test_connection/1`, show result | | `"save"` | FormComponent | Create/update connection, register webhooks, notify parent | | `"delete"` | Index | Unregister webhooks, delete connection | | `"sync"` | Index | Enqueue `ProductSyncWorker` job | ### Webhook Registration Flow When a provider connection is saved, automatically register webhooks with the provider so real-time sync works immediately. **On Save (FormComponent):** ```elixir def handle_event("save", %{"provider" => params}, socket) do save_result = case socket.assigns.connection.id do nil -> Products.create_provider_connection(params) _id -> Products.update_provider_connection(socket.assigns.connection, params) end case save_result do {:ok, connection} -> # Register webhooks after successful save case register_webhooks(connection) do {:ok, _} -> Products.update_provider_connection(connection, %{ config: Map.put(connection.config, "webhooks_registered", true) }) {:error, reason} -> # Log but don't fail - webhooks can be registered later Logger.warning("Failed to register webhooks: #{inspect(reason)}") end notify_parent({:saved, connection}) {:noreply, socket |> put_flash(:info, "Provider connected successfully") |> push_patch(to: socket.assigns.patch)} {:error, changeset} -> {:noreply, assign(socket, form: to_form(changeset))} end end defp register_webhooks(%{provider_type: "printify"} = conn) do Berrypod.Providers.Printify.register_webhooks(conn) end ``` **On Delete (Index):** ```elixir def handle_event("delete", %{"id" => id}, socket) do connection = Products.get_provider_connection!(id) # Unregister webhooks before deleting unregister_webhooks(connection) {:ok, _} = Products.delete_provider_connection(connection) {:noreply, stream_delete(socket, :connections, connection)} end defp unregister_webhooks(%{provider_type: "printify"} = conn) do Berrypod.Providers.Printify.unregister_webhooks(conn) end ``` **Provider Webhook Functions (add to `Providers.Printify`):** ```elixir @webhook_topics ~w(product:publish:started product:deleted shop:disconnected) def register_webhooks(conn) do webhook_url = BerrypodWeb.Endpoint.url() <> "/webhooks/printify" shop_id = get_shop_id(conn) results = Enum.map(@webhook_topics, fn topic -> Client.post(conn, "/shops/#{shop_id}/webhooks.json", %{ topic: topic, url: webhook_url }) end) if Enum.all?(results, &match?({:ok, _}, &1)) do {:ok, :registered} else {:error, :partial_registration} end end def unregister_webhooks(conn) do shop_id = get_shop_id(conn) # List existing webhooks and delete ours case Client.get(conn, "/shops/#{shop_id}/webhooks.json") do {:ok, %{"webhooks" => webhooks}} -> our_url = BerrypodWeb.Endpoint.url() <> "/webhooks/printify" webhooks |> Enum.filter(&(&1["url"] == our_url)) |> Enum.each(&Client.delete(conn, "/shops/#{shop_id}/webhooks/#{&1["id"]}.json")) {:ok, :unregistered} error -> error end end ``` ### UI: Webhook Status Indicator Show webhook status on the connection card: ``` ┌─────────────────────────────────────────────────────────┐ │ 🟢 Printify [Edit] [×] │ │ "My Printify Shop" │ │ Shop: Acme Store • 24 products │ │ Last synced: 5 minutes ago │ │ ✓ Real-time updates enabled [Sync Now] │ ← webhook status └─────────────────────────────────────────────────────────┘ ``` Or if webhooks failed to register: ``` │ ⚠ Real-time updates unavailable (click Sync manually) │ ``` Template snippet: ```heex
<%= if @connection.config["webhooks_registered"] do %> <.icon name="hero-check-circle" class="w-4 h-4 text-green-500" /> Real-time updates enabled <% else %> <.icon name="hero-exclamation-triangle" class="w-4 h-4 text-amber-500" /> Real-time updates unavailable <% end %>
``` ### Context Additions Add to `lib/berrypod/products.ex`: ```elixir @doc """ Enqueues a product sync job for the given provider connection. Returns {:ok, job} or {:error, changeset}. """ def enqueue_sync(%ProviderConnection{} = conn) do %{connection_id: conn.id} |> Berrypod.Workers.ProductSyncWorker.new() |> Oban.insert() end @doc """ Gets product count for a provider connection. """ def count_products_for_connection(connection_id) do from(p in Product, where: p.provider_connection_id == ^connection_id, select: count()) |> Repo.one() end ``` ### Acceptance Criteria 1. **Navigation:** `/admin` shows link to "Providers" in admin nav 2. **List view:** Shows all provider connections with status indicators 3. **Add flow:** Modal form with provider type, name, API key fields 4. **Test connection:** Button validates API key, shows shop name or error 5. **Save:** Creates encrypted connection, registers webhooks, closes modal, updates list 6. **Webhook status:** Card shows "Real-time updates enabled" or warning if registration failed 7. **Edit:** Pre-fills form (API key masked), allows updates, re-registers webhooks if key changed 8. **Delete:** Unregisters webhooks, confirmation dialog, removes connection 9. **Sync:** Button enqueues Oban job, shows "syncing" status 10. **Empty state:** Helpful message when no providers connected 11. **Auth:** Only accessible to authenticated admin users ### Testing ```elixir # test/berrypod_web/live/provider_live_test.exs describe "Index" do setup :register_and_log_in_user test "lists all provider connections", %{conn: conn} do connection = provider_connection_fixture() {:ok, _lv, html} = live(conn, ~p"/admin/providers") assert html =~ connection.name end test "saves new provider connection", %{conn: conn} do {:ok, lv, _html} = live(conn, ~p"/admin/providers/new") # Mock test_connection response expect_test_connection_success() lv |> form("#provider-form", provider: %{name: "Test", api_key: "key123"}) |> render_submit() assert_patch(lv, ~p"/admin/providers") assert render(lv) =~ "Test" end test "deletes provider connection", %{conn: conn} do connection = provider_connection_fixture() {:ok, lv, _html} = live(conn, ~p"/admin/providers") lv |> element("#connection-#{connection.id} button", "Delete") |> render_click() refute render(lv) =~ connection.name end end ``` ### Dependencies - **ProductSyncWorker:** Must exist (stub OK, full implementation separate task) - **Providers.Printify.test_connection/1:** Already implemented - **Providers.Printify.register_webhooks/1:** Implement as part of this task - **Providers.Printify.unregister_webhooks/1:** Implement as part of this task ### Out of Scope (Future Tasks) - Product list/management UI (`/admin/products`) - Real-time sync progress updates (PubSub) - OAuth flow for providers that support it - Multiple connections per provider type --- ## Task: Product Sync Strategy > **Status:** Planned > **Estimate:** ~2.5 hours total > **Prerequisite:** Admin Provider Setup UI ### Goal Implement a robust product sync strategy with three mechanisms: 1. **Manual sync** - Admin clicks "Sync Now" (initial setup, recovery) 2. **Webhook sync** - Printify pushes updates in real-time (primary) 3. **Scheduled sync** - Daily fallback to catch missed webhooks (optional) ### Printify Webhook Events | Event | Trigger | Action | |-------|---------|--------| | `product:publish:started` | Product published to shop | Fetch & upsert product | | `product:deleted` | Product removed from shop | Archive product locally | | `shop:disconnected` | API access revoked | Mark connection as disconnected | ### Files to Create | File | Purpose | |------|---------| | `lib/berrypod/workers/product_sync_worker.ex` | Oban worker for full/single product sync | | `lib/berrypod_web/controllers/webhook_controller.ex` | Receives webhooks from providers | | `lib/berrypod/webhooks/printify_handler.ex` | Printify-specific webhook processing | | `test/berrypod/workers/product_sync_worker_test.exs` | Worker tests | | `test/berrypod_web/controllers/webhook_controller_test.exs` | Webhook endpoint tests | ### Part 1: ProductSyncWorker (~1hr) Oban worker that syncs products from a provider connection. ```elixir defmodule Berrypod.Workers.ProductSyncWorker do use Oban.Worker, queue: :sync, max_attempts: 3, unique: [period: 60, fields: [:args, :queue]] alias Berrypod.Products alias Berrypod.Providers @impl Oban.Worker def perform(%Oban.Job{args: %{"connection_id" => conn_id} = args}) do conn = Products.get_provider_connection!(conn_id) provider = Providers.for_type(conn.provider_type) Products.update_sync_status(conn, "syncing", nil) result = case args do %{"product_id" => product_id} -> # Single product sync (from webhook) sync_single_product(conn, provider, product_id) _ -> # Full sync (manual or scheduled) sync_all_products(conn, provider) end case result do {:ok, stats} -> Products.update_sync_status(conn, "completed", DateTime.utc_now()) {:ok, stats} {:error, reason} -> Products.update_sync_status(conn, "failed", nil) {:error, reason} end end defp sync_all_products(conn, provider) do case provider.fetch_products(conn) do {:ok, products} -> stats = Enum.reduce(products, %{synced: 0, failed: 0}, fn product, acc -> case Products.upsert_product(conn, product) do {:ok, _} -> %{acc | synced: acc.synced + 1} {:error, _} -> %{acc | failed: acc.failed + 1} end end) {:ok, stats} {:error, reason} -> {:error, reason} end end defp sync_single_product(conn, provider, product_id) do case provider.fetch_product(conn, product_id) do {:ok, product} -> Products.upsert_product(conn, product) {:error, :not_found} -> # Product deleted on provider Products.archive_product_by_provider(conn.id, product_id) {:error, reason} -> {:error, reason} end end end ``` **Job types:** - Full sync: `%{connection_id: conn.id}` - Single product: `%{connection_id: conn.id, product_id: "ext_123"}` - Archive: `%{connection_id: conn.id, product_id: "ext_123", action: "archive"}` ### Part 2: Webhook Endpoint (~1.5hr) **Route:** ```elixir # In router.ex (outside auth scopes - public endpoint) post "/webhooks/printify", WebhookController, :printify ``` **Controller:** ```elixir defmodule BerrypodWeb.WebhookController do use BerrypodWeb, :controller alias Berrypod.Webhooks.PrintifyHandler def printify(conn, params) do with :ok <- verify_printify_signature(conn), :ok <- PrintifyHandler.handle(params) do json(conn, %{status: "ok"}) else {:error, :invalid_signature} -> conn |> put_status(401) |> json(%{error: "Invalid signature"}) {:error, reason} -> conn |> put_status(422) |> json(%{error: reason}) end end defp verify_printify_signature(conn) do # Printify signs webhooks with HMAC-SHA256 # Header: X-Printify-Signature signature = get_req_header(conn, "x-printify-signature") |> List.first() body = conn.assigns[:raw_body] secret = Application.get_env(:berrypod, :printify_webhook_secret) expected = :crypto.mac(:hmac, :sha256, secret, body) |> Base.encode16(case: :lower) if Plug.Crypto.secure_compare(signature || "", expected) do :ok else {:error, :invalid_signature} end end end ``` **Handler:** ```elixir defmodule Berrypod.Webhooks.PrintifyHandler do alias Berrypod.Products alias Berrypod.Workers.ProductSyncWorker def handle(%{"type" => "product:publish:started", "resource" => resource}) do %{"shop_id" => shop_id, "id" => product_id} = resource with {:ok, conn} <- find_connection_by_shop(shop_id) do %{connection_id: conn.id, product_id: to_string(product_id)} |> ProductSyncWorker.new() |> Oban.insert() end end def handle(%{"type" => "product:deleted", "resource" => resource}) do %{"shop_id" => shop_id, "id" => product_id} = resource with {:ok, conn} <- find_connection_by_shop(shop_id) do Products.archive_product_by_provider(conn.id, to_string(product_id)) end end def handle(%{"type" => "shop:disconnected", "resource" => %{"shop_id" => shop_id}}) do with {:ok, conn} <- find_connection_by_shop(shop_id) do Products.update_provider_connection(conn, %{enabled: false, sync_status: "disconnected"}) end end def handle(%{"type" => type}) do # Log unknown webhook types but don't fail Logger.info("Unhandled Printify webhook: #{type}") :ok end defp find_connection_by_shop(shop_id) do case Products.get_provider_connection_by_shop_id("printify", shop_id) do nil -> {:error, :connection_not_found} conn -> {:ok, conn} end end end ``` ### Part 3: Scheduled Sync (Optional) Add to Oban config for daily fallback: ```elixir # In config/config.exs config :berrypod, Oban, plugins: [ {Oban.Plugins.Cron, crontab: [ {"0 3 * * *", Berrypod.Workers.ScheduledSyncWorker} # 3 AM daily ]} ] ``` ```elixir defmodule Berrypod.Workers.ScheduledSyncWorker do use Oban.Worker, queue: :sync def perform(_job) do Products.list_provider_connections(enabled: true) |> Enum.each(fn conn -> %{connection_id: conn.id} |> ProductSyncWorker.new(schedule_in: Enum.random(0..300)) # Stagger over 5 min |> Oban.insert() end) :ok end end ``` ### Context Additions Add to `lib/berrypod/products.ex`: ```elixir def archive_product_by_provider(connection_id, provider_product_id) do case get_product_by_provider(connection_id, provider_product_id) do nil -> {:ok, :not_found} product -> update_product(product, %{status: "archived", visible: false}) end end def get_provider_connection_by_shop_id(provider_type, shop_id) do from(c in ProviderConnection, where: c.provider_type == ^provider_type, where: fragment("json_extract(config, '$.shop_id') = ?", ^shop_id) ) |> Repo.one() end ``` ### Webhook Registration Webhook registration with Printify is handled by the Admin Provider Setup UI task: - **On save:** `Providers.Printify.register_webhooks/1` is called - **On delete:** `Providers.Printify.unregister_webhooks/1` cleans up See the "Webhook Registration Flow" section in the Admin Provider Setup UI task for implementation details. **Note:** Webhooks require a public URL. For local dev, use ngrok or similar: ```bash ngrok http 4000 # Then set WEBHOOK_URL=https://abc123.ngrok.io in .env ``` ### Acceptance Criteria 1. **Manual sync:** "Sync Now" button enqueues ProductSyncWorker, products appear 2. **Webhook received:** POST to `/webhooks/printify` with valid signature succeeds 3. **Invalid signature:** Returns 401, no job enqueued 4. **Product published:** Webhook triggers single-product sync 5. **Product deleted:** Webhook archives product locally 6. **Shop disconnected:** Connection marked as disabled 7. **Scheduled sync:** (If implemented) Runs daily, syncs all enabled connections ### Testing ```elixir describe "WebhookController.printify/2" do test "rejects invalid signature" do conn = post(build_conn(), ~p"/webhooks/printify", %{type: "product:deleted"}) assert json_response(conn, 401)["error"] == "Invalid signature" end test "handles product:publish:started" do conn = provider_connection_fixture(config: %{"shop_id" => "12345"}) payload = %{"type" => "product:publish:started", "resource" => %{"shop_id" => "12345", "id" => "prod_1"}} conn = post(signed_conn(payload), ~p"/webhooks/printify", payload) assert json_response(conn, 200) assert_enqueued(worker: ProductSyncWorker, args: %{connection_id: conn.id, product_id: "prod_1"}) end end ``` --- ## Verification (Manual) - Create Printify connection with API key - `Products.test_provider_connection(conn)` returns shop info 2. **Product Sync** - Run `ProductSyncWorker.enqueue(conn.id)` - Products appear in database with images and variants - Re-sync updates existing, doesn't duplicate 3. **Order Submission** - Create order from cart - `Orders.submit_to_provider(order)` returns provider_order_id - Order status updates from provider 4. **Shop Pages** - `/collections/all` shows real products - `/products/:slug` shows product details - Add to cart works with real variant IDs