From adaa564f4c15117c3a4146aafbc1eca1e201c757 Mon Sep 17 00:00:00 2001 From: Jamey Greenwood Date: Wed, 21 Jan 2026 21:56:30 +0000 Subject: [PATCH] docs: add image optimization pipeline plan Detailed implementation plan for automatic image optimization: - Lossless WebP storage in SQLite (26-41% smaller than PNG) - AVIF/WebP/JPEG responsive variants generated to disk cache - Oban for durable async processing with SQLite - 10-phase incremental implementation with checkpoints - Comprehensive test coverage strategy Co-Authored-By: Claude Opus 4.5 --- docs/plans/image-optimization.md | 1542 ++++++++++++++++++++++++++++++ 1 file changed, 1542 insertions(+) create mode 100644 docs/plans/image-optimization.md diff --git a/docs/plans/image-optimization.md b/docs/plans/image-optimization.md new file mode 100644 index 0000000..1f034c7 --- /dev/null +++ b/docs/plans/image-optimization.md @@ -0,0 +1,1542 @@ +# Plan: Automatic Image Optimization Pipeline + +**Location:** `docs/plans/image-optimization.md` +**Purpose:** Track implementation progress - update this file after each phase completes. + +--- + +## Progress Tracker + +**Current Phase:** Not started +**Last Updated:** 2026-01-21 + +| Phase | Status | Commit | +|-------|--------|--------| +| 1. Oban dependency + config | ⬜ Pending | - | +| 2. Migration + Schema | ⬜ Pending | - | +| 3. Optimizer module | ⬜ Pending | - | +| 4. Oban worker | ⬜ Pending | - | +| 5. Media module integration | ⬜ Pending | - | +| 6. VariantCache GenServer | ⬜ Pending | - | +| 7. Responsive image component | ⬜ Pending | - | +| 8. ImageController disk serving | ⬜ Pending | - | +| 9. Mix task for mockups | ⬜ Pending | - | +| 10. Final integration + Lighthouse | ⬜ Pending | - | + +**Legend:** ⬜ Pending | 🔄 In Progress | ✅ Complete | ❌ Blocked + +--- + +## Overview + +Build a reusable image processing pipeline that generates optimized, responsive image variants for all uploaded content (logos, header backgrounds) and product mockups. + +**Key architectural principles:** +1. Original images stored in SQLite as **lossless WebP** blobs (26-41% smaller than PNG) +2. All derived formats (responsive variants, thumbnails) generated to disk cache +3. Disk cache is regenerable - can be deleted and recreated from DB +4. Only generate sizes ≤ source dimensions (no upscaling) +5. Available widths computed dynamically from `source_width` (not stored) + +--- + +## Architecture + +### Database Schema (Source of Truth) + +```elixir +# images table - ONLY stores originals +field :data, :binary # Original image as lossless WebP +field :source_width, :integer # Original dimensions +field :source_height, :integer +field :variants_status, Ecto.Enum, values: [:pending, :processing, :complete, :failed] + +# REMOVED: available_widths - computed from source_width +# REMOVED: thumbnail_data - derived, stored on disk +``` + +### Why Lossless WebP for Storage? + +| Format | Lossless Size vs PNG | Encoding Speed | +|--------|---------------------|----------------| +| PNG | Baseline | Fast | +| WebP | **26-41% smaller** | Fast | +| AVIF | ~20% smaller | Very slow | + +WebP lossless is the optimal choice: best compression for photos, fast encoding. + +### Disk Cache (Derived, Regenerable) + +``` +priv/static/image_cache/ # Gitignored, regenerated on startup +├── {uuid}-thumb.jpg # 200px thumbnail for admin UI +├── {uuid}-400.avif +├── {uuid}-400.webp +├── {uuid}-400.jpg +├── {uuid}-800.avif +├── {uuid}-800.webp +├── {uuid}-800.jpg +├── {uuid}-1200.avif # Only if source >= 1200px +├── {uuid}-1200.webp +└── {uuid}-1200.jpg +``` + +### Size Selection Logic + +Only generate widths ≤ source dimensions (no upscaling): + +```elixir +@all_widths [400, 800, 1200] + +def applicable_widths(source_width) do + @all_widths + |> Enum.filter(&(&1 <= source_width)) + |> case do + [] -> [source_width] # Source smaller than 400px + widths -> widths + end +end +``` + +### Disk Space Analysis + +**Per-image storage breakdown (1200px source):** + +| Format | 400w | 800w | 1200w | Subtotal | +|--------|------|------|-------|----------| +| AVIF | ~15KB | ~40KB | ~70KB | ~125KB | +| WebP | ~25KB | ~60KB | ~100KB | ~185KB | +| **JPEG** | ~40KB | ~100KB | ~180KB | **~320KB** | +| Thumbnail | - | - | - | ~10KB | +| **Total** | | | | **~640KB** | + +**At scale:** +- 1 shop × 100 products = ~64MB +- 10,000 shops × 100 products = **~640GB** + +**Alternative: Skip pre-generating JPEG (on-demand only)** +- JPEG is ~50% of total storage +- Savings: 320GB at 10K shops scale +- Trade-off: First JPEG request takes ~100ms to generate +- Only affects <5% of users (ancient browsers) + +**Recommendation: Pre-generate all formats** because: +1. Consistent latency for all users +2. 640GB is ~$15-30/month cloud storage +3. Simpler architecture (no on-demand generation) +4. Can add JPEG pruning later if needed + +--- + +## Processing Strategy: Oban with Aggressive Pruning + +Using Oban for durable job processing with SQLite. Jobs are pruned aggressively to prevent database bloat. + +### Why Oban over Synchronous? + +| Aspect | Synchronous | Oban | +|--------|-------------|------| +| User experience | Waits 1-2s | Instant response | +| Crash recovery | Startup retry | Automatic retry | +| Visibility | Logs only | Status tracking | +| Complexity | Simple | Moderate | + +For a production-quality system, Oban's durability is worth the small complexity cost. + +### Oban Configuration + +```elixir +# config/config.exs +config :simpleshop_theme, Oban, + engine: Oban.Engines.Lite, # SQLite support + repo: SimpleshopTheme.Repo, + plugins: [ + # Prune completed jobs after 60 seconds - keeps DB lean + {Oban.Plugins.Pruner, max_age: 60} + ], + queues: [images: 2] +``` + +With `max_age: 60`, the Oban tables typically contain: +- 0-10 active jobs during processing +- Failed jobs kept for debugging (can add separate max_age) +- Completed jobs deleted within 1 minute + +### Job Worker + +```elixir +defmodule SimpleshopTheme.Workers.ImageVariants do + use Oban.Worker, queue: :images, max_attempts: 3 + + @impl Oban.Worker + def perform(%Oban.Job{args: %{"image_id" => image_id}}) do + case Images.Optimizer.process_for_image(image_id) do + {:ok, _} -> :ok + {:error, reason} -> {:error, reason} + end + end +end +``` + +### Upload Flow + +```elixir +def upload_image(attrs) do + # Convert to lossless WebP before storing + attrs = convert_to_lossless_webp(attrs) + + case Repo.insert(ImageSchema.changeset(%ImageSchema{}, attrs)) do + {:ok, image} -> + # Enqueue async variant generation + %{image_id: image.id} + |> SimpleshopTheme.Workers.ImageVariants.new() + |> Oban.insert() + + {:ok, image} + + error -> + error + end +end +``` + +### Startup Recovery + +Still needed for cache deletion scenarios: + +```elixir +# In VariantCache GenServer init +defp ensure_all_variants do + # Find images with missing disk variants + ImageSchema + |> where([i], i.variants_status == :complete) + |> where([i], i.is_svg == false) + |> Repo.all() + |> Enum.reject(&disk_variants_exist?/1) + |> Enum.each(fn image -> + # Re-enqueue for processing + %{image_id: image.id} + |> SimpleshopTheme.Workers.ImageVariants.new() + |> Oban.insert() + end) +end +``` + +--- + +## Implementation Steps + +### Step 1: Add Oban Dependency + +**File:** `mix.exs` + +```elixir +defp deps do + [ + # ... existing deps + {:oban, "~> 2.18"} + ] +end +``` + +### Step 2: Migration - Update Image Schema + +**File:** `priv/repo/migrations/YYYYMMDDHHMMSS_add_image_metadata.exs` + +```elixir +defmodule SimpleshopTheme.Repo.Migrations.AddImageMetadata do + use Ecto.Migration + + def change do + alter table(:images) do + add :source_width, :integer + add :source_height, :integer + add :variants_status, :string, default: "pending" + + # Remove thumbnail_data - now derived to disk + remove :thumbnail_data + end + + # Oban tables for job queue + Oban.Migration.up(version: 12) + end +end +``` + +### Step 3: Image Optimizer Module + +**File:** `lib/simpleshop_theme/images/optimizer.ex` + +```elixir +defmodule SimpleshopTheme.Images.Optimizer do + @moduledoc """ + Generates optimized image variants. Only creates sizes ≤ source dimensions. + """ + + require Logger + + @all_widths [400, 800, 1200] + @formats [:avif, :webp, :jpg] + @thumb_size 200 + @cache_dir "priv/static/image_cache" + + def cache_dir, do: @cache_dir + def all_widths, do: @all_widths + + @doc """ + Convert uploaded image to lossless WebP for storage. + """ + def to_lossless_webp(image_data) when is_binary(image_data) do + with {:ok, image} <- Image.from_binary(image_data), + {width, height, _} <- Image.shape(image), + {:ok, webp_data} <- Image.write(image, :memory, suffix: ".webp", quality: 100) do + {:ok, webp_data, width, height} + end + end + + @doc """ + Compute applicable widths from source dimensions. + """ + def applicable_widths(source_width) do + @all_widths + |> Enum.filter(&(&1 <= source_width)) + |> case do + [] -> [source_width] + widths -> widths + end + end + + @doc """ + Process image and generate all applicable variants. + Called by Oban worker. + """ + def process_for_image(image_id) do + alias SimpleshopTheme.{Repo, Media.Image} + + case Repo.get(Image, image_id) do + nil -> + {:error, :not_found} + + %{data: nil} -> + {:error, :no_data} + + %{is_svg: true} = image -> + # SVGs don't need variants + Repo.update!(Image.changeset(image, %{variants_status: :complete})) + {:ok, :svg_skipped} + + %{data: data, source_width: width} = image -> + File.mkdir_p!(@cache_dir) + + with {:ok, vips_image} <- Image.from_binary(data) do + widths = applicable_widths(width) + + # Generate thumbnail + variants in parallel + tasks = [ + Task.async(fn -> generate_thumbnail(vips_image, image_id) end) + | for w <- widths, fmt <- @formats do + Task.async(fn -> generate_variant(vips_image, image_id, w, fmt) end) + end + ] + + Task.await_many(tasks, :timer.seconds(120)) + + # Mark complete + Repo.update!(Image.changeset(image, %{variants_status: :complete})) + {:ok, widths} + end + end + end + + defp generate_thumbnail(image, id) do + path = Path.join(@cache_dir, "#{id}-thumb.jpg") + return_if_exists(path, fn -> + with {:ok, thumb} <- Image.thumbnail(image, @thumb_size), + {:ok, _} <- Image.write(thumb, path, quality: 80) do + :ok + end + end) + end + + defp generate_variant(image, id, width, format) do + path = Path.join(@cache_dir, "#{id}-#{width}.#{format_ext(format)}") + return_if_exists(path, fn -> + with {:ok, resized} <- Image.thumbnail(image, width), + {:ok, _} <- write_format(resized, path, format) do + :ok + end + end) + end + + defp return_if_exists(path, generate_fn) do + if File.exists?(path), do: {:ok, :cached}, else: generate_fn.() + end + + defp format_ext(:jpg), do: "jpg" + defp format_ext(:webp), do: "webp" + defp format_ext(:avif), do: "avif" + + defp write_format(image, path, :avif) do + Image.write(image, path, effort: 5, minimize_file_size: true) + end + + defp write_format(image, path, :webp) do + Image.write(image, path, effort: 6, minimize_file_size: true) + end + + defp write_format(image, path, :jpg) do + Image.write(image, path, quality: 80, minimize_file_size: true) + end + + @doc """ + Check if disk variants exist for an image. + """ + def disk_variants_exist?(image_id, source_width) do + widths = applicable_widths(source_width) + thumb = File.exists?(Path.join(@cache_dir, "#{image_id}-thumb.jpg")) + + variants = Enum.all?(widths, fn w -> + Enum.all?(@formats, fn fmt -> + File.exists?(Path.join(@cache_dir, "#{image_id}-#{w}.#{format_ext(fmt)}")) + end) + end) + + thumb and variants + end +end +``` + +### Step 4: Oban Worker + +**File:** `lib/simpleshop_theme/workers/image_variants.ex` + +```elixir +defmodule SimpleshopTheme.Workers.ImageVariants do + use Oban.Worker, + queue: :images, + max_attempts: 3, + unique: [period: 60] # Prevent duplicate jobs + + alias SimpleshopTheme.Images.Optimizer + + @impl Oban.Worker + def perform(%Oban.Job{args: %{"image_id" => image_id}}) do + case Optimizer.process_for_image(image_id) do + {:ok, _} -> :ok + {:error, reason} -> {:error, reason} + end + end +end +``` + +### Step 5: Update Media Module + +**File:** `lib/simpleshop_theme/media.ex` + +```elixir +alias SimpleshopTheme.Images.Optimizer +alias SimpleshopTheme.Workers.ImageVariants + +def upload_image(attrs) do + # Convert to lossless WebP before storing + with {:ok, webp_data, width, height} <- Optimizer.to_lossless_webp(attrs.data) do + attrs = + attrs + |> Map.put(:data, webp_data) + |> Map.put(:source_width, width) + |> Map.put(:source_height, height) + |> Map.put(:variants_status, :pending) + + case Repo.insert(ImageSchema.changeset(%ImageSchema{}, attrs)) do + {:ok, image} -> + # Enqueue async variant generation + %{image_id: image.id} + |> ImageVariants.new() + |> Oban.insert() + + {:ok, image} + + error -> + error + end + end +end + +# Thumbnail served from disk cache +def get_thumbnail_path(image_id) do + Path.join(Optimizer.cache_dir(), "#{image_id}-thumb.jpg") +end +``` + +### Step 6: Startup Recovery GenServer + +**File:** `lib/simpleshop_theme/images/variant_cache.ex` + +```elixir +defmodule SimpleshopTheme.Images.VariantCache do + @moduledoc """ + Ensures all image variants exist on startup. + Enqueues Oban jobs for any missing variants. + """ + + use GenServer + require Logger + + alias SimpleshopTheme.Repo + alias SimpleshopTheme.Media.Image, as: ImageSchema + alias SimpleshopTheme.Images.Optimizer + alias SimpleshopTheme.Workers.ImageVariants + import Ecto.Query + + def start_link(_opts) do + GenServer.start_link(__MODULE__, [], name: __MODULE__) + end + + @impl true + def init(_) do + # Run async to not block startup + Task.start(fn -> ensure_all_variants() end) + {:ok, %{}} + end + + defp ensure_all_variants do + Logger.info("Checking image variant cache...") + File.mkdir_p!(Optimizer.cache_dir()) + + # 1. Find incomplete images (status != complete) + incomplete = + ImageSchema + |> where([i], i.variants_status != :complete or is_nil(i.variants_status)) + |> where([i], i.is_svg == false) + |> Repo.all() + + # 2. Find complete images with missing disk files (cache was deleted) + complete_missing = + ImageSchema + |> where([i], i.variants_status == :complete) + |> where([i], i.is_svg == false) + |> Repo.all() + |> Enum.reject(fn img -> + Optimizer.disk_variants_exist?(img.id, img.source_width) + end) + + to_process = incomplete ++ complete_missing + + if to_process == [] do + Logger.info("Image cache: all variants up to date") + else + Logger.info("Image cache: enqueueing #{length(to_process)} images for processing") + + Enum.each(to_process, fn image -> + # Reset status and enqueue + Repo.update!(ImageSchema.changeset(image, %{variants_status: :pending})) + + %{image_id: image.id} + |> ImageVariants.new() + |> Oban.insert() + end) + end + end +end +``` + +### Step 7: Responsive Image Component + +**File:** `lib/simpleshop_theme_web/components/shop_components.ex` + +```elixir +@doc """ +Renders a responsive element with AVIF, WebP, and JPEG sources. +Computes available widths from source_width (no separate storage needed). + +## Examples + + <.responsive_image + src="/image_cache/abc123" + source_width={1200} + alt="Product image" + priority={true} + /> +""" +attr :src, :string, required: true, doc: "Base path without size/extension" +attr :alt, :string, required: true +attr :source_width, :integer, required: true, doc: "Original image width" +attr :sizes, :string, default: "100vw" +attr :class, :string, default: "" +attr :width, :integer, default: nil +attr :height, :integer, default: nil +attr :priority, :boolean, default: false + +def responsive_image(assigns) do + alias SimpleshopTheme.Images.Optimizer + + # Compute available widths from source dimensions + available = Optimizer.applicable_widths(assigns.source_width) + default_width = Enum.max(available) + + assigns = + assigns + |> assign(:available_widths, available) + |> assign(:default_width, default_width) + + ~H""" + + + + {@alt} + + """ +end + +defp build_srcset(base, widths, format) do + widths + |> Enum.sort() + |> Enum.map(&"#{base}-#{&1}.#{format} #{&1}w") + |> Enum.join(", ") +end +``` + +### Step 8: Update Thumbnail Serving + +**File:** `lib/simpleshop_theme_web/controllers/image_controller.ex` + +```elixir +def thumbnail(conn, %{"id" => id}) do + thumb_path = Path.join("priv/static/image_cache", "#{id}-thumb.jpg") + + if File.exists?(thumb_path) do + conn + |> put_resp_content_type("image/jpeg") + |> put_resp_header("cache-control", "public, max-age=31536000, immutable") + |> send_file(200, thumb_path) + else + # Fallback: try to generate on-demand + case Media.get_image(id) do + %{data: data} when is_binary(data) -> + # Generate just the thumbnail + with {:ok, image} <- Image.from_binary(data), + {:ok, thumb} <- Image.thumbnail(image, 200), + {:ok, binary} <- Image.write(thumb, :memory, suffix: ".jpg") do + # Save for next time + File.write!(thumb_path, binary) + + conn + |> put_resp_content_type("image/jpeg") + |> put_resp_header("cache-control", "public, max-age=31536000, immutable") + |> send_resp(200, binary) + else + _ -> send_resp(conn, 404, "Thumbnail not available") + end + + _ -> + send_resp(conn, 404, "Image not found") + end + end +end +``` + +### Step 9: Mix Task for Mockups + +**File:** `lib/mix/tasks/optimize_images.ex` + +```elixir +defmodule Mix.Tasks.OptimizeImages do + @shortdoc "Generate optimized variants for product mockups" + @moduledoc """ + Generates responsive image variants (AVIF, WebP, JPEG) for mockup images. + + ## Usage + + mix optimize_images + mix optimize_images --force + mix optimize_images --dir path/to/images + """ + + use Mix.Task + + @default_dir "priv/static/mockups" + @widths [400, 800, 1200] + @formats [:avif, :webp, :jpg] + + def run(args) do + Application.ensure_all_started(:image) + + {opts, _} = OptionParser.parse!(args, strict: [force: :boolean, dir: :string]) + force = opts[:force] || false + dir = opts[:dir] || @default_dir + + originals = find_originals(dir) + IO.puts("Found #{length(originals)} original mockup images") + + originals + |> Task.async_stream(&process(&1, force, dir), + max_concurrency: System.schedulers_online(), + timeout: :timer.minutes(2)) + |> Enum.each(fn + {:ok, {:generated, name, count}} -> + IO.puts(" #{name}: generated #{count} variants") + {:ok, :skipped} -> + :ok + {:exit, reason} -> + IO.puts(" Error: #{inspect(reason)}") + end) + + IO.puts("Done!") + end + + defp find_originals(dir) do + dir + |> Path.join("*.jpg") + |> Path.wildcard() + |> Enum.reject(&is_variant?/1) + end + + defp is_variant?(path) do + String.match?(Path.basename(path, ".jpg"), ~r/-\d+$/) + end + + defp process(path, force, dir) do + name = Path.basename(path, ".jpg") + + if not force and all_variants_exist?(name, dir) do + :skipped + else + with {:ok, image} <- Image.open(path), + {width, _, _} <- Image.shape(image) do + + applicable = Enum.filter(@widths, &(&1 <= width)) + + count = + for w <- applicable, fmt <- @formats do + generate(image, name, w, fmt, dir) + end + |> Enum.count(&(&1 == :generated)) + + {:generated, name, count} + end + end + end + + defp all_variants_exist?(name, dir) do + Enum.all?(@widths, fn w -> + Enum.all?(@formats, fn fmt -> + File.exists?(Path.join(dir, "#{name}-#{w}.#{ext(fmt)}")) + end) + end) + end + + defp generate(image, name, width, format, dir) do + path = Path.join(dir, "#{name}-#{width}.#{ext(format)}") + + if File.exists?(path) do + :cached + else + {:ok, resized} = Image.thumbnail(image, width) + {:ok, _} = write(resized, path, format) + :generated + end + end + + defp ext(:jpg), do: "jpg" + defp ext(:webp), do: "webp" + defp ext(:avif), do: "avif" + + defp write(image, path, :avif), do: Image.write(image, path, effort: 5, minimize_file_size: true) + defp write(image, path, :webp), do: Image.write(image, path, effort: 6, minimize_file_size: true) + defp write(image, path, :jpg), do: Image.write(image, path, quality: 80, minimize_file_size: true) +end +``` + +--- + +## File Changes Summary + +### Create: +- `lib/simpleshop_theme/images/optimizer.ex` - Core optimization logic +- `lib/simpleshop_theme/images/variant_cache.ex` - Startup recovery GenServer +- `lib/simpleshop_theme/workers/image_variants.ex` - Oban worker +- `lib/mix/tasks/optimize_images.ex` - Mockup batch processing +- `priv/repo/migrations/*_add_image_metadata.exs` - Schema + Oban tables + +### Modify: +- `mix.exs` - Add Oban dependency +- `config/config.exs` - Oban configuration +- `lib/simpleshop_theme/media/image.ex` - Add source_width/height, remove thumbnail_data +- `lib/simpleshop_theme/media.ex` - Convert to WebP, enqueue Oban job +- `lib/simpleshop_theme/application.ex` - Add Oban + VariantCache to supervision +- `lib/simpleshop_theme_web/components/shop_components.ex` - Responsive image component +- `lib/simpleshop_theme_web/controllers/image_controller.ex` - Serve thumbnails from disk +- `.gitignore` - Add `/priv/static/image_cache/` + +--- + +## Key Design Decisions + +### 1. Database stores ONLY originals as lossless WebP +- 26-41% smaller than PNG +- `source_width/height`: for computing applicable widths +- `variants_status`: tracks processing state +- NO `thumbnail_data` - derived to disk +- NO `available_widths` - computed from source_width + +### 2. Only generate sizes ≤ source (no upscaling) +- Small logos might only get 400w +- Large images get 400, 800, 1200 +- Component computes srcset dynamically from source_width + +### 3. Oban for durable async processing +- Instant upload response (no user wait) +- Automatic retry on failure +- Aggressive pruning (60s max_age) keeps DB lean +- Startup recovery for deleted cache + +### 4. Task.async for parallelism within Oban jobs +- **Oban**: handles job scheduling, persistence, retry +- **Task.async**: handles parallel execution *within* a single job +- This is the standard pattern - Oban for durability, Task for concurrency +- If the job fails, all 10 variants retry together (acceptable since they share source) + +### 5. Idempotent generation +- Checks if file exists before generating +- Safe to run multiple times +- `--force` flag for full regeneration + +--- + +## Phased Implementation (with Checkpoints) + +Each phase is a self-contained unit. After each phase: +1. Run the specified tests +2. Perform manual verification +3. Get user confirmation +4. Make semantic commit +5. Update Progress Tracker above + +### Phase 1: Oban Dependency + Config + +**Files to modify:** +- `mix.exs` - Add `{:oban, "~> 2.18"}` +- `config/config.exs` - Add Oban config + +**Tests:** `mix deps.get && mix compile` + +**Manual verification:** +```bash +mix phx.server +# Server should start without errors +``` + +**Commit:** `feat: add oban dependency for background jobs` + +--- + +### Phase 2: Migration + Schema Changes + +**Files to create/modify:** +- `priv/repo/migrations/*_add_image_metadata.exs` - New migration +- `lib/simpleshop_theme/media/image.ex` - Add source_width/height, variants_status; remove thumbnail_data + +**Tests:** `mix ecto.migrate && mix test test/simpleshop_theme/media_test.exs` (if exists) + +**Manual verification:** +```bash +# Check migration applied +sqlite3 simpleshop_theme_dev.db ".schema images" +# Should show new columns: source_width, source_height, variants_status +# Should NOT have: thumbnail_data +``` + +**Commit:** `feat: add image metadata fields for optimization pipeline` + +--- + +### Phase 3: Optimizer Module + +**Files to create:** +- `lib/simpleshop_theme/images/optimizer.ex` +- `test/simpleshop_theme/images/optimizer_test.exs` +- `test/support/fixtures/image_fixtures.ex` +- `test/fixtures/sample_1200x800.jpg` (test image) + +**Tests:** `mix test test/simpleshop_theme/images/optimizer_test.exs` + +**Manual verification:** +```elixir +# In iex -S mix +alias SimpleshopTheme.Images.Optimizer +Optimizer.applicable_widths(1500) # Should return [400, 800, 1200] +Optimizer.applicable_widths(300) # Should return [300] +``` + +**Commit:** `feat: add image optimizer module` + +--- + +### Phase 4: Oban Worker + +**Files to create:** +- `lib/simpleshop_theme/workers/image_variants.ex` +- `test/simpleshop_theme/workers/image_variants_test.exs` + +**Files to modify:** +- `lib/simpleshop_theme/application.ex` - Add Oban to supervision tree + +**Tests:** `mix test test/simpleshop_theme/workers/image_variants_test.exs` + +**Manual verification:** +```bash +mix phx.server +# Check logs for "Oban started" or similar +``` + +**Commit:** `feat: add image variants oban worker` + +--- + +### Phase 5: Media Module Integration + +**Files to modify:** +- `lib/simpleshop_theme/media.ex` - Update upload_image to use optimizer + enqueue worker + +**Tests:** `mix test test/simpleshop_theme/media_test.exs` (existing tests should pass) + +**Manual verification:** +1. Go to `/admin/theme` +2. Upload a new logo image +3. Check `priv/static/image_cache/` for generated variants +4. Check database: `sqlite3 simpleshop_theme_dev.db "SELECT id, source_width, variants_status FROM images ORDER BY inserted_at DESC LIMIT 1"` + +**Commit:** `feat: integrate optimizer with image uploads` + +--- + +### Phase 6: VariantCache GenServer + +**Files to create:** +- `lib/simpleshop_theme/images/variant_cache.ex` + +**Files to modify:** +- `lib/simpleshop_theme/application.ex` - Add VariantCache to supervision tree + +**Tests:** Manual (startup behavior) + +**Manual verification:** +```bash +# Delete the cache and restart +rm -rf priv/static/image_cache/ +mix phx.server +# Check logs for "Image cache: enqueueing X images for processing" +# Check that cache directory is recreated with variants +``` + +**Commit:** `feat: add variant cache startup recovery` + +--- + +### Phase 7: Responsive Image Component + +**Files to create:** +- `test/simpleshop_theme_web/components/shop_components_test.exs` + +**Files to modify:** +- `lib/simpleshop_theme_web/components/shop_components.ex` - Add responsive_image component + +**Tests:** `mix test test/simpleshop_theme_web/components/shop_components_test.exs` + +**Manual verification:** +```elixir +# In iex -S mix +# Verify the component renders correct HTML +``` + +**Commit:** `feat: add responsive image component` + +--- + +### Phase 8: ImageController Disk Serving + +**Files to modify:** +- `lib/simpleshop_theme_web/controllers/image_controller.ex` - Update thumbnail to serve from disk + +**Tests:** Existing controller tests should pass + +**Manual verification:** +1. Upload an image via admin +2. Navigate to `/images/{id}/thumbnail` +3. Check browser DevTools → Network → verify served from disk (fast response, correct headers) + +**Commit:** `feat: serve thumbnails from disk cache` + +--- + +### Phase 9: Mix Task for Mockups + +**Files to create:** +- `lib/mix/tasks/optimize_images.ex` +- `test/mix/tasks/optimize_images_test.exs` + +**Files to modify:** +- `.gitignore` - Add `/priv/static/image_cache/` and variant patterns + +**Tests:** `mix test test/mix/tasks/optimize_images_test.exs` + +**Manual verification:** +```bash +mix optimize_images +# Should show "Found X original mockup images" +# Check priv/static/mockups/ for generated variants (.avif, .webp, .jpg at various sizes) +ls -la priv/static/mockups/ +``` + +**Commit:** `feat: add mix task for mockup image optimization` + +--- + +### Phase 10: Final Integration + Lighthouse + +**Tests:** `mix test` (full test suite) + +**Manual verification:** +1. Start fresh: `rm -rf priv/static/image_cache/ && mix ecto.reset` +2. Run: `mix phx.server` +3. Upload logo, header images via admin +4. Verify variants generated +5. Check shop pages use responsive images +6. Run Lighthouse: + ```bash + npx lighthouse http://localhost:4000 --output=html --view + ``` +7. Verify Performance score improved + +**Commit:** `chore: finalize image optimization pipeline` + +--- + +## Verification Steps + +1. **Run migration:** + ```bash + mix ecto.migrate + ``` + +2. **Process existing mockups:** + ```bash + mix optimize_images + ``` + +3. **Upload a test image via admin UI** + - Should complete in ~1-2 seconds + - Check `priv/static/image_cache/` for generated files + +4. **Test startup recovery:** + ```bash + rm -rf priv/static/image_cache/ + mix phx.server + # Check logs for regeneration + ``` + +5. **Verify responsive images in browser:** + - DevTools → Network → Img + - Check that AVIF loads (Content-Type header) + - Resize to mobile, reload, verify smaller variants + +6. **Run tests:** + ```bash + mix test test/simpleshop_theme/images/ + mix test test/simpleshop_theme/workers/ + ``` + +7. **Re-run Lighthouse:** + ```bash + npx lighthouse http://localhost:4000 --output=html --view + ``` + +--- + +## Testing Strategy + +### Principles +1. **Test behavior, not implementation** - focus on inputs/outputs +2. **DRY via shared fixtures** - one `image_fixture()` function +3. **Avoid redundancy** - don't test the same logic in multiple places +4. **Real library calls** - use actual Image library, minimal mocking + +### Test Files to Create + +``` +test/ +├── simpleshop_theme/ +│ ├── images/ +│ │ └── optimizer_test.exs # Core optimization logic +│ └── workers/ +│ └── image_variants_test.exs # Oban worker +├── simpleshop_theme_web/ +│ └── components/ +│ └── shop_components_test.exs # responsive_image component +├── mix/ +│ └── tasks/ +│ └── optimize_images_test.exs # Mix task tests +├── support/ +│ └── fixtures/ +│ └── image_fixtures.ex # Shared test helpers +└── fixtures/ + └── sample_1200x800.jpg # Test image file +``` + +### Test Implementation + +**`test/support/fixtures/image_fixtures.ex`:** +```elixir +defmodule SimpleshopTheme.ImageFixtures do + alias SimpleshopTheme.Repo + alias SimpleshopTheme.Media.Image + + @sample_jpeg File.read!("test/fixtures/sample_1200x800.jpg") + + def sample_jpeg, do: @sample_jpeg + + def image_fixture(attrs \\ %{}) do + {:ok, webp, w, h} = SimpleshopTheme.Images.Optimizer.to_lossless_webp(@sample_jpeg) + + defaults = %{ + image_type: "product", + filename: "test.jpg", + content_type: "image/webp", + file_size: byte_size(webp), + data: webp, + source_width: w, + source_height: h, + variants_status: :pending, + is_svg: false + } + + %Image{} + |> Image.changeset(Map.merge(defaults, attrs)) + |> Repo.insert!() + end + + def svg_fixture do + svg = ~s() + + %Image{} + |> Image.changeset(%{ + image_type: "logo", + filename: "logo.svg", + content_type: "image/svg+xml", + file_size: byte_size(svg), + data: svg, + is_svg: true, + svg_content: svg, + variants_status: :pending + }) + |> Repo.insert!() + end + + def cache_path(id, width, format) do + ext = if format == :jpg, do: "jpg", else: Atom.to_string(format) + Path.join(SimpleshopTheme.Images.Optimizer.cache_dir(), "#{id}-#{width}.#{ext}") + end + + def cleanup_cache do + cache_dir = SimpleshopTheme.Images.Optimizer.cache_dir() + if File.exists?(cache_dir), do: File.rm_rf!(cache_dir) + end +end +``` + +**`test/simpleshop_theme/images/optimizer_test.exs`:** +```elixir +defmodule SimpleshopTheme.Images.OptimizerTest do + use SimpleshopTheme.DataCase, async: false + + alias SimpleshopTheme.Images.Optimizer + import SimpleshopTheme.ImageFixtures + + setup do + cleanup_cache() + on_exit(&cleanup_cache/0) + :ok + end + + describe "applicable_widths/1" do + test "returns all widths for large source" do + assert [400, 800, 1200] = Optimizer.applicable_widths(1500) + end + + test "excludes widths larger than source" do + assert [400, 800] = Optimizer.applicable_widths(900) + assert [400] = Optimizer.applicable_widths(500) + end + + test "returns source width when smaller than minimum" do + assert [300] = Optimizer.applicable_widths(300) + assert [50] = Optimizer.applicable_widths(50) + end + end + + describe "to_lossless_webp/1" do + test "converts image and returns dimensions" do + {:ok, webp, width, height} = Optimizer.to_lossless_webp(sample_jpeg()) + + assert is_binary(webp) + assert width == 1200 + assert height == 800 + # WebP magic bytes + assert <<_::binary-size(4), "WEBP", _::binary>> = webp + end + + test "returns error for invalid data" do + assert {:error, _} = Optimizer.to_lossless_webp("not an image") + end + end + + describe "process_for_image/1" do + test "generates all variants for 1200px image" do + image = image_fixture(source_width: 1200) + + assert {:ok, [400, 800, 1200]} = Optimizer.process_for_image(image.id) + + # Verify files exist + for w <- [400, 800, 1200], fmt <- [:avif, :webp, :jpg] do + assert File.exists?(cache_path(image.id, w, fmt)), + "Missing #{w}.#{fmt}" + end + + # Thumbnail exists + assert File.exists?(cache_path(image.id, "thumb", :jpg)) + end + + test "generates only applicable widths for smaller image" do + # Create a smaller test image + image = image_fixture(source_width: 600) + + assert {:ok, [400]} = Optimizer.process_for_image(image.id) + + assert File.exists?(cache_path(image.id, 400, :avif)) + refute File.exists?(cache_path(image.id, 800, :avif)) + end + + test "skips SVG images" do + image = svg_fixture() + assert {:ok, :svg_skipped} = Optimizer.process_for_image(image.id) + end + + test "returns error for missing image" do + assert {:error, :not_found} = Optimizer.process_for_image(Ecto.UUID.generate()) + end + + test "is idempotent - skips existing files" do + image = image_fixture() + {:ok, _} = Optimizer.process_for_image(image.id) + + # Get file modification time + path = cache_path(image.id, 400, :avif) + {:ok, %{mtime: mtime1}} = File.stat(path) + + # Process again + Process.sleep(1000) # Ensure different mtime if regenerated + {:ok, _} = Optimizer.process_for_image(image.id) + + {:ok, %{mtime: mtime2}} = File.stat(path) + assert mtime1 == mtime2, "File was regenerated" + end + end + + describe "disk_variants_exist?/2" do + test "returns true when all variants exist" do + image = image_fixture(source_width: 1200) + {:ok, _} = Optimizer.process_for_image(image.id) + + assert Optimizer.disk_variants_exist?(image.id, 1200) + end + + test "returns false when variants missing" do + image = image_fixture() + refute Optimizer.disk_variants_exist?(image.id, 1200) + end + end +end +``` + +**`test/simpleshop_theme/workers/image_variants_test.exs`:** +```elixir +defmodule SimpleshopTheme.Workers.ImageVariantsTest do + use SimpleshopTheme.DataCase, async: false + use Oban.Testing, repo: SimpleshopTheme.Repo + + alias SimpleshopTheme.Workers.ImageVariants + alias SimpleshopTheme.Media.Image + import SimpleshopTheme.ImageFixtures + + setup do + cleanup_cache() + on_exit(&cleanup_cache/0) + :ok + end + + test "processes image and marks complete" do + image = image_fixture(variants_status: :pending) + + assert :ok = perform_job(ImageVariants, %{image_id: image.id}) + + updated = Repo.get!(Image, image.id) + assert updated.variants_status == :complete + end + + test "handles missing image gracefully" do + assert {:error, :not_found} = + perform_job(ImageVariants, %{image_id: Ecto.UUID.generate()}) + end + + test "skips SVG and marks complete" do + image = svg_fixture() + + assert :ok = perform_job(ImageVariants, %{image_id: image.id}) + + updated = Repo.get!(Image, image.id) + assert updated.variants_status == :complete + end +end +``` + +**`test/simpleshop_theme_web/components/shop_components_test.exs`:** +```elixir +defmodule SimpleshopThemeWeb.ShopComponentsTest do + use SimpleshopThemeWeb.ConnCase, async: true + + import Phoenix.LiveViewTest + alias SimpleshopThemeWeb.ShopComponents + + describe "responsive_image/1" do + test "builds srcset with all widths for 1200px source" do + html = + render_component(&ShopComponents.responsive_image/1, %{ + src: "/images/test", + source_width: 1200, + alt: "Test image" + }) + + # AVIF source + assert html =~ ~s(type="image/avif") + assert html =~ "/images/test-400.avif 400w" + assert html =~ "/images/test-800.avif 800w" + assert html =~ "/images/test-1200.avif 1200w" + + # WebP source + assert html =~ ~s(type="image/webp") + + # Fallback img + assert html =~ ~s(src="/images/test-1200.jpg") + assert html =~ ~s(alt="Test image") + end + + test "excludes widths larger than source" do + html = + render_component(&ShopComponents.responsive_image/1, %{ + src: "/images/test", + source_width: 600, + alt: "Small image" + }) + + assert html =~ "400w" + refute html =~ "800w" + refute html =~ "1200w" + end + + test "uses source width when smaller than 400" do + html = + render_component(&ShopComponents.responsive_image/1, %{ + src: "/images/tiny", + source_width: 200, + alt: "Tiny" + }) + + assert html =~ "200w" + refute html =~ "400w" + end + + test "sets lazy loading by default" do + html = + render_component(&ShopComponents.responsive_image/1, %{ + src: "/images/test", + source_width: 1200, + alt: "Test" + }) + + assert html =~ ~s(loading="lazy") + assert html =~ ~s(decoding="async") + refute html =~ ~s(fetchpriority) + end + + test "sets eager loading when priority=true" do + html = + render_component(&ShopComponents.responsive_image/1, %{ + src: "/images/test", + source_width: 1200, + alt: "Test", + priority: true + }) + + assert html =~ ~s(loading="eager") + assert html =~ ~s(decoding="sync") + assert html =~ ~s(fetchpriority="high") + end + + test "includes width/height when provided" do + html = + render_component(&ShopComponents.responsive_image/1, %{ + src: "/images/test", + source_width: 1200, + alt: "Test", + width: 600, + height: 400 + }) + + assert html =~ ~s(width="600") + assert html =~ ~s(height="400") + end + end +end +``` + +**`test/mix/tasks/optimize_images_test.exs`:** +```elixir +defmodule Mix.Tasks.OptimizeImagesTest do + use ExUnit.Case, async: false + + import ExUnit.CaptureIO + + @test_dir "test/fixtures/mockups_test" + + setup do + # Create test directory with a sample image + File.mkdir_p!(@test_dir) + + # Copy sample image to test directory + sample = File.read!("test/fixtures/sample_1200x800.jpg") + File.write!(Path.join(@test_dir, "test-product.jpg"), sample) + + on_exit(fn -> + File.rm_rf!(@test_dir) + end) + + :ok + end + + describe "run/1" do + test "generates variants for mockup images" do + output = capture_io(fn -> + Mix.Tasks.OptimizeImages.run(["--dir", @test_dir]) + end) + + assert output =~ "Found 1 original mockup images" + assert output =~ "test-product: generated" + assert output =~ "Done!" + + # Verify generated files + assert File.exists?(Path.join(@test_dir, "test-product-400.avif")) + assert File.exists?(Path.join(@test_dir, "test-product-400.webp")) + assert File.exists?(Path.join(@test_dir, "test-product-400.jpg")) + assert File.exists?(Path.join(@test_dir, "test-product-800.avif")) + assert File.exists?(Path.join(@test_dir, "test-product-800.webp")) + assert File.exists?(Path.join(@test_dir, "test-product-800.jpg")) + assert File.exists?(Path.join(@test_dir, "test-product-1200.avif")) + assert File.exists?(Path.join(@test_dir, "test-product-1200.webp")) + assert File.exists?(Path.join(@test_dir, "test-product-1200.jpg")) + end + + test "skips already generated variants" do + # First run + capture_io(fn -> Mix.Tasks.OptimizeImages.run(["--dir", @test_dir]) end) + + # Second run should skip + output = capture_io(fn -> + Mix.Tasks.OptimizeImages.run(["--dir", @test_dir]) + end) + + assert output =~ "Found 1 original mockup images" + refute output =~ "generated" # Should skip, not generate + end + + test "regenerates with --force flag" do + # First run + capture_io(fn -> Mix.Tasks.OptimizeImages.run(["--dir", @test_dir]) end) + + # Get mtime of generated file + path = Path.join(@test_dir, "test-product-400.avif") + {:ok, %{mtime: mtime1}} = File.stat(path) + + Process.sleep(1000) + + # Force regeneration + output = capture_io(fn -> + Mix.Tasks.OptimizeImages.run(["--dir", @test_dir, "--force"]) + end) + + assert output =~ "generated" + + {:ok, %{mtime: mtime2}} = File.stat(path) + assert mtime2 > mtime1, "File should have been regenerated" + end + + test "excludes variant files from processing" do + # Create a file that looks like a variant (should be ignored) + variant_path = Path.join(@test_dir, "test-product-400.jpg") + File.write!(variant_path, "fake variant") + + output = capture_io(fn -> + Mix.Tasks.OptimizeImages.run(["--dir", @test_dir]) + end) + + # Should only find the original, not the fake variant + assert output =~ "Found 1 original mockup images" + end + + test "handles empty directory" do + empty_dir = Path.join(@test_dir, "empty") + File.mkdir_p!(empty_dir) + + output = capture_io(fn -> + Mix.Tasks.OptimizeImages.run(["--dir", empty_dir]) + end) + + assert output =~ "Found 0 original mockup images" + assert output =~ "Done!" + end + end +end +``` + +### What's NOT Tested (Intentionally) + +- **VariantCache GenServer** - Startup behavior is tested via integration/manual testing +- **ImageController thumbnail** - Covered by existing controller tests or integration + +### Test Coverage Goals + +| Module | Coverage Goal | Notes | +|--------|--------------|-------| +| Optimizer | 90%+ | Core logic, all edge cases | +| ImageVariants worker | 80%+ | Happy path + error handling | +| responsive_image component | 90%+ | All attribute combinations | +| Mix.Tasks.OptimizeImages | 80%+ | Variant generation, skip logic, force flag | +| VariantCache | Manual | Startup behavior |