add canonical URLs, robots.txt, and sitemap.xml
Canonical: all shop pages now assign og_url (reusing the existing og:url assign), which the layout renders as <link rel="canonical">. Collection pages strip the sort param so ?sort=price_asc doesn't create a duplicate canonical. robots.txt: dynamic controller disallows /admin/, /api/, /users/, /webhooks/, /checkout/. Removed robots.txt from static_paths so it goes through the router instead of Plug.Static. sitemap.xml: auto-generated from all visible products + categories + static pages, served as application/xml. 8 tests. Also updates PROGRESS.md: marks tasks 55, 58, 59, 61, 62 as done. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
jamey
250
docs/plans/abandoned-cart.md
Normal file
250
docs/plans/abandoned-cart.md
Normal file
@@ -0,0 +1,250 @@
|
||||
# Abandoned cart recovery
|
||||
|
||||
> Status: Planned
|
||||
> Tasks: #75–77 in PROGRESS.md
|
||||
> Tier: 4 (Growth & content)
|
||||
|
||||
## Goal
|
||||
|
||||
Recover genuinely lost sales from customers who started checkout but didn't complete it — without invading their privacy, spamming them, or relying on dark patterns.
|
||||
|
||||
## The privacy line
|
||||
|
||||
Most abandoned cart implementations are invasive:
|
||||
- Track anonymous visitors via cookies before any consent
|
||||
- Email anyone who added to cart, regardless of engagement
|
||||
- Send multiple escalating follow-ups ("Last chance!", "We miss you!")
|
||||
- Use tracking pixels to know if the email was opened
|
||||
- Hold the data indefinitely
|
||||
|
||||
**Berrypod's approach is narrower and more honest.** The key distinction is *consent signal*: we only contact customers who explicitly reached the Stripe Checkout page and entered their email. That's a strong signal of genuine purchase intent — they were mid-payment, not just browsing.
|
||||
|
||||
Anonymous cart sessions with no email are never touched. We don't know who they are and we don't try to find out.
|
||||
|
||||
## Legal basis
|
||||
|
||||
### UK — PECR soft opt-in (Regulation 22)
|
||||
|
||||
The strongest legal ground. PECR allows sending unsolicited marketing email without prior explicit consent if:
|
||||
|
||||
1. You obtained the email address **in the course of a sale or negotiation of a sale** — the ICO has confirmed this covers incomplete checkouts where the customer started but didn't finish
|
||||
2. You're marketing your **own similar products/services** — same shop, same type of products they were trying to buy
|
||||
3. The customer was given a **clear opportunity to opt out** at the time their details were collected, and in every subsequent message
|
||||
4. They have **not opted out**
|
||||
|
||||
A single abandoned cart reminder fits squarely. The ICO has specifically addressed abandoned cart emails in its guidance on PECR.
|
||||
|
||||
### EU — Legitimate interests (GDPR Article 6(1)(f))
|
||||
|
||||
For EU customers (or UK shops with EU-based customers post-Brexit), the soft opt-in rule doesn't apply directly — it's a UK PECR provision. The equivalent ePrivacy Directive is implemented differently across member states.
|
||||
|
||||
The lawful basis would be **legitimate interests**, provided:
|
||||
|
||||
- The email is clearly transactional-adjacent (not general marketing — it's specifically about the items they tried to buy)
|
||||
- A **Legitimate Interests Assessment (LIA)** is documented: the shop owner's interest in recovering a genuine purchase is weighed against the customer's right to privacy. A single non-pushy reminder, easy to opt out of, with prompt data deletion, tips the balance in favour of legitimate interests.
|
||||
- Proper opt-out, no tracking, data deletion
|
||||
|
||||
The shop owner should document this LIA in their privacy policy.
|
||||
|
||||
### What makes it non-compliant (and what we avoid)
|
||||
|
||||
- Multiple emails / drip sequences
|
||||
- Tracking pixels to know if the email was opened or the link clicked
|
||||
- Storing the abandoned cart data indefinitely
|
||||
- No real unsubscribe mechanism
|
||||
- Emailing anonymous cart sessions (no email captured)
|
||||
- No mention in privacy policy
|
||||
|
||||
## How it works
|
||||
|
||||
### The trigger: `checkout.session.expired`
|
||||
|
||||
Stripe fires this webhook when a Checkout session expires without payment. Sessions expire after 24 hours by default. The expired session object contains:
|
||||
|
||||
- `customer_details.email` — if the customer entered their email on the Stripe page (may be absent if they abandoned before that step)
|
||||
- `line_items` — what was in their cart
|
||||
- `amount_total` — cart total at time of checkout
|
||||
- `expires_at` — when it expired
|
||||
|
||||
We already handle `checkout.session.completed` in `stripe_webhook_controller.ex`. This is a sibling handler.
|
||||
|
||||
### Flow
|
||||
|
||||
```
|
||||
checkout.session.expired webhook fires
|
||||
↓
|
||||
Extract customer_details.email from session
|
||||
↓
|
||||
If no email → discard (customer never identified themselves)
|
||||
↓
|
||||
Check suppression list → if unsubscribed → discard
|
||||
↓
|
||||
Check for existing paid order with same stripe_session_id → if found → discard
|
||||
(handles race: session expired but payment webhook fired first)
|
||||
↓
|
||||
Store AbandonedCart record:
|
||||
- customer_email
|
||||
- line_items (JSON snapshot)
|
||||
- cart_total
|
||||
- stripe_session_id
|
||||
- expired_at
|
||||
↓
|
||||
Enqueue AbandonedCartEmailWorker (Oban, delay ~1 hour)
|
||||
↓
|
||||
Worker sends single plain-text email
|
||||
↓
|
||||
Mark AbandonedCart as emailed (emailed_at timestamp)
|
||||
```
|
||||
|
||||
### Schema
|
||||
|
||||
```elixir
|
||||
create table(:abandoned_carts, primary_key: false) do
|
||||
add :id, :binary_id, primary_key: true
|
||||
add :customer_email, :string, null: false
|
||||
add :stripe_session_id, :string, null: false
|
||||
add :line_items, :map, null: false # JSON snapshot of cart contents
|
||||
add :cart_total, :integer # in minor units
|
||||
add :expired_at, :utc_datetime, null: false
|
||||
add :emailed_at, :utc_datetime # nil until email sent
|
||||
timestamps()
|
||||
end
|
||||
|
||||
create unique_index(:abandoned_carts, [:stripe_session_id])
|
||||
create index(:abandoned_carts, [:customer_email])
|
||||
create index(:abandoned_carts, [:inserted_at]) # for pruning
|
||||
```
|
||||
|
||||
```elixir
|
||||
create table(:email_suppressions, primary_key: false) do
|
||||
add :id, :binary_id, primary_key: true
|
||||
add :email, :string, null: false
|
||||
add :reason, :string # "unsubscribed", "bounced", etc.
|
||||
add :suppressed_at, :utc_datetime, null: false
|
||||
timestamps()
|
||||
end
|
||||
|
||||
create unique_index(:email_suppressions, [:email])
|
||||
```
|
||||
|
||||
### The email
|
||||
|
||||
Plain text. No tracking pixels. No HTML. Consistent with the existing order confirmation and shipping notification emails.
|
||||
|
||||
```
|
||||
Subject: You left something behind
|
||||
|
||||
Hi,
|
||||
|
||||
You recently started a checkout at [shop name] but didn't complete it.
|
||||
|
||||
Your cart had:
|
||||
- Classic Tee (M, Black) × 1 — £25.00
|
||||
- Tote Bag × 2 — £18.00
|
||||
|
||||
Total: £43.00
|
||||
|
||||
If you'd like to complete your order, head to [shop URL] and add these items again.
|
||||
|
||||
We're only sending this once.
|
||||
|
||||
—
|
||||
[shop name]
|
||||
|
||||
Don't want to hear from us? Unsubscribe: [unsubscribe link]
|
||||
```
|
||||
|
||||
Key points:
|
||||
- "We're only sending this once" — sets expectations, builds trust
|
||||
- No urgency language ("Act now!", "Limited stock!")
|
||||
- No tracking link for the cart — just link to the shop homepage. Cart is session-based and can't be restored via a link anyway. Just show what they had.
|
||||
- Plain text, no images, no tracking pixels
|
||||
- Unsubscribe link at the bottom — signed token, one click
|
||||
|
||||
### Unsubscribe
|
||||
|
||||
Unsubscribe link is a signed token: `Phoenix.Token.sign(endpoint, "email-unsub", email)`.
|
||||
|
||||
`/unsubscribe/:token` route:
|
||||
- Verify the token
|
||||
- Insert into `email_suppressions`
|
||||
- Show a simple "You've been unsubscribed" page
|
||||
- Suppression is checked before every email send (recovery, newsletter, order updates — all of them)
|
||||
|
||||
### 30-day data pruning
|
||||
|
||||
New Oban cron job: `AbandonedCartPruneWorker` runs nightly.
|
||||
|
||||
```elixir
|
||||
def perform(_job) do
|
||||
cutoff = DateTime.add(DateTime.utc_now(), -30, :day)
|
||||
Repo.delete_all(from a in AbandonedCart, where: a.inserted_at < ^cutoff)
|
||||
:ok
|
||||
end
|
||||
```
|
||||
|
||||
After 30 days, the abandoned cart record is gone. If someone comes back and buys after 30 days, that's fine — it's a fresh order, not connected to the old abandoned cart.
|
||||
|
||||
### Stripe Checkout footer notice
|
||||
|
||||
Stripe allows custom text on the Checkout page footer via the `custom_text` parameter on the Checkout session. Add a brief note:
|
||||
|
||||
```elixir
|
||||
custom_text: %{
|
||||
after_submit: %{
|
||||
message: "If your payment doesn't complete, we may send you one follow-up email. You can unsubscribe at any time."
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
This is the collection-time notice required for PECR soft opt-in, and satisfies the GDPR transparency requirement for legitimate interests.
|
||||
|
||||
### Privacy policy
|
||||
|
||||
The shop's privacy policy template (currently a static page) should include a paragraph covering:
|
||||
- What data is collected during incomplete checkout (email, cart contents)
|
||||
- That a single recovery email may be sent
|
||||
- How to unsubscribe
|
||||
- That data is deleted after 30 days
|
||||
|
||||
This is already in the template pages that come with Berrypod — the privacy page just needs this section added.
|
||||
|
||||
## What it deliberately can't do
|
||||
|
||||
- **Track anonymous sessions** — our cart is session-based. If no email was captured, we have no identity to contact. This is a feature, not a limitation.
|
||||
- **Send multiple emails** — the schema has a single `emailed_at` field. The worker checks it before sending and won't send again.
|
||||
- **Track email opens or link clicks** — plain text emails, no tracking pixels, no redirected links
|
||||
- **Target general browsers** — only customers who got as far as entering their email on Stripe. Someone who viewed products and added to cart but never clicked Checkout is never contacted.
|
||||
|
||||
## Admin dashboard
|
||||
|
||||
A small card on the admin dashboard (or `/admin/orders`):
|
||||
|
||||
- Abandoned carts in last 30 days: X
|
||||
- Emails sent: Y
|
||||
- Estimated recovery rate (orders placed within 48h of an abandoned cart from the same email): Z%
|
||||
|
||||
This gives the shop owner visibility without being creepy about it. The metric is useful for understanding checkout friction — a high abandonment rate at Stripe suggests a pricing, trust, or UX issue.
|
||||
|
||||
## Files to create/modify
|
||||
|
||||
- Migration — `abandoned_carts` and `email_suppressions` tables
|
||||
- `lib/berrypod/orders/abandoned_cart.ex` — schema
|
||||
- `lib/berrypod/orders/email_suppression.ex` — schema
|
||||
- `lib/berrypod/orders.ex` — `create_abandoned_cart/1`, `check_suppression/1`
|
||||
- `lib/berrypod/workers/abandoned_cart_email_worker.ex` — Oban job
|
||||
- `lib/berrypod/workers/abandoned_cart_prune_worker.ex` — Oban cron
|
||||
- `lib/berrypod/notifier/order_notifier.ex` — `send_cart_recovery/1`
|
||||
- `lib/berrypod_web/controllers/stripe_webhook_controller.ex` — handle `checkout.session.expired`
|
||||
- `lib/berrypod_web/controllers/checkout_controller.ex` — add `custom_text` to session
|
||||
- `lib/berrypod_web/controllers/unsubscribe_controller.ex` — new, handles `/unsubscribe/:token`
|
||||
- Router — `/unsubscribe/:token` route
|
||||
- Config — add `AbandonedCartPruneWorker` to Oban crontab
|
||||
|
||||
## Tests
|
||||
|
||||
- Webhook handler: email present → creates record, no email → discards, suppressed → discards, paid order exists → discards
|
||||
- Email worker: sends email, marks as emailed, doesn't send twice
|
||||
- Prune worker: deletes records older than 30 days, keeps recent ones
|
||||
- Unsubscribe: valid token → suppressed, invalid token → error, already suppressed → idempotent
|
||||
- Suppression check: suppressed email → blocked from all sends
|
||||
259
docs/plans/activity-log.md
Normal file
259
docs/plans/activity-log.md
Normal file
@@ -0,0 +1,259 @@
|
||||
# Activity log & order timeline
|
||||
|
||||
> Status: Planned
|
||||
> Tasks: #89–92 in PROGRESS.md
|
||||
> Tier: 3.5 (Business tools)
|
||||
|
||||
## Goal
|
||||
|
||||
One table. Two views.
|
||||
|
||||
**Order timeline** — on the order detail page, replace the static field cards with a chronological feed showing everything that happened to that order: payment confirmed, email sent, submitted to provider, in production, shipped (with tracking), delivery confirmed, any errors and retries. The shop owner can see the complete lifecycle without digging through separate panels.
|
||||
|
||||
**Global activity feed** — `/admin/activity` — a reverse-chronological stream of all meaningful events across the system: orders, syncs, emails, abandoned carts. Filterable. Errors and warnings bubble up as "needs attention" with a count badge on the admin nav.
|
||||
|
||||
Same data source for both. No separate "notifications" table. The activity log is the notifications.
|
||||
|
||||
---
|
||||
|
||||
## Schema
|
||||
|
||||
```elixir
|
||||
create table(:activity_log, primary_key: false) do
|
||||
add :id, :binary_id, primary_key: true
|
||||
add :event_type, :string, null: false # "order.shipped", "email.confirmation_sent", etc.
|
||||
add :level, :string, null: false # "info", "warning", "error"
|
||||
add :order_id, references(:orders, type: :binary_id, on_delete: :nilify_all)
|
||||
add :payload, :map # JSON snapshot — what was relevant at the time
|
||||
add :message, :string, null: false # human-readable, shown in the UI
|
||||
add :resolved_at, :utc_datetime # nil until acknowledged (for "needs attention")
|
||||
add :occurred_at, :utc_datetime, null: false
|
||||
timestamps()
|
||||
end
|
||||
|
||||
create index(:activity_log, [:order_id])
|
||||
create index(:activity_log, [:occurred_at])
|
||||
create index(:activity_log, [:level, :resolved_at]) # for "needs attention" query
|
||||
```
|
||||
|
||||
### Why `occurred_at` separate from `inserted_at`
|
||||
|
||||
For most events they'll be the same. But if a webhook arrives late (Stripe retries, provider delays) the `occurred_at` reflects when the event actually happened (from the webhook timestamp), while `inserted_at` is when we recorded it. The timeline sorts by `occurred_at`.
|
||||
|
||||
---
|
||||
|
||||
## Event taxonomy
|
||||
|
||||
### Order events (all carry `order_id`)
|
||||
|
||||
| Event type | Level | When fired | Message template |
|
||||
|------------|-------|------------|-----------------|
|
||||
| `order.created` | info | Stripe `checkout.session.completed` webhook | "Order placed — £43.00 via Stripe" |
|
||||
| `order.email.confirmation_sent` | info | After confirmation email sends | "Order confirmation sent to customer@example.com" |
|
||||
| `order.submitted` | info | After successful provider submission | "Submitted to Printify (PFY-12345)" |
|
||||
| `order.submission_failed` | error | On provider submission error | "Submission to Printify failed: insufficient stock for M Black" |
|
||||
| `order.submission_retried` | info | On retry after failure | "Submission retried — attempt 2 of 3" |
|
||||
| `order.production` | info | Provider webhook: in production | "In production at Printify" |
|
||||
| `order.shipped` | info | Provider webhook: shipped | "Shipped via DHL — tracking: 1Z999…" |
|
||||
| `order.email.shipping_sent` | info | After shipping email sends | "Shipping notification sent to customer@example.com" |
|
||||
| `order.delivered` | info | Provider webhook: delivered | "Delivered" |
|
||||
| `order.cancelled` | warning | On cancellation | "Order cancelled" |
|
||||
| `order.refunded` | info | On refund | "Refunded £43.00" |
|
||||
|
||||
### System events (no `order_id`)
|
||||
|
||||
| Event type | Level | When fired | Message template |
|
||||
|------------|-------|------------|-----------------|
|
||||
| `sync.started` | info | ProductSyncWorker begins | "Product sync started (Printify)" |
|
||||
| `sync.completed` | info | Sync finishes successfully | "Product sync complete — 47 products, 3 updated" |
|
||||
| `sync.failed` | error | Sync throws an error | "Product sync failed: API rate limit exceeded" |
|
||||
| `abandoned_cart.created` | info | Abandoned cart recorded | "Abandoned cart — customer@example.com, £43.00" |
|
||||
| `abandoned_cart.email_sent` | info | Recovery email sent | "Recovery email sent to customer@example.com" |
|
||||
| `abandoned_cart.recovered` | info | Customer buys after recovery | "Cart recovered — order SS-260223-0042 placed" |
|
||||
| `email.bounced` | warning | Email delivery failure reported | "Email bounced: customer@example.com (order confirmation)" |
|
||||
|
||||
---
|
||||
|
||||
## ActivityLog context
|
||||
|
||||
```elixir
|
||||
defmodule Berrypod.ActivityLog do
|
||||
def log_event(event_type, message, opts \\ []) do
|
||||
%ActivityLog{}
|
||||
|> ActivityLog.changeset(%{
|
||||
event_type: event_type,
|
||||
level: opts[:level] || "info",
|
||||
order_id: opts[:order_id],
|
||||
payload: opts[:payload] || %{},
|
||||
message: message,
|
||||
occurred_at: opts[:occurred_at] || DateTime.utc_now()
|
||||
})
|
||||
|> Repo.insert()
|
||||
# Failures are silent — never crash a business-critical path for a log entry
|
||||
|> case do
|
||||
{:ok, entry} -> {:ok, entry}
|
||||
{:error, _} -> :ok
|
||||
end
|
||||
end
|
||||
|
||||
def list_for_order(order_id) do
|
||||
from(a in ActivityLog,
|
||||
where: a.order_id == ^order_id,
|
||||
order_by: [asc: a.occurred_at]
|
||||
) |> Repo.all()
|
||||
end
|
||||
|
||||
def list_recent(opts \\ []) do
|
||||
limit = opts[:limit] || 50
|
||||
level = opts[:level]
|
||||
|
||||
query = from(a in ActivityLog, order_by: [desc: a.occurred_at], limit: ^limit)
|
||||
query = if level, do: where(query, [a], a.level == ^level), else: query
|
||||
Repo.all(query)
|
||||
end
|
||||
|
||||
def count_needing_attention do
|
||||
from(a in ActivityLog,
|
||||
where: a.level in ["warning", "error"] and is_nil(a.resolved_at)
|
||||
) |> Repo.aggregate(:count)
|
||||
end
|
||||
|
||||
def resolve(id) do
|
||||
from(a in ActivityLog, where: a.id == ^id)
|
||||
|> Repo.update_all(set: [resolved_at: DateTime.utc_now()])
|
||||
end
|
||||
|
||||
def resolve_all_for_order(order_id) do
|
||||
from(a in ActivityLog,
|
||||
where: a.order_id == ^order_id and is_nil(a.resolved_at)
|
||||
) |> Repo.update_all(set: [resolved_at: DateTime.utc_now()])
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
**Key principle:** `log_event/3` never raises. If the DB insert fails for any reason, the calling code carries on. A business-critical path (processing a Stripe webhook, submitting an order) must not fail because the activity log had an error.
|
||||
|
||||
---
|
||||
|
||||
## Instrumentation points
|
||||
|
||||
Where to add `ActivityLog.log_event/3` calls in the existing codebase:
|
||||
|
||||
| File | Event(s) | Notes |
|
||||
|------|---------|-------|
|
||||
| `stripe_webhook_controller.ex` | `order.created` | Already has the order and amount from the webhook |
|
||||
| `notifier/order_notifier.ex` | `order.email.confirmation_sent`, `order.email.shipping_sent` | After successful send |
|
||||
| `workers/order_submission_worker.ex` | `order.submitted`, `order.submission_failed`, `order.submission_retried` | On perform success/failure, retry attempt count from job |
|
||||
| `workers/fulfilment_status_worker.ex` (or provider webhooks) | `order.production`, `order.shipped`, `order.delivered`, `order.cancelled` | On status change only (guard: don't log if status didn't change) |
|
||||
| `workers/product_sync_worker.ex` | `sync.started`, `sync.completed`, `sync.failed` | Include counts in payload |
|
||||
| `workers/abandoned_cart_email_worker.ex` (planned) | `abandoned_cart.email_sent`, `abandoned_cart.recovered` | When built |
|
||||
|
||||
---
|
||||
|
||||
## Order timeline UI
|
||||
|
||||
Below the line items table on `/admin/orders/:id`, a new "Activity" card:
|
||||
|
||||
```
|
||||
● 14:23 Order placed — £43.00 via Stripe
|
||||
● 14:24 Order confirmation sent to customer@example.com
|
||||
● 14:25 Submitted to Printify (PFY-78234)
|
||||
● 14:26 In production at Printify
|
||||
● 16 Feb Shipped via DHL — tracking: 1Z999AA10123456784
|
||||
● 16 Feb Shipping notification sent to customer@example.com
|
||||
● 18 Feb Delivered
|
||||
```
|
||||
|
||||
For an order with a problem:
|
||||
|
||||
```
|
||||
● 10:00 Order placed — £28.00 via Stripe
|
||||
● 10:01 Order confirmation sent to customer@example.com
|
||||
⚠ 10:02 Submission to Printify failed: variant out of stock
|
||||
● 10:07 Retrying — attempt 2 of 3
|
||||
● 10:07 Submitted to Printify (PFY-78299)
|
||||
● 10:08 In production at Printify
|
||||
```
|
||||
|
||||
Each entry shows a dot (●) or warning icon (⚠) and the timestamp + message. No separate fields to cross-reference. The whole story is here.
|
||||
|
||||
The timeline is loaded once on mount (no streaming needed — order events are append-only and infrequent). A `phx-click="refresh"` button for orders that are still in progress.
|
||||
|
||||
---
|
||||
|
||||
## Global activity feed — `/admin/activity`
|
||||
|
||||
Reverse-chronological list of all events. Two tabs:
|
||||
|
||||
- **All activity** — everything, newest first, paginated (50 per page)
|
||||
- **Needs attention** — errors and warnings where `resolved_at` is nil
|
||||
|
||||
Each row:
|
||||
- Icon (coloured by level — green info, amber warning, red error)
|
||||
- Event type label
|
||||
- Message
|
||||
- Relative time ("2 hours ago")
|
||||
- Link to related order if `order_id` present
|
||||
- "Resolve" button for warnings/errors
|
||||
|
||||
**"Needs attention" badge on admin nav:**
|
||||
|
||||
A small count badge on the Activity nav item when `count_needing_attention/0 > 0`. Polled on a `Process.send_after` timer (every 60 seconds) — not a PubSub subscription (no need for real-time here).
|
||||
|
||||
```
|
||||
[⚑ Activity 3]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 90-day pruning
|
||||
|
||||
Oban cron job, runs nightly:
|
||||
|
||||
```elixir
|
||||
def perform(_job) do
|
||||
cutoff = DateTime.add(DateTime.utc_now(), -90, :day)
|
||||
Repo.delete_all(from a in ActivityLog, where: a.inserted_at < ^cutoff)
|
||||
:ok
|
||||
end
|
||||
```
|
||||
|
||||
90 days is plenty for business purposes. Resolved errors and successful info events don't need to be kept indefinitely.
|
||||
|
||||
---
|
||||
|
||||
## What this replaces / improves
|
||||
|
||||
Currently the order detail has:
|
||||
- A "Fulfilment" card with scattered timestamps (`submitted_at`, `shipped_at`, `delivered_at`) as key/value pairs
|
||||
- A `fulfilment_error` field that overwrites itself (only the last error is visible)
|
||||
|
||||
After this:
|
||||
- All events are preserved — if a submission failed and was retried, you see both entries
|
||||
- Errors are never lost — `fulfilment_error` on the order is still kept for quick status, but the full history is in the log
|
||||
- Emails are visible — currently you have no way to know if the confirmation email actually sent successfully
|
||||
|
||||
---
|
||||
|
||||
## Files to create/modify
|
||||
|
||||
- Migration — `activity_log` table
|
||||
- `lib/berrypod/activity_log.ex` — schema
|
||||
- `lib/berrypod/activity_log/` or `lib/berrypod/activity_log.ex` — context functions
|
||||
- `lib/berrypod_web/live/admin/activity.ex` — new LiveView for global feed
|
||||
- `lib/berrypod_web/live/admin/order_show.ex` — load and render order timeline
|
||||
- Admin nav — add Activity link with "needs attention" badge
|
||||
- Router — `/admin/activity` route
|
||||
- Instrumentation in: `stripe_webhook_controller.ex`, `notifier/order_notifier.ex`, `workers/order_submission_worker.ex`, `workers/fulfilment_status_worker.ex`, `workers/product_sync_worker.ex`
|
||||
- Oban crontab config — add `ActivityLogPruneWorker`
|
||||
|
||||
---
|
||||
|
||||
## Tasks
|
||||
|
||||
| # | Task | Est |
|
||||
|---|------|-----|
|
||||
| 89 | `activity_log` schema + migration + `ActivityLog` context (`log_event/3`, `list_for_order/1`, `list_recent/1`, `count_needing_attention/0`, `resolve/1`) | 1.5h |
|
||||
| 90 | Instrument existing event points — stripe webhook, OrderNotifier, OrderSubmissionWorker, fulfilment status, ProductSyncWorker | 1.5h |
|
||||
| 91 | Order timeline component + add to order detail page (`/admin/orders/:id`) | 1.5h |
|
||||
| 92 | Global `/admin/activity` LiveView — all activity feed, "needs attention" tab, "resolve" action, count badge on admin nav | 2h |
|
||||
188
docs/plans/favicon.md
Normal file
188
docs/plans/favicon.md
Normal file
@@ -0,0 +1,188 @@
|
||||
# Favicon & site icon management
|
||||
|
||||
> Status: Planned
|
||||
> Tasks: #86–88 in PROGRESS.md
|
||||
> Tier: 3 (Compliance & quality) — ships before page editor
|
||||
|
||||
## Goal
|
||||
|
||||
Upload one source image, get a complete, best-practice favicon setup generated automatically. No manual resizing, no wrestling with `.ico` files, no outdated `<link>` tags. Works like the best favicon generators online — sensible defaults, small amount of customisation, nothing more.
|
||||
|
||||
## Logo vs favicon
|
||||
|
||||
Often the same thing, often not. A wide wordmark logo (`Robin's Nature Shop` with a leaf motif) is completely unreadable at 16×16. A favicon wants to be a tight square mark — just the symbol, no text.
|
||||
|
||||
**Default behaviour:** use the existing logo upload as the favicon source (toggled on). The toggle lives in the theme editor, next to the logo upload section. If the shop owner hasn't uploaded a logo yet, the favicon falls back to a generic Berrypod icon.
|
||||
|
||||
**Override:** if the logo isn't suitable, upload a separate icon image (PNG or SVG, ideally 512×512 or larger). This becomes the favicon source instead. Stored as `image_type: "icon"` in the existing `images` table — same pattern as `"logo"` and `"header"`.
|
||||
|
||||
## What gets generated
|
||||
|
||||
From a single source image, an Oban worker generates and stores everything:
|
||||
|
||||
| Output | Size | Purpose |
|
||||
|--------|------|---------|
|
||||
| `favicon.svg` | Source SVG | Modern browsers (Chrome 80+, Firefox 41+, Safari 13.1+) — if source is SVG |
|
||||
| `favicon-32x32.png` | 32×32 | PNG fallback for browsers without SVG favicon support |
|
||||
| `apple-touch-icon.png` | 180×180 | iOS "Add to Home Screen" |
|
||||
| `icon-192.png` | 192×192 | Android / PWA manifest |
|
||||
| `icon-512.png` | 512×512 | PWA splash screen |
|
||||
|
||||
**`.ico` file:** `libvips` (used by the `image` library) doesn't output ICO natively. ICO is only needed for very old browsers and some Windows desktop edge cases (taskbar pinning). A pre-baked generic `favicon.ico` ships in `priv/static/` as a passive fallback — browsers request it at startup without a `<link>` tag, so it just needs to exist. Not worth generating dynamically.
|
||||
|
||||
**SVG dark mode:** if the source is SVG, the served `favicon.svg` gets a `prefers-color-scheme: dark` media query injected wrapping the fill colour. The icon adapts automatically to browser/OS dark mode — something no raster format can do.
|
||||
|
||||
```svg
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 32 32">
|
||||
<style>
|
||||
.icon-mark { fill: #1a1a1a; }
|
||||
@media (prefers-color-scheme: dark) {
|
||||
.icon-mark { fill: #ffffff; }
|
||||
}
|
||||
</style>
|
||||
<!-- shop's own SVG content -->
|
||||
</svg>
|
||||
```
|
||||
|
||||
If the source is not an SVG, only the PNG variants are generated — no SVG favicon served.
|
||||
|
||||
## Generation pipeline
|
||||
|
||||
Follows the existing image optimization pattern: upload triggers an Oban job, job generates variants and stores them in the DB.
|
||||
|
||||
```elixir
|
||||
defmodule Berrypod.Workers.FaviconGeneratorWorker do
|
||||
use Oban.Worker, queue: :media
|
||||
|
||||
def perform(%Job{args: %{"source_image_id" => id}}) do
|
||||
image = Media.get_image!(id)
|
||||
|
||||
with {:ok, resized_32} <- resize(image, 32),
|
||||
{:ok, resized_180} <- resize(image, 180),
|
||||
{:ok, resized_192} <- resize(image, 192),
|
||||
{:ok, resized_512} <- resize(image, 512) do
|
||||
Media.store_favicon_variants(%{
|
||||
source_image_id: id,
|
||||
png_32: resized_32,
|
||||
png_180: resized_180,
|
||||
png_192: resized_192,
|
||||
png_512: resized_512,
|
||||
svg: (if image.is_svg, do: inject_dark_mode(image.svg_content))
|
||||
})
|
||||
end
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
## Storage schema
|
||||
|
||||
New table `favicon_variants` — one row, updated on each regeneration:
|
||||
|
||||
```elixir
|
||||
create table(:favicon_variants, primary_key: false) do
|
||||
add :id, :binary_id, primary_key: true
|
||||
add :source_image_id, references(:images, type: :binary_id)
|
||||
add :png_32, :binary
|
||||
add :png_180, :binary
|
||||
add :png_192, :binary
|
||||
add :png_512, :binary
|
||||
add :svg, :text # dark-mode-injected SVG content; nil if source not SVG
|
||||
add :generated_at, :utc_datetime
|
||||
timestamps()
|
||||
end
|
||||
```
|
||||
|
||||
Single-row pattern (enforced by application logic) — only one favicon variant set at a time. On regeneration, the row is upserted.
|
||||
|
||||
## Serving
|
||||
|
||||
Generated files can't go in `priv/static/` (ephemeral filesystem in Docker/Fly.io). Instead, served via a `FaviconController` at the expected root paths:
|
||||
|
||||
```
|
||||
GET /favicon.svg → serves svg column (if available)
|
||||
GET /favicon-32x32.png → serves png_32 column
|
||||
GET /apple-touch-icon.png → serves png_180 column
|
||||
GET /icon-192.png → serves png_192 column
|
||||
GET /icon-512.png → serves png_512 column
|
||||
GET /site.webmanifest → dynamically generated JSON from settings
|
||||
GET /favicon.ico → static file from priv/static/ (pre-baked fallback)
|
||||
```
|
||||
|
||||
All image responses include `Cache-Control: public, max-age=86400` (1 day) and an `ETag` based on `generated_at`. Browsers cache aggressively once served.
|
||||
|
||||
If no favicon has been generated yet, responses fall through to a bundled default icon in `priv/static/defaults/`.
|
||||
|
||||
## Dynamic `site.webmanifest`
|
||||
|
||||
Served as `application/manifest+json`, generated fresh from settings on each request (cached with a short TTL):
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "Robin's Nature Shop",
|
||||
"short_name": "Robin's",
|
||||
"theme_color": "#2d4a3e",
|
||||
"background_color": "#ffffff",
|
||||
"display": "minimal-ui",
|
||||
"start_url": "/",
|
||||
"icons": [
|
||||
{
|
||||
"src": "/icon-192.png",
|
||||
"sizes": "192x192",
|
||||
"type": "image/png"
|
||||
},
|
||||
{
|
||||
"src": "/icon-512.png",
|
||||
"sizes": "512x512",
|
||||
"type": "image/png",
|
||||
"purpose": "maskable"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
`theme_color` is read from the active theme settings (the primary/accent colour). `background_color` defaults to white. `short_name` is configurable — defaults to shop name truncated at 12 characters.
|
||||
|
||||
## `<head>` additions in `shop_root.html.heex`
|
||||
|
||||
```html
|
||||
<!-- Favicon & PWA -->
|
||||
<link rel="icon" href="/favicon.svg" type="image/svg+xml" />
|
||||
<link rel="icon" href="/favicon-32x32.png" type="image/png" />
|
||||
<link rel="apple-touch-icon" href="/apple-touch-icon.png" />
|
||||
<link rel="manifest" href="/site.webmanifest" />
|
||||
<meta name="theme-color" content={@theme_settings.primary_colour || "#000000"} />
|
||||
```
|
||||
|
||||
The SVG `<link>` is listed first — browsers pick the first format they support. Older browsers fall through to the PNG.
|
||||
|
||||
## Customisation options
|
||||
|
||||
Surfaced in the theme editor, in the same section as the logo upload:
|
||||
|
||||
- **Favicon source** — "Use logo as icon" toggle (on by default) / "Upload separate icon" file input
|
||||
- **Short name** — shown under the icon on home screens; defaults to shop name (truncated)
|
||||
- **Icon background colour** — the fill shown when Android crops to a circle/squircle; defaults to theme background colour
|
||||
- **Theme colour** — browser chrome tint on Android and in the task switcher; defaults to theme primary colour
|
||||
|
||||
That's it. No more options — the generator handles everything else.
|
||||
|
||||
## Files to create/modify
|
||||
|
||||
- Migration — `favicon_variants` table
|
||||
- `lib/berrypod/media.ex` — `store_favicon_variants/1`, `get_favicon_variants/0`, `get_icon_image/0`
|
||||
- `lib/berrypod/media/image.ex` — add `"icon"` to `image_type` validation
|
||||
- `lib/berrypod/workers/favicon_generator_worker.ex` — new Oban job
|
||||
- `lib/berrypod_web/controllers/favicon_controller.ex` — serves all favicon routes + manifest
|
||||
- `lib/berrypod_web/router.ex` — favicon routes (before the catch-all)
|
||||
- `lib/berrypod_web/components/layouts/shop_root.html.heex` — `<link>` tags + theme-color meta
|
||||
- `lib/berrypod_web/live/admin/theme/index.ex` — icon upload section (alongside logo)
|
||||
- `priv/static/favicon.ico` — pre-baked generic fallback
|
||||
- `priv/static/defaults/` — bundled default favicon PNGs shown before any upload
|
||||
|
||||
## Tasks
|
||||
|
||||
| # | Task | Est |
|
||||
|---|------|-----|
|
||||
| 86 | Favicon source upload + settings — add `image_type: "icon"` to schema, "use logo as icon" toggle, icon upload in theme editor, `FaviconGeneratorWorker` Oban job, `favicon_variants` table | 2.5h |
|
||||
| 87 | `FaviconController` serving all favicon routes + dynamic `site.webmanifest`; add `<link>` tags and `theme-color` meta to `shop_root.html.heex`; bundle default fallback icons | 1.5h |
|
||||
| 88 | SVG dark mode injection for SVG-source favicons; icon background / short name customisation in theme editor | 1h |
|
||||
237
docs/plans/legal-page-generator.md
Normal file
237
docs/plans/legal-page-generator.md
Normal file
@@ -0,0 +1,237 @@
|
||||
# Legal page generator
|
||||
|
||||
> Status: Planned
|
||||
> Tasks: #83–85 in PROGRESS.md
|
||||
> Tier: 4 (Growth & content), Phase 1 can ship earlier
|
||||
|
||||
## Goal
|
||||
|
||||
Replace the static `PreviewData` placeholder content on the four policy/legal pages with generated content that's factually accurate for each shop — because Berrypod knows exactly what it does, who processes what data, and how fulfilment works.
|
||||
|
||||
Not a generic "fill in the blanks" template. Not LLM-generated waffle. A set of conditional paragraph functions that produce correct, shop-specific content from actual settings and provider data.
|
||||
|
||||
## The problem with current approach
|
||||
|
||||
All four content pages (`/privacy`, `/terms`, `/delivery`, `/about`) call `PreviewData.*_content()` which returns hardcoded placeholder text. Shop owners are expected to replace it manually — but most won't, or will copy-paste something generic that doesn't match how Berrypod actually works.
|
||||
|
||||
Berrypod already knows:
|
||||
- Which providers are connected (Printify, Printful — each with different lead times)
|
||||
- Which countries it ships to (from the `shipping_rates` table)
|
||||
- Whether VAT is enabled and the shop country
|
||||
- Whether abandoned cart recovery is enabled
|
||||
- Whether a newsletter is enabled
|
||||
- The shop name and contact email
|
||||
|
||||
This is enough to produce accurate, legally grounded content automatically.
|
||||
|
||||
## Pages covered
|
||||
|
||||
### 1. Privacy policy (`/privacy`)
|
||||
|
||||
**Always included:**
|
||||
|
||||
- What's collected: name, email, shipping address from orders (legal basis: contract performance, Article 6(1)(b) UK/EU GDPR)
|
||||
- Payment: processed by Stripe, card data never touches the shop
|
||||
- Analytics: privacy-first, no cookies, no personal data stored — server-side only, includes device type, country (derived from IP, not stored), referrer
|
||||
- Cookies: session cookie for cart and auth (strictly necessary, no consent required); country preference cookie for shipping rates. No tracking cookies, no third-party analytics cookies.
|
||||
- Sharing: shipping details shared with the connected provider(s) — names dynamically inserted
|
||||
- Retention: order data kept for 7 years (UK statutory accounting requirement); analytics data kept for 2 years
|
||||
- Contact: shop contact email from settings
|
||||
- Rights: right of access, rectification, deletion (with caveat: statutory retention periods apply), right to object to marketing
|
||||
|
||||
**Conditional sections:**
|
||||
|
||||
- Abandoned cart recovery enabled → "If you enter your email on our checkout page but don't complete payment, we may send you a single follow-up email. This is the only email you'll receive. You can unsubscribe at any time using the link in the email. We delete this data after 30 days." (UK PECR soft opt-in / EU legitimate interests — depending on shop country)
|
||||
- Newsletter enabled → email marketing section: subscription basis, how to unsubscribe, no third-party sharing
|
||||
- Stripe Tax enabled → "Tax calculation is handled by Stripe, which processes transaction and location data to determine applicable rates."
|
||||
|
||||
**Shop country drives jurisdiction language:**
|
||||
- UK → "under UK GDPR and PECR"
|
||||
- EU country → "under the EU General Data Protection Regulation (GDPR)"
|
||||
- US, AU, other → generic "applicable data protection laws"
|
||||
|
||||
---
|
||||
|
||||
### 2. Delivery & returns (`/delivery`)
|
||||
|
||||
This is the most data-rich page — Berrypod has real numbers from the DB.
|
||||
|
||||
**Production lead times** — driven by connected provider(s):
|
||||
|
||||
| Provider | Typical production |
|
||||
|----------|--------------------|
|
||||
| Printify | 2–7 business days |
|
||||
| Printful | 2–5 business days |
|
||||
|
||||
If both providers are connected, show combined note ("production times vary by product").
|
||||
|
||||
**Shipping destinations** — derived from `shipping_rates` table:
|
||||
- Query distinct countries with rates → group into regions → list with approximate delivery windows
|
||||
- If no shipping data: generic placeholder (fallback)
|
||||
|
||||
**Returns — POD-specific and legally accurate:**
|
||||
|
||||
This is where generic templates get it wrong. The correct position for print-on-demand:
|
||||
|
||||
- Consumer Contracts Regulations Regulation 28(1)(b) — "goods made to the consumer's specifications or clearly personalised" are **exempt** from the 14-day statutory right to cancel. Every POD product qualifies. This is the legal exemption that applies, and most shops don't cite it correctly.
|
||||
- Consumer Rights Act 2015 still applies to defective goods — if the item arrives damaged or with a printing defect, the customer is entitled to a repair, replacement, or refund.
|
||||
- The generated policy states this clearly: no change-of-mind returns (citing the exemption), but reprints/refunds for defects (citing CRA).
|
||||
|
||||
**Contact and cancellation window:**
|
||||
- Contact email from settings
|
||||
- Cancellation window: ~2 hours after ordering (before production begins)
|
||||
|
||||
---
|
||||
|
||||
### 3. Terms of service (`/terms`)
|
||||
|
||||
**Always included:**
|
||||
|
||||
- Governing law: driven by shop country setting
|
||||
- UK → "English law"
|
||||
- Ireland → "Irish law and EU regulations"
|
||||
- etc.
|
||||
- Products: made to order, colour variance disclaimer, all sales final (with returns caveat)
|
||||
- Payment: via Stripe, orders only confirmed on successful payment
|
||||
- Intellectual property: designs are the property of the shop owner; customers receive a licence for personal use
|
||||
- Limitations: we're not liable for delays caused by the print provider or postal service
|
||||
- Changes: terms may be updated, current version always at this URL
|
||||
|
||||
**Conditional:**
|
||||
- VAT enabled + registered → "prices include VAT where applicable"
|
||||
- Newsletter → marketing communications clause
|
||||
|
||||
---
|
||||
|
||||
### 4. Cookie policy
|
||||
|
||||
Currently a section within the privacy policy. Can be a standalone page if desired, or remain embedded.
|
||||
|
||||
**Berrypod's actual cookies (exhaustive):**
|
||||
|
||||
| Cookie | Purpose | Duration | Consent required? |
|
||||
|--------|---------|----------|-------------------|
|
||||
| `_berrypod_session` | Session state: cart contents, auth | Session | No — strictly necessary |
|
||||
| `country_code` | Remember shipping country preference | 1 year | No — strictly necessary for service |
|
||||
|
||||
That's it. No analytics cookies. No tracking. No third-party embeds. The generated cookie policy is short and accurate.
|
||||
|
||||
---
|
||||
|
||||
### 5. About page
|
||||
|
||||
Not generated — it's the shop owner's own story, Berrypod can't write that for them. But the existing placeholder template should be clearly labelled as placeholder and easy to replace. The page editor (task #19) handles this properly. No changes needed here for Phase 1.
|
||||
|
||||
---
|
||||
|
||||
## Content block format
|
||||
|
||||
The generator produces lists of `content_blocks` in the format already used by `<.rich_text>`:
|
||||
|
||||
```elixir
|
||||
[
|
||||
%{type: :lead, text: "..."},
|
||||
%{type: :heading, text: "..."},
|
||||
%{type: :paragraph, text: "..."},
|
||||
%{type: :list, items: ["...", "..."]},
|
||||
%{type: :closing, text: "..."}
|
||||
]
|
||||
```
|
||||
|
||||
No changes to the template layer — it already knows how to render these. The generator just produces better data.
|
||||
|
||||
---
|
||||
|
||||
## Generator module
|
||||
|
||||
`lib/berrypod/legal_pages.ex` — one public function per page:
|
||||
|
||||
```elixir
|
||||
defmodule Berrypod.LegalPages do
|
||||
alias Berrypod.{Settings, Shipping, Providers}
|
||||
|
||||
def privacy_content do
|
||||
shop_name = Settings.get(:shop_name) || "this shop"
|
||||
contact_email = Settings.get(:contact_email)
|
||||
shop_country = Settings.get(:shop_country, "GB")
|
||||
abandoned_cart_enabled = Settings.get(:abandoned_cart_enabled, false)
|
||||
newsletter_enabled = Settings.get(:newsletter_enabled, false)
|
||||
|
||||
base_sections()
|
||||
|> maybe_add_abandoned_cart(abandoned_cart_enabled)
|
||||
|> maybe_add_newsletter(newsletter_enabled)
|
||||
|> add_jurisdiction(shop_country)
|
||||
|> add_contact(shop_name, contact_email)
|
||||
end
|
||||
|
||||
def delivery_content do
|
||||
providers = Providers.connected_providers()
|
||||
shipping_countries = Shipping.list_countries_with_rates()
|
||||
|
||||
production_section(providers)
|
||||
++ shipping_section(shipping_countries)
|
||||
++ returns_section()
|
||||
++ cancellation_section()
|
||||
end
|
||||
|
||||
def terms_content do
|
||||
shop_name = Settings.get(:shop_name) || "this shop"
|
||||
shop_country = Settings.get(:shop_country, "GB")
|
||||
vat_enabled = Settings.get(:vat_enabled, false)
|
||||
|
||||
base_terms(shop_name)
|
||||
|> add_governing_law(shop_country)
|
||||
|> maybe_add_vat_clause(vat_enabled)
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Two phases
|
||||
|
||||
### Phase 1 — replace PreviewData (no page editor needed)
|
||||
|
||||
Wire `LegalPages.*_content()` into the existing `Content` LiveView, replacing the three `PreviewData.*_content()` calls for privacy, delivery, and terms. The about page stays as-is (it's the shop owner's story).
|
||||
|
||||
The generated content shows in the live shop immediately. No admin UI needed yet — the content is always accurate because it reflects real settings.
|
||||
|
||||
### Phase 2 — page editor integration
|
||||
|
||||
When the page editor (task #19) ships, add:
|
||||
|
||||
- "Regenerate from settings" button per page — reruns the generator and replaces stored content
|
||||
- Content marked as "auto-generated" vs "customised" — so the admin can tell what's been manually edited
|
||||
- Generator runs automatically when relevant settings change (provider connected, VAT toggled, abandoned cart enabled) — a PubSub broadcast triggers regeneration
|
||||
|
||||
---
|
||||
|
||||
## What the generator is and isn't
|
||||
|
||||
**Is:**
|
||||
- Factually accurate based on real Berrypod behaviour
|
||||
- Legally grounded (cites correct UK statutes: PECR, Consumer Contracts Regulations, Consumer Rights Act)
|
||||
- Useful as a starting point that's better than any generic template
|
||||
|
||||
**Isn't:**
|
||||
- Legal advice. The generated pages include a brief footer note: "This policy was auto-generated based on how this shop is configured. You should review it and seek independent legal advice if you're unsure."
|
||||
- Comprehensive for edge cases (international VAT registration, non-UK statutory frameworks beyond GDPR)
|
||||
- A substitute for a solicitor if the shop does complex things
|
||||
|
||||
---
|
||||
|
||||
## Files to create/modify
|
||||
|
||||
- `lib/berrypod/legal_pages.ex` — new, generator functions for each page
|
||||
- `lib/berrypod_web/live/shop/content.ex` — replace three `PreviewData.*_content()` calls with `LegalPages.*_content()`
|
||||
- Phase 2: page editor admin UI for saved/regenerated page content
|
||||
|
||||
---
|
||||
|
||||
## Tasks
|
||||
|
||||
| # | Task | Est |
|
||||
|---|------|-----|
|
||||
| 83 | `LegalPages` module — generate accurate privacy, delivery, and terms content from settings + provider + shipping data | 2.5h |
|
||||
| 84 | Wire `LegalPages` into `Content` LiveView — replace `PreviewData` calls, add tests | 45m |
|
||||
| 85 | Page editor integration — "Regenerate" button, auto-regenerate on settings change, customised vs auto label | 1.5h (depends on task #19) |
|
||||
471
docs/plans/profit-aware-pricing.md
Normal file
471
docs/plans/profit-aware-pricing.md
Normal file
@@ -0,0 +1,471 @@
|
||||
# Profit-aware pricing, tax & sales
|
||||
|
||||
> Status: Planned
|
||||
> Tasks: #63–71 in PROGRESS.md
|
||||
> Tier: 3.5 (Business tools)
|
||||
|
||||
## Goal
|
||||
|
||||
Shop owners always know exactly what they're making on every sale. No hidden costs, no surprises. The system shows the full cost breakdown (provider cost, shipping, Stripe fees, tax) and prevents selling at a loss.
|
||||
|
||||
Sales replace discount codes — transparent, visible, same price for everyone. No dark patterns.
|
||||
|
||||
## Design principles
|
||||
|
||||
- **Default is simple.** Most small POD sellers aren't VAT registered. By default there's no tax — prices are prices and all revenue is profit.
|
||||
- **No surprises at checkout.** Tax-inclusive pricing is the default (covers UK, EU, AU, NZ, Japan — most of the world). US/CA sellers can opt into tax-exclusive display.
|
||||
- **Margin guard.** The system prevents shop owners from accidentally selling at a loss — whether through pricing or discounts.
|
||||
- **No discount codes.** The empty "enter code" box at checkout is a dark pattern. Sales are visible, transparent, and the same for everyone.
|
||||
|
||||
## Stripe does the heavy lifting
|
||||
|
||||
Many features in this plan can be handled by Stripe rather than reimplemented in Berrypod. The rule: let Stripe do what Stripe is good at, keep in Berrypod what requires knowledge of our costs.
|
||||
|
||||
| Feature | Who handles it | Why |
|
||||
|---------|---------------|-----|
|
||||
| Tax calculation at checkout | **Stripe Tax** | Automatic correct rates for 50+ countries. Rules change constantly — Stripe keeps them current. One line of config: `automatic_tax: %{enabled: true}` on the Checkout session. Costs 0.5% per transaction (~12.5p on a £25 sale). Far cheaper than building and maintaining tax tables ourselves. |
|
||||
| Tax display at checkout | **Stripe Tax** | Stripe Checkout shows the tax line automatically when Stripe Tax is enabled. |
|
||||
| Tax registration management | **Stripe Dashboard** | Shop owner registers their VAT/GST number in Stripe, not in Berrypod. Stripe handles the jurisdiction logic. |
|
||||
| Exact Stripe fees per order | **Balance Transaction API** | After payment, fetch the real fee from `stripe_charge.balance_transaction`. No estimation needed — get the actual number. |
|
||||
| "inc. VAT" on shop pages | **Berrypod** | Stripe only acts at checkout. We still need to show tax-inclusive labelling on product/cart pages before checkout. |
|
||||
| Profit calculation | **Berrypod** | Stripe doesn't know our provider costs. |
|
||||
| Margin warnings & guards | **Berrypod** | Stripe can't help here. |
|
||||
| Sales & promotions | **Berrypod** | Stripe has coupons/promo codes but those are the dark pattern we're avoiding. Our sales are transparent price changes, not codes. |
|
||||
| Price editor | **Berrypod** | Internal business intelligence — Stripe can't help. |
|
||||
|
||||
**Net effect on the plan:**
|
||||
- **#65 (Stripe fees)** — simplified. Fetch real fee from Balance Transaction API after payment. No estimation, no configurable rate tables.
|
||||
- **#66 (Tax)** — simplified. Configure tax registration in Stripe Dashboard. Add `automatic_tax` to Checkout session. We only need a "registered" toggle in Berrypod settings to drive the shop-page "inc. VAT" display and the profit calculation.
|
||||
- **No country-specific tax rate tables** needed in Berrypod at all.
|
||||
|
||||
## What we already have
|
||||
|
||||
| Data point | Source | Status |
|
||||
|-----------|--------|--------|
|
||||
| Variant selling price | `product_variants.price` | Synced from both providers |
|
||||
| Variant cost (Printify) | `product_variants.cost` | Synced directly from API (`var["cost"]`) |
|
||||
| Variant cost (Printful) | `product_variants.cost` | Always `nil` — sync API doesn't include cost |
|
||||
| Price snapshot at order | `order_items.unit_price` | Captured at order creation |
|
||||
| Shipping rates | `shipping_rates` table | Provider shipping costs per country/blueprint |
|
||||
| `ProductVariant.profit/1` | Helper function | Exists but unused anywhere |
|
||||
| `compare_at_price` | `product_variants.compare_at_price` | Synced from providers (strikethrough price) |
|
||||
|
||||
## What's missing
|
||||
|
||||
| Data point | Where needed | Notes |
|
||||
|-----------|-------------|-------|
|
||||
| Printful variant cost | `product_variants.cost` | Need catalog API cross-reference during sync |
|
||||
| Cost snapshot on orders | `order_items.unit_cost` | New field — snapshot at order creation |
|
||||
| Order-level cost totals | `orders.total_cost`, `orders.gross_profit` | New fields — calculated from items |
|
||||
| Exact Stripe fee per order | `orders.stripe_fee` | Fetch from Balance Transaction API post-payment |
|
||||
| Tax registration toggle | Settings | Drives "inc. VAT" display and profit calculations |
|
||||
| Shop country | Settings | Drives tax display mode defaults (inclusive vs exclusive) |
|
||||
| Sales/promotions | New `sales` table | Scoped discounts with date ranges |
|
||||
| Minimum margin threshold | Settings | Prevents pricing/discounts below floor |
|
||||
|
||||
---
|
||||
|
||||
## Task breakdown
|
||||
|
||||
### #63 — Fix Printful cost sync (45m)
|
||||
|
||||
**Problem:** Printful's `GET /store/products/{id}` response includes `retail_price` (the seller's price) but not the production cost. Printify includes `cost` directly in the variant data.
|
||||
|
||||
**Solution:** Printful's catalog API (`GET /products/{catalog_product_id}/variants`) returns the base cost per variant. During sync, cross-reference catalog variant data to populate `cost`.
|
||||
|
||||
**Files:**
|
||||
- `lib/berrypod/providers/printful.ex` — `normalize_variant/1` currently hardcodes `cost: nil`
|
||||
- Already have `catalog_product_id` from the sync response (`sv["product"]["product_id"]`)
|
||||
- Already fetch catalog data for colour hex codes — extend this to grab cost too
|
||||
|
||||
**Approach:**
|
||||
- In `sync_product/2`, after fetching the store product, also fetch catalog variants
|
||||
- Build a lookup map: `catalog_variant_id => cost`
|
||||
- In `normalize_variant/1`, look up cost from the catalog map
|
||||
- Parse cost from catalog `price` field (this is what Printful charges the seller)
|
||||
|
||||
**Tests:**
|
||||
- Update Printful provider test stubs to include catalog variant cost data
|
||||
- Assert `cost` is populated on synced Printful variants
|
||||
|
||||
---
|
||||
|
||||
### #64 — Cost snapshot on orders (1.5h)
|
||||
|
||||
**Migration:**
|
||||
```elixir
|
||||
alter table(:order_items) do
|
||||
add :unit_cost, :integer # provider cost snapshot in minor units (pence/cents)
|
||||
end
|
||||
|
||||
alter table(:orders) do
|
||||
add :total_cost, :integer # sum of (unit_cost * qty) for all items
|
||||
add :gross_profit, :integer # subtotal - total_cost - shipping_cost (before fees/tax)
|
||||
end
|
||||
```
|
||||
|
||||
**Order creation changes:**
|
||||
- In `Orders.create_order/1`, look up `ProductVariant.cost` for each item
|
||||
- Snapshot as `unit_cost` on each `OrderItem`
|
||||
- Calculate `total_cost` = sum of `unit_cost * quantity` across items
|
||||
- Calculate `gross_profit` = `subtotal - total_cost`
|
||||
- If variant cost is `nil` (legacy data or missing), store `nil` — don't guess
|
||||
|
||||
**Post-payment update:**
|
||||
- After Stripe webhook updates `shipping_cost`, recalculate: `gross_profit = subtotal - total_cost - shipping_cost`
|
||||
|
||||
**Files:**
|
||||
- New migration
|
||||
- `lib/berrypod/orders/order.ex` — add fields to schema
|
||||
- `lib/berrypod/orders/order_item.ex` — add `unit_cost` field
|
||||
- `lib/berrypod/orders.ex` — `create_order/1` enrichment logic
|
||||
|
||||
**Tests:**
|
||||
- Order creation with cost snapshots
|
||||
- Order with nil cost (Printful legacy, unknown cost)
|
||||
- Gross profit calculation with and without shipping
|
||||
|
||||
---
|
||||
|
||||
### #65 — Exact Stripe fees (45m)
|
||||
|
||||
Rather than estimating Stripe fees from configurable rate tables, fetch the real fee from Stripe's Balance Transaction API after payment.
|
||||
|
||||
**How:**
|
||||
- After `checkout.session.completed` webhook, the payment intent has a charge
|
||||
- Fetch `stripe_charge.balance_transaction` to get the exact fee
|
||||
- Stripity Stripe: `Stripe.BalanceTransaction.retrieve(balance_transaction_id)`
|
||||
- Returns `%{fee: integer}` in the same currency/minor units
|
||||
|
||||
**New field:**
|
||||
```elixir
|
||||
alter table(:orders) do
|
||||
add :stripe_fee, :integer # exact fee in minor units, set after payment
|
||||
end
|
||||
```
|
||||
|
||||
**Flow:**
|
||||
```
|
||||
checkout.session.completed webhook
|
||||
→ retrieve payment_intent → charge → balance_transaction
|
||||
→ extract fee
|
||||
→ update order: stripe_fee, recalculate gross_profit
|
||||
```
|
||||
|
||||
**Files:**
|
||||
- Migration (add field)
|
||||
- `lib/berrypod/orders/order.ex` — add field
|
||||
- `lib/berrypod_web/controllers/stripe_webhook_controller.ex` — fetch and store after payment
|
||||
|
||||
**Tests:**
|
||||
- Webhook handler stores correct fee from mocked Balance Transaction response
|
||||
|
||||
---
|
||||
|
||||
### #66 — Tax toggle & Stripe Tax (1.5h)
|
||||
|
||||
**How tax works globally:**
|
||||
|
||||
| Market | Display | Rule |
|
||||
|--------|---------|------|
|
||||
| UK, EU, AU, NZ, JP, most of world | Tax-inclusive | Legal requirement for B2C. Price = price. |
|
||||
| US, CA | Tax-exclusive | Tax added at checkout. Cultural norm. |
|
||||
|
||||
**What Stripe Tax handles (no code needed):**
|
||||
- Correct tax rate per customer location + product type
|
||||
- Tax line shown in Stripe Checkout
|
||||
- Tax amount on `checkout.session` object after payment
|
||||
- Shop owner registers their VAT/GST number in Stripe Dashboard — not in Berrypod
|
||||
|
||||
**What Berrypod needs:**
|
||||
|
||||
**New settings:**
|
||||
- `shop_country` — ISO code (e.g. "GB", "US", "DE"). Drives display defaults.
|
||||
- `tax_registered` — boolean, default `false`
|
||||
- `tax_display` — `"inclusive"` (default for UK/EU/AU) or `"exclusive"` (default for US/CA). Auto-set from country but editable.
|
||||
|
||||
**When `tax_registered` is OFF (default):**
|
||||
- No "inc. VAT" label anywhere
|
||||
- Profit = revenue - costs - Stripe fee. Simple.
|
||||
- `automatic_tax` NOT sent to Stripe Checkout
|
||||
|
||||
**When `tax_registered` is ON:**
|
||||
- Add `automatic_tax: %{enabled: true}` to Stripe Checkout session
|
||||
- Stripe calculates and displays the correct tax at checkout
|
||||
- For tax-inclusive shops (UK/EU/AU): show "inc. VAT" on product pages, cart, and invoice
|
||||
- For tax-exclusive shops (US/CA): show "+ tax" on product pages, cart
|
||||
- Snapshot `tax_amount` from `checkout.session.total_details.amount_tax` onto order after payment
|
||||
- Profit calculation: `gross_profit = subtotal - total_cost - shipping_cost - tax_amount - stripe_fee`
|
||||
|
||||
**New field:**
|
||||
```elixir
|
||||
alter table(:orders) do
|
||||
add :tax_amount, :integer # from Stripe, set after payment (nil if not registered)
|
||||
end
|
||||
```
|
||||
|
||||
**Profit calculation comparison:**
|
||||
```
|
||||
Not VAT registered (default): VAT registered, tax-inclusive (UK):
|
||||
£25.00 price £25.00 price
|
||||
-£8.00 provider cost -£4.17 VAT (from Stripe)
|
||||
-£0.58 Stripe fee (exact) -£8.00 provider cost
|
||||
──────── -£0.58 Stripe fee (exact)
|
||||
£16.42 profit ────────
|
||||
£12.25 profit
|
||||
```
|
||||
|
||||
**Admin UI:**
|
||||
- Settings page section: "Tax"
|
||||
- Country selector → sets `tax_display` default
|
||||
- Toggle: "I'm registered for VAT/GST/Sales tax"
|
||||
- When on: link to Stripe Dashboard to add tax registration + display mode selector
|
||||
- Preview: "Your customers see: £25.00 inc. VAT" or "$25.00 + tax"
|
||||
|
||||
**Files:**
|
||||
- `lib/berrypod/settings.ex` — new setting keys
|
||||
- Admin settings LiveView — new section
|
||||
- Shop layout/components — conditional "inc. VAT" / "+ tax" display
|
||||
- `lib/berrypod_web/controllers/checkout_controller.ex` — add `automatic_tax` when registered
|
||||
- `lib/berrypod_web/controllers/stripe_webhook_controller.ex` — snapshot `tax_amount`
|
||||
- Migration — `add :tax_amount, :integer` on orders
|
||||
|
||||
**Tests:**
|
||||
- Checkout session includes `automatic_tax` when registered, not when unregistered
|
||||
- Tax amount snapshotted from webhook payload
|
||||
- "inc. VAT" shown/hidden based on settings
|
||||
|
||||
---
|
||||
|
||||
### #67 — Admin profit dashboard (3h)
|
||||
|
||||
**Route:** `/admin/profit` (or section within existing dashboard)
|
||||
|
||||
**Per-product margins table:**
|
||||
- Product name, variant count
|
||||
- Selling price range (min–max across variants)
|
||||
- Cost range (min–max, or "unknown" if nil)
|
||||
- Margin % range
|
||||
- Flagged rows for low/negative margins
|
||||
- Sortable by margin (worst first is useful)
|
||||
|
||||
**Per-order profit breakdown:**
|
||||
- Recent orders table: order #, date, subtotal, cost, shipping, Stripe fee, tax (if applicable), profit, margin %
|
||||
- Click to expand: per-item breakdown
|
||||
- Colour-coded: green (healthy), amber (thin), red (loss)
|
||||
|
||||
**Overall P&L summary cards:**
|
||||
- Total revenue (period)
|
||||
- Total provider costs
|
||||
- Total Stripe fees
|
||||
- Total tax liability (if registered)
|
||||
- **Net profit**
|
||||
- Average margin %
|
||||
- Period selector (7d / 30d / 90d / all time)
|
||||
|
||||
**Handles unknowns gracefully:**
|
||||
- Orders with `nil` cost show "unknown" — not wrong numbers, not excluded
|
||||
- Note: "X orders have unknown cost (pre-cost-tracking, or Printful products before sync)"
|
||||
|
||||
**Files:**
|
||||
- `lib/berrypod_web/live/admin/profit_live.ex` — new LiveView
|
||||
- `lib/berrypod/orders.ex` — profit query functions
|
||||
- Router — new admin route
|
||||
- Admin nav — new sidebar link
|
||||
|
||||
---
|
||||
|
||||
### #68 — Profit-aware price editor (2h)
|
||||
|
||||
**Where:** Admin products page (new or enhanced)
|
||||
|
||||
**For each variant, show a live breakdown:**
|
||||
```
|
||||
Selling price: £25.00
|
||||
Provider cost: -£8.00
|
||||
Shipping (avg): -£3.50
|
||||
Stripe fee (est): -£0.58
|
||||
VAT (if reg'd): -£4.17
|
||||
────────
|
||||
Your profit: £8.75 (35%)
|
||||
```
|
||||
|
||||
**Live updating:** phx-change on price input — profit recalculates as they type.
|
||||
|
||||
**Warnings:**
|
||||
- Amber if margin < minimum threshold (e.g. 15%)
|
||||
- Red + message if negative: "You'd lose £X.XX per sale at this price"
|
||||
- Warning only — doesn't block saving (the margin guard on sales is the hard block)
|
||||
|
||||
**Minimum price suggestion:**
|
||||
- "Minimum price for 20% margin: £X.XX"
|
||||
- Quick-set button
|
||||
|
||||
**Settings:**
|
||||
- `minimum_margin_percentage` — configurable (default 20%). Used for warnings here and as the hard floor for sales (#70).
|
||||
|
||||
**Note on Stripe fee in price editor:** Since we can only get exact fees *after* payment (Balance Transaction API), the price editor uses an estimated fee based on the shop's country (e.g. UK domestic 1.5% + 20p). This is clearly labelled "est." — it's for guidance, not accounting.
|
||||
|
||||
**Files:**
|
||||
- Admin products LiveView — margin breakdown component
|
||||
- `lib/berrypod/products.ex` — margin calculation helpers
|
||||
|
||||
---
|
||||
|
||||
### #69 — Sales & promotions (3h)
|
||||
|
||||
**Schema:**
|
||||
```elixir
|
||||
create table(:sales, primary_key: false) do
|
||||
add :id, :binary_id, primary_key: true
|
||||
add :name, :string, null: false # "Summer sale", "Black Friday"
|
||||
add :discount_type, :string, null: false # "percentage" or "fixed"
|
||||
add :discount_value, :integer, null: false # 20 (= 20%) or 500 (= £5.00)
|
||||
add :scope, :string, null: false # "all", "category", "products"
|
||||
add :scope_value, :string # category slug or comma-separated product IDs
|
||||
add :starts_at, :utc_datetime, null: false
|
||||
add :ends_at, :utc_datetime, null: false
|
||||
add :active, :boolean, default: true
|
||||
timestamps()
|
||||
end
|
||||
```
|
||||
|
||||
**Sale price calculation:**
|
||||
```elixir
|
||||
def sale_price(variant, sale) do
|
||||
case sale.discount_type do
|
||||
"percentage" -> variant.price - round(variant.price * sale.discount_value / 100)
|
||||
"fixed" -> max(variant.price - sale.discount_value, 0)
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
**Active sale resolution:**
|
||||
```elixir
|
||||
def active_sale_for(product, now \\ DateTime.utc_now()) do
|
||||
# Check product-specific sales first, then category, then catalogue-wide
|
||||
# Return the best (highest discount) applicable sale
|
||||
# Only return if starts_at <= now < ends_at and active == true
|
||||
end
|
||||
```
|
||||
|
||||
**Where sales apply:**
|
||||
- Product listing pages — original price struck through + sale price + "X% off" badge
|
||||
- Product detail page — same treatment
|
||||
- Cart — sale prices used for line items
|
||||
- Checkout — sale prices sent to Stripe as `unit_amount`
|
||||
|
||||
**Admin UI:**
|
||||
- `/admin/sales` — list of sales (active, scheduled, ended)
|
||||
- Create/edit: name, discount type/value, scope picker, date range
|
||||
- Preview: "This sale will reduce 'Classic Tee' from £25.00 to £20.00 (20% off)"
|
||||
|
||||
**Files:**
|
||||
- New migration + schema (`lib/berrypod/sales/sale.ex`)
|
||||
- `lib/berrypod/sales.ex` — context module
|
||||
- Product display helpers — inject sale pricing
|
||||
- Cart hydration — apply sale prices
|
||||
- `lib/berrypod_web/controllers/checkout_controller.ex` — use sale prices for Stripe line items
|
||||
- Admin LiveView for sale management
|
||||
- Shop components — sale badges, strikethrough pricing
|
||||
|
||||
---
|
||||
|
||||
### #70 — Margin guard on sales (1h)
|
||||
|
||||
When creating or updating a sale, the system calculates worst-case profit for every affected variant. Hard block if any would breach the minimum threshold.
|
||||
|
||||
**Calculation for each affected variant:**
|
||||
```
|
||||
sale_price = apply_discount(variant.price, sale)
|
||||
cost = variant.cost (nil → skip guard, can't verify)
|
||||
shipping = lowest shipping rate for this product (domestic)
|
||||
stripe_fee = estimated_fee(sale_price) # approximate, for guidance
|
||||
tax = extracted from sale_price if registered
|
||||
profit = sale_price - cost - shipping - stripe_fee - tax
|
||||
margin = profit / sale_price * 100
|
||||
```
|
||||
|
||||
**If any variant breaches threshold:**
|
||||
- Show which variants fail and the numbers
|
||||
- "Classic Tee (S, Blue): £15.00 sale price → £4.42 profit (29%) — below 30% minimum"
|
||||
- Suggest: "Maximum discount for 30% margin on all variants: 15%"
|
||||
- Block save
|
||||
|
||||
**Edge cases:**
|
||||
- `nil` cost: skip guard, show "Cost unknown for X variants — margin can't be verified"
|
||||
- Fixed discount ≥ price: flagged as "would make product free"
|
||||
|
||||
**Files:**
|
||||
- `lib/berrypod/sales.ex` — validation in changeset or create function
|
||||
- Admin sales LiveView — error display + suggestions
|
||||
|
||||
---
|
||||
|
||||
### #71 — Announcement bar (1.5h)
|
||||
|
||||
**Settings:**
|
||||
- `announcement_text` — e.g. "Summer sale — 20% off everything until Friday!"
|
||||
- `announcement_link` — optional link target (e.g. "/collections/sale")
|
||||
- `announcement_active` — boolean
|
||||
- Auto-populated from active sale, but freely editable
|
||||
|
||||
**Display:**
|
||||
- Full-width bar at top of every shop page, above the header
|
||||
- Dismissable — localStorage remembers dismissal for session
|
||||
- Theme-aware — uses CSS custom properties from the existing theme system
|
||||
- Hidden when no active announcement
|
||||
|
||||
**Admin UI:**
|
||||
- Settings section or inline on the sales page
|
||||
- Live preview
|
||||
|
||||
**Files:**
|
||||
- Shop layout — announcement bar component
|
||||
- Settings keys
|
||||
- Admin UI
|
||||
- CSS — minimal, theme-aware
|
||||
- JS hook for localStorage dismissal
|
||||
|
||||
---
|
||||
|
||||
## Data flow
|
||||
|
||||
```
|
||||
Provider API (Printify/Printful)
|
||||
↓
|
||||
Sync: variant.cost populated
|
||||
(Printify: direct; Printful: catalog API cross-reference)
|
||||
↓
|
||||
Admin price editor
|
||||
- Shows live margin breakdown
|
||||
- Warnings on thin/negative margins
|
||||
↓
|
||||
Sale created (with margin guard)
|
||||
- Hard block if any variant breaches minimum threshold
|
||||
↓
|
||||
Shop: sale prices displayed (strikethrough + badge)
|
||||
↓
|
||||
Cart: sale prices applied to line items
|
||||
↓
|
||||
Checkout:
|
||||
- Sale prices sent to Stripe as unit_amount
|
||||
- automatic_tax enabled (if registered) → Stripe calculates tax
|
||||
↓
|
||||
Payment completed — Stripe webhook fires:
|
||||
- shipping_cost updated from Stripe session
|
||||
- tax_amount snapshotted from session.total_details.amount_tax
|
||||
- stripe_fee fetched from Balance Transaction API (exact, not estimated)
|
||||
- gross_profit = subtotal - total_cost - shipping_cost - tax_amount - stripe_fee
|
||||
↓
|
||||
Profit dashboard:
|
||||
- Per-product margins
|
||||
- Per-order P&L (with exact fees and tax)
|
||||
- Overall business health
|
||||
```
|
||||
|
||||
## Migration path
|
||||
|
||||
All new fields are additive. Existing orders will have `nil` for `unit_cost`, `total_cost`, `gross_profit`, `stripe_fee`, and `tax_amount`. The profit dashboard handles `nil` gracefully — shows "unknown" not wrong numbers.
|
||||
|
||||
Backfilling is possible but not essential. The dashboard is most useful for orders going forward.
|
||||
500
docs/plans/url-redirects.md
Normal file
500
docs/plans/url-redirects.md
Normal file
@@ -0,0 +1,500 @@
|
||||
# URL redirects
|
||||
|
||||
> Status: Planned
|
||||
> Tasks: #78–81 in PROGRESS.md
|
||||
> Tier: 3 (Compliance & quality — SEO dependency)
|
||||
|
||||
## Goal
|
||||
|
||||
Preserve link equity and customer experience when product URLs change or products are removed. Automatically handle the most common cases, use analytics data to identify what actually matters, and surface anything ambiguous for admin review.
|
||||
|
||||
## Why it matters
|
||||
|
||||
Product slugs in Berrypod are generated from product titles via `Slug.slugify(title)`. When a provider renames a product, the next sync generates a new slug and the old URL becomes a 404. These old URLs may be:
|
||||
|
||||
- Indexed by Google (losing SEO rank)
|
||||
- Shared on social media, in emails, in newsletters
|
||||
- Bookmarked by returning customers
|
||||
|
||||
Most redirect implementations just provide a manual table. The insight here is that we already have analytics data recording which paths have had real human traffic — so we can separate 404s that matter (broken real URLs) from noise (bot scanners, `/wp-admin` probes, etc.) without any manual work.
|
||||
|
||||
## Three layers
|
||||
|
||||
### Layer 1: Automatic redirect creation on slug change
|
||||
|
||||
The most common case. When a product's title changes during sync, the slug changes, and the old `/products/old-slug` URL breaks. We detect this automatically in `upsert_product/2`.
|
||||
|
||||
**Hook point:** `lib/berrypod/products.ex:421–425` — the `product ->` branch in `upsert_product/2` where `update_product(product, attrs)` is called. At this point we have `product.slug` (old) and can compute the new slug from `attrs[:title]`.
|
||||
|
||||
```elixir
|
||||
product ->
|
||||
old_slug = product.slug
|
||||
new_slug = Slug.slugify(attrs[:title] || attrs["title"])
|
||||
|
||||
case update_product(product, attrs) do
|
||||
{:ok, updated_product} ->
|
||||
if old_slug != updated_product.slug do
|
||||
Redirects.create_auto(%{
|
||||
from_path: "/products/#{old_slug}",
|
||||
to_path: "/products/#{updated_product.slug}",
|
||||
source: :auto_slug_change
|
||||
})
|
||||
end
|
||||
{:ok, updated_product, :updated}
|
||||
|
||||
error -> error
|
||||
end
|
||||
```
|
||||
|
||||
`create_auto/1` uses `on_conflict: :nothing` on the `from_path` unique index — safe to call repeatedly if sync runs multiple times.
|
||||
|
||||
### Layer 2: A `redirects` table checked early in the Plug pipeline
|
||||
|
||||
One table, one Plug, all redirect types flow through the same path.
|
||||
|
||||
**Plug position:** Added to the `:browser` pipeline in `router.ex`, before routing. Checks a path, 301s and halts if a redirect exists, otherwise passes through.
|
||||
|
||||
```elixir
|
||||
# router.ex
|
||||
pipeline :browser do
|
||||
...
|
||||
plug BerrypodWeb.Plugs.Redirects
|
||||
...
|
||||
end
|
||||
```
|
||||
|
||||
```elixir
|
||||
defmodule BerrypodWeb.Plugs.Redirects do
|
||||
import Plug.Conn
|
||||
alias Berrypod.Redirects
|
||||
|
||||
def init(opts), do: opts
|
||||
|
||||
def call(%{request_path: path} = conn, _opts) do
|
||||
case Redirects.lookup(path) do
|
||||
{:ok, redirect} ->
|
||||
Redirects.increment_hit_count(redirect)
|
||||
|
||||
conn
|
||||
|> put_resp_header("location", redirect.to_path)
|
||||
|> send_resp(redirect.status_code, "")
|
||||
|> halt()
|
||||
|
||||
:not_found ->
|
||||
conn
|
||||
end
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
**Caching:** The redirect lookup is on the hot path for every request. Use ETS for an in-memory cache, populated on app start and invalidated on any redirect create/update/delete.
|
||||
|
||||
```elixir
|
||||
# On app start, load all redirects into ETS
|
||||
Redirects.warm_cache()
|
||||
|
||||
# On redirect change, invalidate
|
||||
Redirects.invalidate_cache(from_path)
|
||||
```
|
||||
|
||||
The ETS table maps `from_path` (binary) → `{to_path, status_code}`. Cache miss falls through to DB. Given redirects are rare and mostly set-and-forget, the cache hit rate should be near 100% after warmup.
|
||||
|
||||
### Layer 3: Analytics-powered 404 monitoring
|
||||
|
||||
When a 404 fires, most hits are bots and scanners. The signal that distinguishes a real broken URL from noise is analytics history: if a path appears in `events` with prior real pageviews, it was a genuine product page.
|
||||
|
||||
**404 handler hook:** The existing `error.ex` LiveView renders 404s. Add a side-effect: when a 404 fires on a path matching `/products/:slug` or `/collections/:slug`, query analytics and potentially auto-resolve.
|
||||
|
||||
```elixir
|
||||
defp maybe_log_broken_url(path) do
|
||||
prior_hits = Analytics.count_pageviews_for_path(path)
|
||||
|
||||
if prior_hits > 0 do
|
||||
BrokenUrls.record(%{
|
||||
path: path,
|
||||
prior_analytics_hits: prior_hits
|
||||
})
|
||||
attempt_auto_resolve(path, prior_hits)
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
**Auto-resolution attempt:**
|
||||
|
||||
For `/products/:slug` 404s, extract the slug and run it through the FTS5 search index to find the most likely current product:
|
||||
|
||||
```elixir
|
||||
defp attempt_auto_resolve("/products/" <> old_slug, _hits) do
|
||||
query = String.replace(old_slug, "-", " ")
|
||||
|
||||
case Search.search_products(query, limit: 1) do
|
||||
[%{score: score, slug: new_slug}] when score > @confidence_threshold ->
|
||||
Redirects.create_auto(%{
|
||||
from_path: "/products/#{old_slug}",
|
||||
to_path: "/products/#{new_slug}",
|
||||
source: :analytics_detected,
|
||||
confidence: score
|
||||
})
|
||||
|
||||
_ ->
|
||||
# No confident match - leave in broken_urls for admin review
|
||||
:ok
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
The `@confidence_threshold` needs tuning — FTS5 BM25 scores are negative (more negative = better match). Start conservative; it's better to leave something for manual review than to auto-redirect to the wrong product.
|
||||
|
||||
For **deleted products** with no match, the redirect target defaults to the product's last known category collection page if that's inferable (from the path or broken_url record), otherwise falls back to `/`.
|
||||
|
||||
---
|
||||
|
||||
## Schemas
|
||||
|
||||
### `redirects` table
|
||||
|
||||
```elixir
|
||||
create table(:redirects, primary_key: false) do
|
||||
add :id, :binary_id, primary_key: true
|
||||
add :from_path, :string, null: false # "/products/old-classic-tee"
|
||||
add :to_path, :string, null: false # "/products/classic-tee-v2" or "/"
|
||||
add :status_code, :integer, default: 301 # 301 permanent, 302 temporary
|
||||
add :source, :string, null: false # "auto_slug_change" | "analytics_detected" | "admin"
|
||||
add :confidence, :float # FTS5 match score for analytics_detected, nil otherwise
|
||||
add :hit_count, :integer, default: 0 # incremented each time this redirect fires
|
||||
timestamps()
|
||||
end
|
||||
|
||||
create unique_index(:redirects, [:from_path])
|
||||
create index(:redirects, [:source])
|
||||
```
|
||||
|
||||
### `broken_urls` table
|
||||
|
||||
```elixir
|
||||
create table(:broken_urls, primary_key: false) do
|
||||
add :id, :binary_id, primary_key: true
|
||||
add :path, :string, null: false
|
||||
add :prior_analytics_hits, :integer, default: 0 # pageviews before the 404 started
|
||||
add :recent_404_count, :integer, default: 1 # 404s since it broke
|
||||
add :first_seen_at, :utc_datetime, null: false
|
||||
add :last_seen_at, :utc_datetime, null: false
|
||||
add :status, :string, default: "pending" # "pending" | "resolved" | "ignored"
|
||||
add :resolved_redirect_id, :binary_id # FK to redirects when resolved
|
||||
timestamps()
|
||||
end
|
||||
|
||||
create unique_index(:broken_urls, [:path])
|
||||
create index(:broken_urls, [:status])
|
||||
create index(:broken_urls, [:prior_analytics_hits]) # sort by impact
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Admin UI
|
||||
|
||||
**Route:** `/admin/redirects`
|
||||
|
||||
### Tab 1: Active redirects
|
||||
|
||||
Table of all redirects with columns: from path, to path, source (badge: auto/detected/manual), hit count, created at. Delete button to remove. Edit to change destination.
|
||||
|
||||
Sources:
|
||||
- `auto_slug_change` — created automatically when sync detected a slug change. Trust these.
|
||||
- `analytics_detected` — created from analytics + FTS5 match. Show confidence score. Worth reviewing.
|
||||
- `admin` — manually created.
|
||||
|
||||
### Tab 2: Broken URLs (pending review)
|
||||
|
||||
Table sorted by `prior_analytics_hits` descending — highest impact broken URLs at the top.
|
||||
|
||||
Columns: path, prior traffic (from analytics), 404s since breaking, first seen.
|
||||
|
||||
Each row has a quick action: enter a redirect destination and save, or mark as ignored (e.g. it's a legitimate 404 from a product intentionally removed).
|
||||
|
||||
Pre-filled suggestion from FTS5 search (same logic as auto-resolution, just surfaced for human confirmation rather than applied automatically).
|
||||
|
||||
### Tab 3: Dead links
|
||||
|
||||
See below — dead link monitoring surfaces here alongside redirects, since they're two sides of the same problem.
|
||||
|
||||
### Tab 4: Create redirect
|
||||
|
||||
Simple form: from path, to path, status code (301/302). For manual one-off redirects (external links, social posts, etc.).
|
||||
|
||||
---
|
||||
|
||||
## Data flow
|
||||
|
||||
```
|
||||
Provider renames product
|
||||
↓
|
||||
ProductSyncWorker → upsert_product/2
|
||||
↓
|
||||
old_slug != new_slug detected
|
||||
↓
|
||||
Redirects.create_auto({from: /products/old, to: /products/new})
|
||||
→ ETS cache invalidated
|
||||
|
||||
─────
|
||||
|
||||
Customer visits /products/old-slug
|
||||
↓
|
||||
BerrypodWeb.Plugs.Redirects checks ETS cache
|
||||
↓ hit
|
||||
301 → /products/new-slug
|
||||
hit_count incremented
|
||||
|
||||
─────
|
||||
|
||||
Bot/customer visits an unknown broken URL
|
||||
↓
|
||||
Plug: no redirect found → pass through
|
||||
↓
|
||||
Router: no match → 404 LiveView
|
||||
↓
|
||||
Analytics.count_pageviews_for_path(path)
|
||||
↓
|
||||
0 hits → likely a bot, discard silently
|
||||
> 0 hits → real broken URL
|
||||
↓
|
||||
BrokenUrls.record(path, prior_hits)
|
||||
↓
|
||||
Attempt FTS5 auto-resolve
|
||||
↓ confident match
|
||||
Redirects.create_auto({..., source: :analytics_detected})
|
||||
↓ no match
|
||||
Left in broken_urls for admin review
|
||||
|
||||
─────
|
||||
|
||||
Admin opens /admin/redirects → broken URLs tab
|
||||
↓
|
||||
Sees sorted list of broken URLs by prior traffic
|
||||
↓
|
||||
Enters destination → creates redirect
|
||||
↓
|
||||
ETS cache warmed → Plug now catches future requests
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
## Dead link monitoring
|
||||
|
||||
Redirects fix *incoming* broken URLs. Dead link monitoring fixes *outgoing* broken links in your own content — nav links, footer links, social URLs, announcement bar targets, rich text content, product descriptions. Two sides of the same problem.
|
||||
|
||||
### Why Berrypod can do this better than external tools
|
||||
|
||||
External link checkers (Ahrefs, Screaming Frog, etc.) crawl your site periodically from the outside. They can't know *why* a link broke or *when* it's about to break. Berrypod knows:
|
||||
|
||||
- Exactly which URLs are valid (it owns the router and the DB)
|
||||
- When products are deleted or renamed (sync events)
|
||||
- Where every admin-configured link is stored (settings keys)
|
||||
|
||||
This means internal links can be validated **instantly and without any HTTP request** — just check the router and DB. External links need an async HTTP HEAD check via Oban.
|
||||
|
||||
### Sources of links in Berrypod
|
||||
|
||||
| Source | Type | When to check |
|
||||
|--------|------|---------------|
|
||||
| Nav/footer links (settings) | Internal or external | On save + when referenced product changes |
|
||||
| Social links (settings) | External | On save + weekly Oban job |
|
||||
| Announcement bar target URL (settings) | Internal or external | On save |
|
||||
| Rich text content (future page editor) | Internal or external | On save + when referenced product changes |
|
||||
| Product descriptions (synced from providers) | Potentially external | After each sync |
|
||||
| Contact page email | Not a URL | Format validation only |
|
||||
|
||||
**Note:** Links rendered *from DB data* (product cards, collection listings) are safe by construction — you only render a link if the product/collection exists. The risk is entirely in user-entered free-text URLs stored in settings or content.
|
||||
|
||||
### Two-phase validation
|
||||
|
||||
**Phase 1: Internal links — instant router + DB check**
|
||||
|
||||
```elixir
|
||||
defmodule Berrypod.LinkValidator do
|
||||
alias BerrypodWeb.Router.Helpers
|
||||
|
||||
def validate(url) when is_binary(url) do
|
||||
uri = URI.parse(url)
|
||||
|
||||
cond do
|
||||
# External URL — queue for async check
|
||||
uri.host != nil -> {:external, url}
|
||||
|
||||
# Internal — check router match
|
||||
true -> validate_internal(uri.path)
|
||||
end
|
||||
end
|
||||
|
||||
defp validate_internal("/products/" <> slug) do
|
||||
case Products.get_product_by_slug(slug) do
|
||||
%{visible: true, status: "active"} -> :ok
|
||||
%{visible: false} -> {:dead, :product_hidden}
|
||||
nil -> {:dead, :product_not_found}
|
||||
end
|
||||
end
|
||||
|
||||
defp validate_internal("/collections/" <> slug) do
|
||||
if Products.category_exists?(slug), do: :ok, else: {:dead, :category_not_found}
|
||||
end
|
||||
|
||||
defp validate_internal(path) do
|
||||
# Check against router for known static paths
|
||||
case Phoenix.Router.route_info(BerrypodWeb.Router, "GET", path, "") do
|
||||
:error -> {:dead, :no_route}
|
||||
_match -> :ok
|
||||
end
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
**Phase 2: External links — async Oban job**
|
||||
|
||||
```elixir
|
||||
defmodule Berrypod.Workers.ExternalLinkCheckWorker do
|
||||
use Oban.Worker, queue: :default, max_attempts: 2
|
||||
|
||||
def perform(%{args: %{"url" => url, "source_key" => source_key}}) do
|
||||
case Req.head(url, receive_timeout: 10_000, redirect: true) do
|
||||
{:ok, %{status: status}} when status < 400 -> :ok
|
||||
{:ok, %{status: status}} -> record_dead_link(url, source_key, status)
|
||||
{:error, _} -> record_dead_link(url, source_key, :unreachable)
|
||||
end
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
Rate limiting: one check per URL per 24 hours. Don't hammer external servers.
|
||||
|
||||
### Event-driven invalidation
|
||||
|
||||
The smart part. Rather than only checking periodically, hook into the events that *cause* dead links:
|
||||
|
||||
**On product deleted/made invisible:**
|
||||
```elixir
|
||||
# After Products.delete_product/1 or hiding a product
|
||||
DeadLinks.scan_stored_links_for_path("/products/#{old_slug}")
|
||||
# Finds any nav/footer/content links pointing to that path → flags them
|
||||
```
|
||||
|
||||
**On product slug change:**
|
||||
The redirect is created automatically (existing plan). Additionally:
|
||||
```elixir
|
||||
# Stored links pointing to the old slug are now stale
|
||||
# Flag them with a "link moved" status + the new destination
|
||||
DeadLinks.flag_moved_links("/products/#{old_slug}", "/products/#{new_slug}")
|
||||
# Admin sees: "Your footer links to /products/old-name — this moved to /products/new-name. Update it?"
|
||||
```
|
||||
|
||||
This is more actionable than just "link is broken" — it tells you where it moved to.
|
||||
|
||||
**On admin saves any content with URLs:**
|
||||
Validate immediately. Internal links checked synchronously (fast). External links enqueued for async check.
|
||||
|
||||
### Schema
|
||||
|
||||
```elixir
|
||||
create table(:stored_links, primary_key: false) do
|
||||
add :id, :binary_id, primary_key: true
|
||||
add :url, :string, null: false # the full URL or path
|
||||
add :source_key, :string, null: false # e.g. "settings.footer_link_1", "nav.about"
|
||||
add :link_type, :string, null: false # "internal" or "external"
|
||||
add :status, :string, default: "ok" # "ok" | "dead" | "moved" | "unchecked"
|
||||
add :http_status, :integer # last HTTP status for external links
|
||||
add :dead_reason, :string # "product_not_found", "no_route", "unreachable", etc.
|
||||
add :moved_to, :string # when status is "moved", the new destination
|
||||
add :last_checked_at, :utc_datetime
|
||||
timestamps()
|
||||
end
|
||||
|
||||
create unique_index(:stored_links, [:url, :source_key])
|
||||
create index(:stored_links, [:status])
|
||||
create index(:stored_links, [:link_type])
|
||||
```
|
||||
|
||||
### Admin UI: Dead links tab
|
||||
|
||||
Table of all dead/moved/unchecked stored links, sorted by status (dead first, then moved, then unchecked).
|
||||
|
||||
Columns: source (where the link is — "Footer", "Nav", "Announcement bar"), URL, status badge, last checked, action.
|
||||
|
||||
Actions:
|
||||
- **Dead:** "Edit" (opens the relevant settings section pre-focused on that field) — or "Ignore" if intentional
|
||||
- **Moved:** "Update link" one-click to replace old URL with the new destination in the source setting
|
||||
- **Unchecked:** "Check now" to trigger immediate validation
|
||||
|
||||
Dashboard integration: a small badge on the admin dashboard card ("3 dead links") to draw attention without being annoying. Cleared when all are resolved or ignored.
|
||||
|
||||
### Weekly Oban cron job
|
||||
|
||||
Re-check all external links stored in `stored_links`. Internal links don't need periodic re-checking — they're validated on demand and on data-change events, which is more efficient.
|
||||
|
||||
```elixir
|
||||
# In Oban crontab
|
||||
{"0 3 * * 1", Berrypod.Workers.WeeklyExternalLinkCheckWorker}
|
||||
```
|
||||
|
||||
The weekly job enqueues one `ExternalLinkCheckWorker` job per external stored link, with rate limiting.
|
||||
|
||||
### What it deliberately doesn't do
|
||||
|
||||
- **Doesn't crawl rendered HTML** — too fragile, too slow. We work from structured data (settings keys, content blocks), not parsed HTML.
|
||||
- **Doesn't check links in transactional emails** — those are templates, not user content.
|
||||
- **Doesn't validate email addresses** — format check only, not SMTP validation (too invasive).
|
||||
- **Doesn't check links in product images** — image URLs are managed by the Media pipeline, not free-text.
|
||||
|
||||
### Relationship to redirect system
|
||||
|
||||
| Problem | Solution |
|
||||
|---------|----------|
|
||||
| Visitor hits a broken URL | **Redirect** — 301 to new location |
|
||||
| Your own content links to a broken URL | **Dead link fix** — update the link in your content |
|
||||
| Product renamed — old URL works | Redirect created automatically |
|
||||
| Product renamed — your nav still says old URL | Dead link flagged as "moved" with suggestion |
|
||||
|
||||
They complement each other. The redirect preserves SEO and visitor experience for external links you can't control (social posts, other websites linking to you). The dead link monitor fixes links you *can* control — your own navigation, content, and settings.
|
||||
|
||||
---
|
||||
|
||||
## Implementation notes
|
||||
|
||||
**Slug change detection is safe to add with no behaviour change** for products that don't change slug. The `on_conflict: :nothing` insert ensures idempotency across repeated syncs.
|
||||
|
||||
**The FTS5 confidence threshold** should be tuned conservatively at first. An incorrect auto-redirect (wrong product) is worse than no redirect. Admin review catches the gaps.
|
||||
|
||||
**ETS cache invalidation** needs to happen on: redirect created, updated, deleted. Simple `GenServer` or `:persistent_term` approach — at the scale of a single-tenant shop, the full redirect table easily fits in memory.
|
||||
|
||||
**Redirect chains** (A → B → C) should be detected and flattened on creation. If a new redirect's `to_path` is itself an existing `from_path`, follow it and set the new redirect's `to_path` to the final destination. Avoids multi-hop redirects.
|
||||
|
||||
**Status code guidance:**
|
||||
- `301` Permanent — use for slug changes and deleted products. Tells Google to update its index.
|
||||
- `302` Temporary — only for sales/temporary campaigns. Tells Google to keep the original URL indexed.
|
||||
|
||||
---
|
||||
|
||||
## Files to create/modify
|
||||
|
||||
- Migration — `redirects` and `broken_urls` tables
|
||||
- `lib/berrypod/redirects/redirect.ex` — schema
|
||||
- `lib/berrypod/redirects/broken_url.ex` — schema
|
||||
- `lib/berrypod/redirects.ex` — context: `lookup/1`, `create_auto/1`, `create_manual/1`, `warm_cache/0`, `invalidate_cache/1`, `increment_hit_count/1`, `list_broken_urls/0`, `record_broken_url/2`
|
||||
- `lib/berrypod_web/plugs/redirects.ex` — new Plug
|
||||
- `lib/berrypod/products.ex` — slug change detection in `upsert_product/2`
|
||||
- `lib/berrypod_web/live/shop/error.ex` — hook analytics query on 404
|
||||
- `lib/berrypod_web/live/admin/redirects_live.ex` — new LiveView (3 tabs)
|
||||
- Router — `/admin/redirects` route, ETS cache warm on startup
|
||||
- Admin nav — new sidebar link
|
||||
|
||||
## Tests
|
||||
|
||||
- `upsert_product/2` with title change creates redirect automatically
|
||||
- `upsert_product/2` with no title change does not create redirect
|
||||
- Redirect Plug: matching path → 301, no match → passthrough
|
||||
- Redirect Plug: ETS cache hit (no DB call)
|
||||
- 404 handler: path with analytics history → broken_url record created
|
||||
- 404 handler: path with no analytics history → nothing recorded
|
||||
- FTS5 auto-resolve: confident match → redirect created; no match → broken_url pending
|
||||
- Redirect chain flattening: A→B, new B→C → stored as A→C
|
||||
- `hit_count` incremented on each redirect fire
|
||||
Reference in New Issue
Block a user