shoppy-index-configuration

Version-controlled home for the Meilisearch index settings used by the Shoppy search stack — the products index and the categories index — plus the research that drives changes to them.

Shoppy's search is powered by Meilisearch: product-sync copies the catalog from PostgreSQL into Meilisearch and applies the settings defined here, and the BFF (bff) serves search queries against those indexes. This repo is the single source of truth for how those indexes are configured, so changes to search behaviour (ranking, filters, facets, sorting, typo tolerance, hybrid/embedding setup) start here.

Repository layout

shoppy-index-configuration/
├── shoppy-products.json    # Meilisearch settings template for the products index
├── shoppy-categories.json  # Meilisearch settings template for the categories index
├── docs/
│   └── research-brief-meilisearch-search-relevance.md  # singular vs plural relevance findings + recommendation
└── AGENTS.md               # workflow conventions (labels, branches, commits, PRs)

File	Applies to	Supersedes
shoppy-products.json	Products index (e.g. shoppy-test-v9, fresh index per run)	shoppy-test.json (retired)
shoppy-categories.json	Categories index (e.g. shoppy-categories-v2)	shoppy-categories.json (retired)

This repo is the canonical source for the index settings. They used to be kept as loose files in the indexes/ directory of the shoppy monorepo; that monorepo is retired, so its copies are stale and must not be used or edited. Keeping them here under version control lets settings be reviewed, changed, and replayed (a fresh index is created per run, so settings changes apply automatically at the next sync).

Index settings overview

Both files are Meilisearch index-settings documents (the payload for the index settings API). Highlights:

Products (shoppy-products.json)

Setting	Value
searchableAttributes	name, description, brand
filterableAttributes	categories.level0, categories.level1, storeId, brand, categoryListingRank, price.tags
sortableAttributes	price.amountAsCents, price.comparativePricing.priceAsCents, categoryListingRank, is_special
rankingRules	words, typo, proximity, attribute, sort, exactness
prefixSearch	indexingTime (query terms match words starting with them)
proximityPrecision	byWord
typoTolerance	enabled; 1 typo ≥ 3 chars, 2 typos ≥ 5 chars; disabled on a, an, and, the
faceting	maxValuesPerFacet: 100, facet values sorted alpha
pagination	maxTotalHits: 1000
embedders.mxbai	REST embedder via LM Studio (http://10.1.5.170:1234/v1/embeddings, text-embedding-nomic-embed-text-v2-moe), template: "a product from {brand} with a description of {name}"

Categories (shoppy-categories.json)

Setting	Value
searchableAttributes	name
filterableAttributes	parent_id, id
sortableAttributes	name
rankingRules	words, typo, proximity, sort, exactness
prefixSearch	indexingTime
embedders.mxbai	REST embedder via LM Studio (http://10.1.1.173:1234/v1/embeddings, same model), template: "a product category called {name} with a description of {description}"

Where the settings are consumed

product-scraper ──▶ product-ingestor ──▶ products-database (PostgreSQL)
                                             │
                        shoppy-embeddings ───┤  (vector embeddings via LM Studio)
                                             ▼
                                  product-sync (CLI)
                                             │  reads MEILISEARCH_INDEX_CONFIG_PRODUCTS /
                                             │  MEILISEARCH_INDEX_CONFIG_CATEGORIES
                                             ▼
                             Meilisearch (products + categories indexes)
                                             │  settings come from THIS repo
                                             ▼
                                       bff (GraphQL BFF)
                                             │  search_products, store filter, sort, facets
                                             ▼
                             shoppy-website / shoppy-dashboard

• product-sync applies these settings when it creates an index. It resolves the settings files via MEILISEARCH_INDEX_CONFIG_PRODUCTS / MEILISEARCH_INDEX_CONFIG_CATEGORIES, which default to this repo's shoppy-products.json / shoppy-categories.json; set them only to override that deliberately.
• bff queries the indexes — search_products with sort from get_sort_by_option (SEARCH default price_low_to_high → sort: ["price.amountAsCents:asc"]), rankingScoreThreshold: 0.7, facets on brand/categories.level0/categories.level1, and a storeId filter merged from the shoppy-stores header.

Resolved: Meilisearch blocked private-IP embedder calls

Applying settings to a fresh index used to fail the settingsUpdate task with vector_embedding_error. Since the v1.34 security fix (2026-01-26), Meilisearch refuses outbound requests to non-global IPs by default, and both embedder URLs tracked here are RFC-1918 private (10.1.5.170, 10.1.1.173). The block was in the Meilisearch instance configuration — not in these settings files, and not in the LM Studio servers.

Resolved. The instance now allows the private range:

MEILI_EXPERIMENTAL_ALLOWED_IP_NETWORKS=10.0.0.0/8 meilisearch
# or: meilisearch --experimental-allowed-ip-networks "10.0.0.0/8"

Verified against the live instance on 2026-09-17: shoppy-test-v9 has the mxbai embedder registered, a settings PATCH from these files succeeds, document additions compute embeddings, and a hybrid search returns 200. References: meilisearch#6152, meilisearch#6223.

Search relevance: known issue & applied fix

docs/research-brief-meilisearch-search-relevance.md documents why singular queries (banana, apple) failed to surface actual produce while plurals (bananas, apples) worked, with a live A/B repro. Key takeaways:

• No stemming in Meilisearch: banana and bananas are distinct tokens. prefixSearch: "indexingTime" means banana still matches plural-named products, so matching isn't the problem — ordering is.
• The attribute ranking rule was missing from the products settings — now added: shoppy-products.json ships rankingRules: [words, typo, proximity, attribute, sort, exactness], and the live products index matches. shoppy-categories.json still omits it. Without it, a match at the start of name is worth no more than a match buried in name/description.
• The sort rule dominates single-word queries. With the BFF's default price asc, the sort rule (order 3) decides everything once words/typo/proximity tie — so the cheapest product mentioning "banana" (e.g. a $0.80 yoghurt pouch) beats actual bananas.
• Hybrid search is inert: the BFF never sends a hybrid parameter, so the mxbai embedder is unused at query time. Embeddings themselves work again (see Resolved).

Recommended fix — implemented for products (shoppy-products.json, issue #3); still pending for categories:

- "rankingRules": ["words", "typo", "proximity", "sort", "exactness"]
+ "rankingRules": ["words", "typo", "proximity", "attribute", "sort", "exactness"]

• Empirically gets actual bananas/apples into the top 1–3 for singular queries even with the app's price-asc default sort (e.g. banana/coles → Bananas #1).
• Do not move exactness before sort (penalises plural-named produce; tested worse) and do not add banana↔bananas synonyms (no measurable benefit given prefixSearch).
• In force for products; applies automatically when the products index is next created (or after a settings re-apply) — no data changes needed. Optionally mirror in shoppy-categories.json for consistency.

See the research brief for the full findings, A/B data, and validation steps.

Making changes

Settings changes are reviewed like any other code change — see AGENTS.md for conventions:

• feature / bug / enhancement changes are issue-driven: branch feature|bug/issue-<N>/<slug>, commits issue-<N>: <summary>, PR closes the issue.
• documentation / chore changes skip issues and are date-driven: branch documentation|chore/<YYYY-MM-DD>/<slug>, commits <type>: <summary>.
• PRs are squash-merged; merging is the repo owner's decision.

Related repositories

Repo	Role
shoppy (retired monorepo)	Former home of these settings (indexes/, e.g. shoppy-test.json); stale — do not use
product-sync	Applies these settings while syncing PostgreSQL → Meilisearch
bff	GraphQL BFF that queries the indexes (search params, store filter, sort)
products-database	Authoritative PostgreSQL schema the catalog is synced from
product-ingestor / product-scraper	ETL + scrapers feeding the database
shoppy-embeddings / shoppy-text-embedder	Vector embedding services (LM Studio backend)
shoppy-website / shoppy-dashboard	Frontends consuming the BFF