# BuyDubai & Dalia — Updated Plan, Gate Proofs & `S00/M1` Baseline Response to INVRT (Mark Gullickson)

**Date:** 24 October 2026  
**From:** Anthony Booth / Articulate (`buydubai.io`, `rentdubai.io`, Dalia Knowledge Service)  
**To:** Mark Gullickson ("Gully" / INVRT)  
**Re:** Integration of 24 Oct 2026 INVRT Handoff Pack (`BuyDubai-Dalia-Implementation-Handoff.pdf`, `Technical-Contracts.md`, `Build-Stories.md`) with our Facts Data Cube, Master Registers, Website Parity Gate, and Read-Only `S00 / M1` Baseline

---

## 1. Executive Summary — What Has Changed & What Is Proven (`[V]`) Today

We have integrated your 24 October 2026 delivery (`BuyDubai-Dalia-Implementation-Handoff.pdf`, `Technical-Contracts.md`, `Build-Stories.md`) into our master execution plan (`PROJECT-buydubai-cube-and-community-pages.md` §10).

Before handing over to the `S00–S21` story build loop, we completed a 10-turn builder + independent frontier auditor (`Grok`) verification sprint on Studio (`~/ClaudeWork/cube-stage/`) that resolves the foundational data-truth, identity-crosswalk, and website-parity blockers in your `M1`, `M2`, `M3`, and `M7` milestones:

1. **Master Identity & Metric Registers (`M2` / `M3`) — `independently reviewed in staging [V]` (`04-grok.md`, `08-grok.md`)**:
   - Deterministic registers built and gated at `~/ClaudeWork/cube-stage/registers/out/`:
     - `communities.csv` (`CM*`), `community_alias.csv`, `page_dir_map.csv` (`61` mapped site folders), `page_dir_retired.csv` (`3` retired DLD-area-only folders: `al-barsha`, `al-jaddaf`, `ras-al-khor`), `page_dir_unresolved.csv` (`0` rows).
     - `buildings.csv` (`BL*`), `building_alias.csv` (deterministic `pf_seg1 + all-time dld_areas` crosswalk; all 13 prior LLM-guessed project mappings dropped and replaced).
     - `metrics.csv` (`55` cube metric families across `192` `(metric, slice_type)` pairs; `verify_registry_formulas.py` passes **`27/27`** checks).
     - `validate_registers.py` outputs **`RESULT: ALL CHECKS PASS`** (`0` gaps).
2. **Deterministic Facts Data Cube (`M1` / `M3` / `M7`) — `independently reviewed in staging [V]` (`06-grok.md`)**:
   - Rebuilt `~/ClaudeWork/cube-stage/RealEstate/facts/facts.parquet` + `cube_meta.json` (`9,645,491` rows, `1,009,660` suppressed, `cube_hash = 74bc4c527cb8716a6d41bda00b7a7e64da4b0cc1703b2b1b37179ad56e65b3a6`).
   - Verified across two independent back-to-back builds (`SELECT * FROM b1 EXCEPT ALL SELECT * FROM b2 = 0 rows` in both directions, `0` NULL `fact_id`, `0` duplicate `fact_id`, `0` unsuppressed NULL `value`, `0` unsuppressed `lr_*` with `n < 10`).
3. **Website Parity & Release Gate Proven on `/town-square/` (`M7` / `S13` / `S19`) — `independently reviewed in staging [V]` (`09-grok-gate.md`)**:
   - Confirmed Anthony's architectural rule: *no cube was used for the legacy site build; every builder must consume `factlib.py` at build time.*
   - Rebuilt all `14` pages in `/town-square/` (`index.html`, `insights`, `traffic`, `yield`, `budget`, `towers`, `parking`, `collections`, and `6` bedroom sub-pages) from the Turn 6 cube (`cube_hash 74bc4c52...`).
   - Independently verified by Grok (`09-grok-gate.md`):
     - `community_eval.py town-square --local site-after`: **`91.9 / 100 PASS`** (`Data 35/35`, `Content 26/30`, `Place 15.9/20`, `Quality 15/15`, `41` internal links / `0` broken, `15/15` JVC hub sections).
     - `lint_pages.py site-after facts/cube_meta.json`: **`0` problems across all 14 pages** (`14/14` pages carry a valid `facts-manifest` matching `cube_hash 74bc4c52...`).
4. **Execution Role Split (`Builder + Frontier Auditor`)**:
   - **Builder (`GLM 5.3 Flash` / Claude)**: Executes bounded code/SQL slices (`S00–S21`) and community rebuilds.
   - **Independent Frontier Auditor (`Grok`)**: Never writes application code; audits diffs, runs deterministic fixtures (`F01–F36`), and gates promotion along your 6-rung status ladder (`source-present` $\rightarrow$ `implemented/wired` $\rightarrow$ `locally tested` $\rightarrow$ `independently reviewed` $\rightarrow$ `integrated` $\rightarrow$ `live`).

---

## 2. Read-Only `S00 / M1` Baseline Package (Peninsula Three Trace & Complete SQL Locations)

Per `Build-Stories.md` (`S00`) and `09-grok-gate.md` §3.5, all baseline inspections below were executed **strictly read-only** (`SELECT` / local file inspection only — zero database writes, zero `TRUNCATE`).

### 2.1 Peninsula Three Root-Cause Trace (`M1` / `F06` / `F35`)

Your handoff (`BuyDubai-Dalia-Implementation-Handoff.pdf` p. 3 & `Build-Stories.md` `S00`) flagged two specific contradictions on **Peninsula Three**:
1. **12-month sales count**: Website reported **`156`** vs `onto_c.tower_card` reporting **`35`** DLD sales.
2. **Completion status**: Website/catalogue reported **`"Completed"`** vs DLD Project Register reporting **`"ACTIVE"` at `93.1%`**.

Here is the exact read-only trace from `registers/out/buildings.csv` and `facts.parquet` (`cube_hash 74bc4c52...`):

| Dimension | Read-Only Baseline Value | Root Cause of Prior Discrepancy | Contract Resolution (`S02`, `S04`, `S13`) |
|---|---|---|---|
| **Canonical Identity (`registers/out/buildings.csv`)** | `building_id = BL001215`, `name = Peninsula Three`, `pf_slug = business-bay-peninsula-peninsula-three`, `community_id = CM0020` (`business-bay`), `dld_area = BUSINESS BAY`, `dld_project = PENINSULA THREE`, `dld_building = PENINSULA THREE`, `match_score = 0.805 (high)` | `onto_c` and `onto_g` used separate slug/project keys without a single canonical `BL*` building crosswalk. | Use `registers/out/buildings.csv` (`BL001215`) + `building_alias.csv` as the reversible crosswalk (`S02`, `F01–F03`) without `TRUNCATE` rebuilds. |
| **12-Month Sales (`156/158` vs `35`)** | In `facts.parquet` (`slt='tower'`, `sl='business-bay-peninsula-peninsula-three'`, `sub='homes'`, `beds='all'`):<br>• `period='ttm'`, `st='all'`: **`158` sales** (`AED 283,314,360` volume; was `156` at the earlier static site build snapshot)<br>• `period='ttm'`, `st='ready'`: **`137` sales** (`AED 248,484,360`)<br>• `period='ttm'`, `st='offplan'`: **`21` sales** (`AED 34,830,000`)<br>• `period='2025'`, `st='all'`: **`132` sales** (all `132` `st='offplan'`, `0` ready)<br>• `2026-Q1`: `50` sales (`49` ready) · `2026-Q2`: `49` sales (`47` ready) · `2026-Q3`: `38` sales (`37` ready) | The website (`156` at build time, `158` in current cube) reported `st=all` (`ready + offplan`) TTM residential sales matched via `pf_units` + `PENINSULA THREE`. Meanwhile `onto_c.tower_market` (`35` sales) queried an un-reconciled Supabase slice before the `2026` `dld_live` ready-transfer rows and `pf_units` crosswalk were joined. | Both `/api/dalia` (`get_metrics`) and static page builders (`factlib.py`) read the same `MetricObservation` cohort (`st='all'` `158` vs `st='ready'` `137` vs `st='offplan'` `21`, `period='ttm'`) stamped with `cube_hash` (`F06–F10`, `F35`). |
| **Completion Status (`"Completed"` vs `"ACTIVE"` `93.1%`)** | • **Physical handover / market activity**: `0` ready sales in 2025 $\rightarrow$ `137` ready title-deed sales in TTM 2026 + `449` Ejari leases (`2025–2026`) + `136` active rental ads (`rbld=business-bay/peninsula-three`).<br>• **DLD Project Register (`dld_projects`)**: Still records escrow project `PENINSULA THREE` as `"ACTIVE"` at `93.1%` completion. | Physical handover and title-deed registration began in Q1 2026, but DLD's construction escrow register lags final completion certification (`93.1% ACTIVE`). | `CanonicalProjectRecord` (`Technical-Contracts.md` §3.1, `F03`) surfaces **both** `catalogue_status: "Completed"` (backed by `137` ready sales + `449` leases) and `dld_register_status: "ACTIVE"`, `dld_completion_pct: 93.1`, plus `reconciliation_notes` explaining the escrow-vs-handover lag. |

### 2.2 Complete Untruncated SQL Definitions Located on Studio (`S00`)

Your `Build-Stories.md` (`S00` & §6.1) requested the complete, untruncated SQL bodies for `mart.tower_12m` and the 7 truncated RAG/ontology functions. Here are their exact locations on Studio (`~/ClaudeWork/RealEstate/`), ready for read-only archival into `S00`:

1. **`mart.tower_12m` (Materialized View & Grain Rules)**:
   - **File**: `~/ClaudeWork/RealEstate/warehouse/mart/mart.sql` (lines `166–195`, plus `mart.sales_by_grain` at lines `143–164` and `mart.freshness` at lines `200–204`).
   - **Exact Formula Verified**: Aggregates `mart.sales` (`is_market_sale AND is_residential AND psf_ok`, `txn_date > current_date - interval '12 months'`) and `mart.leases` (`is_residential AND psf_ok`, `start_date > current_date - interval '12 months'`) separately at grain `(project_key, unit_class, area_key)`, then `FULL JOIN`s 1:1. `sale_psf` is computed over `filter (where not is_offplan)` (`ready_n >= 5` for `publishable_sale`), `rent_psf` and `median_rent` over `filter (where is_new)` (`new_n >= 5` for `publishable_rent`), and `gross_yield_pct = 100 * l.rent_psf / nullif(s.sale_psf, 0)` when `ready_n >= 10 AND new_n >= 10`.
2. **`rag_search` (v1 Hard Filter) & `rag_kwquery` / `rag_refresh_term_df`**:
   - **File**: `~/ClaudeWork/RealEstate/corpus/sql/003_rag.sql` (lines `46–135`).
   - **Behaviour**: Hard-filters `rag_chunk` by `community = any(r.communities) OR r.communities = '{}'`, `building = any(r.building_slug) OR r.building_slug = '{}'`, `min_tier`, and `after`. Because empty arrays `{}` pass the filter, untagged chunks leak across communities (`Build-Stories.md` `S14` / `F17`).
3. **`rag_search2` (RRF `k=60` + SQL Priors) & `rag_search` Delegation Wrapper**:
   - **File**: `~/ClaudeWork/RealEstate/corpus/sql/003c_alt_rag_search2.sql` (lines `25–113`).
   - **Behaviour**: Treats `community` (`+0.006` match / `-0.003` mismatch) and `building` (`+0.008` match / `-0.002` mismatch) as soft score boosts rather than hard pre-filters.
4. **`rag_retag_docs` & `rag_search_v2`**:
   - **File**: `~/ClaudeWork/RealEstate/corpus/sql/003c_rag_fixes.sql` (lines `19–115`).
5. **`onto_g` Tool Functions (`resolve`, `neighbors`, `card`, `near`, `near_entity`, `docs`, `metrics`)**:
   - **File**: `~/ClaudeWork/RealEstate/ontology/grok/sql/005_tools.sql` (lines `1–65`).
6. **Remaining In-Database Functions (`onto_c.tower_card`, `rag_retag`, local `hybrid_search`)**:
   - Exported read-only via `SELECT pg_get_functiondef(oid)` in `S00` before any `M2/M5` migration runs.

---

## 3. Answers to Your 8 Open Implementation Choices (`Build-Stories.md` §6)

| # | `Build-Stories.md` §6 Open Question | Agreed Decision & Owner |
|---|---|---|
| **1** | **Complete mart formulas & 7 truncated SQL bodies** | **Resolved (§2.2 above)** — full DDL located in `warehouse/mart/mart.sql` and `corpus/sql/003*.sql`; remaining DB-only functions exported read-only via `pg_get_functiondef` in `S00`. (**Owner: Builder `S00` / Grok Gate**) |
| **2** | **Chosen RAG variant & production topology (`Studio` vs `Supabase dubai-data`)** | **Canonical pre-filtered RAG (`S14`, `F17`) + Topology Latency Budget**: Local `rag-svc` on Studio (`bge-m3` MPS + `bge-reranker-v2-m3`) runs in `~80–250 ms`, whereas Supabase `dubai-data` runs on **Small compute (2 GB RAM)** where unfiltered `rag_search` / `rag_search2` takes **`1.4–2.9s`**. In `S14`, we replace the 3 competing `rag_search*` variants with one canonical SQL function that **pre-filters by authorised `doc_ids` (`onto_g.doc_link`) before top-k ranking** (and rejects empty-array `{}` leakage unless explicitly global), cutting both cross-entity contamination and Small-compute scan cost. (**Owner: Builder `S14` / Anthony**) |
| **3** | **Authenticated client/session mapping (`consumer_public` vs `paul_agent`)** | `/api/dalia` derives `mode` (`consumer_public` vs `paul_agent`) and `client_id` from server-verified authentication headers/session tokens — never from unverified client payload fields (`F27`, `F28`). (**Owner: Builder `S15` / Gully**) |
| **4** | **Source rights, robots & retention policy (`corpus_source`)** | Tier A/B official public datasets (DLD, RERA, Ejari, DM, KHDA) and first-party BuyDubai guides are `public_citable`. Third-party listings/videos carry explicit `rights_class` (`internal_assist_only` vs `citable_with_attribution`). Google Places hero photos are cached locally as `.jpg` files with mandatory `Photo: <authorDisplayName>, Google Maps` attribution (`09-grok-gate.md` Fix E). (**Owner: Anthony / Builder `S01`**) |
| **5** | **Current `buydubai_dalia_counts` RPC & per-mode spend config** | Keep `buydubai_dalia_counts` (20-question cap + lead-capture prompt) strictly for `consumer_public` (`F29`). In `paul_agent` mode, disable the 20-question cap and lead-capture prompt, and enforce a per-turn/daily USD token-spend cap (`F27`, `F36`). (**Owner: Builder `S15`, `S20`**) |
| **6** | **Unit/layout/listing coverage & freshness (`verified_available_inventory` vs `historical_research_candidates`)** | Current local data holds DLD transactions, Ejari leases, and `pf_units` (`4,135–5,390` records), not a live verified MLS offer feed. Accordingly, `find_properties` (`S06–S07`) returns `verified_available_inventory = []` unless backed by a fresh (`<= 7d`) verified offer feed, and labels DLD/Ejari/cube matches as `historical_research_candidates` (`F12–F15`). Unsupported areas (e.g. Dubai Marina / JLT when only BB/JVC/staged communities are loaded in a tool slice) return `coverage_status = "coverage_missing"` rather than silently substituting Business Bay or JVC (`F15`). (**Owner: Builder `S03`, `S06–S07`**) |
| **7** | **Paul's first channel** | **Web brief console + shareable client brief (`S16`, `F31`) first**, followed by the Lampline WhatsApp bridge (`S17`, `F32`) calling the exact same `/api/dalia` backend contract. (**Owner: Anthony / Gully `S16–S18`**) |
| **8** | **Test runners & quality/performance thresholds** | Existing deterministic gates on Studio (`validate_registers.py`, `verify_registry_formulas.py`, `test_cube.py`, `community_eval.py >= 85`, `lint_pages.py == 0`) are joined by `tests/dalia_fixtures_runner.py` enforcing your `F01–F36` contract fixtures at every story gate. (**Owner: Builder + Grok Gate `S00–S21`**) |

---

## 4. Next Immediate Actions

1. **Publish `/town-square/` (`14` pages) & Apply `takedown_redirects.csv` (`38` URLs)** on `buydubai.io` (`/motor-city/` remains live until rebuilt per `10-grok.md` §2 because it has `notshown=0, factids=27`; clean HEAD + `/town-square/` publish vs full stale-hash rebuild pending Anthony's choice in `10-grok.md` §4).
2. **Execute Reduced-Cost Hero Photo Pass (`hero_photos_reduced.py`)** (gated `PASS` by Grok in `10-grok.md` §1: `25` buildings across `5` communities = `~$0.525` total list cost [`$0.021/building`], local `.jpg` byte caching + `Google Maps` attribution, `$2.00` runtime hard cap).
3. **Run Stories `S00` $\rightarrow$ `S05` (`M1–M3`)** using the Fast Builder (`GLM 5.3 Flash` / Claude) + Independent Frontier Auditor (`Grok`) loop, starting with the formal `S00` read-only snapshot artifact and `S02` non-destructive Postgres crosswalk (`onto_g` $\leftrightarrow$ `registers/out/buildings.csv` & `communities.csv`).