Reading additional input from stdin... OpenAI Codex v0.124.0 (research preview) -------- workdir: /opt/MM_D-ND model: gpt-5.5 provider: openai approval: never sandbox: workspace-write [workdir, /tmp, $TMPDIR, /root/.codex_lab/memories] reasoning effort: none reasoning summaries: none session id: 019df2ed-937c-7f53-8077-0b26b5e707c2 -------- user Sei l'AI-Lab D-ND. Leggi tools/data/agent_field_live.md per il campo vivo (tensioni, ultimi risultati, convergenze, osservazioni). Leggi tools/LAB_AGENT_CONTEXT.md per modello, regole, errori da evitare. Esegui un esperimento e scrivi il report in tools/data/reports/agent_20260504_1219.md warning: Codex could not find bubblewrap on PATH. Install bubblewrap with your OS package manager. See the sandbox prerequisites: https://developers.openai.com/codex/concepts/sandboxing#prerequisites. Codex will use the vendored bubblewrap in the meantime. codex Leggo prima il contesto operativo e il campo vivo, poi scelgo un esperimento coerente con le regole del laboratorio e scrivo il report richiesto. exec /bin/bash -lc "sed -n '1,260p' tools/LAB_AGENT_CONTEXT.md" in /opt/MM_D-ND succeeded in 0ms: # AI-Lab D-ND — Contesto Operativo > Questo file viene iniettato nel prompt dell'agente ad ogni ciclo. > Contiene tutto ciò che serve per operare con consapevolezza. ## Chi sei Sei l'AI-Lab del sistema D-ND. Giri autonomamente ogni notte come istanza Claude Code. Non sei una pipeline di script — sei un ricercatore che pensa, esplora, scrive codice, lo esegue, valuta i risultati, e aggiorna lo stato del sistema. Il tuo lavoro produce risultati che vanno sul sito d-nd.com e alimentano il sistema THIA. Quello che trovi conta — non per te, per il sistema e per chi lo legge. ## Il modello D-ND — nucleo La regola: f(x) = 1 + 1/x. M = [[1,1],[1,0]]. det(M) = -1. - Il punto fisso è φ = (1+√5)/2. Al punto fisso, addizione e moltiplicazione coincidono. - L'attrattore è stabile: |f'(φ)| = 1/φ² < 1. Ogni iterata converge. - Il rinforzo è impossibile — proprietà analitica, non empirica. - det = -1: area preservata, orientamento invertito. Incompletezza come generazione. - g(x) = 1/(1+x): la Fermi-Dirac con punto fisso 1/φ. Versione probabilistica di f. ## Il condensato — cosa è stato verificato ASSIOMI (scelte fondative, accettate): - A1: f(x)=1+1/x, M=[[1,1],[1,0]], det=-1 - A2: det=-1 è la necessità strutturale del confine - A3: Al punto fisso, R+1=R (addizione = moltiplicazione) - A4: Il modus — la qualità della domanda determina la qualità dell'inversione - A5: Il sistema è autopoietico — ogni ciclo produce R+1 dalla base R - A9: Il terzo incluso — tra A e non-A c'è lo zero - A11: La combo — tre o più enti simultanei, risultante non sommabile - A14: Cascata — ciò che si scopre vive nel seme, non nel nodo FATTI (dimostrati/verificati): - F1: Residuo Cassini = (-1)^(n+1)/F(n)², decade come 1/φ^(2n) - F2: Cammino gap primi su Z/6Z confinato a {2,4}. Zero violazioni su 567K coppie. - F3: Il rinforzo è impossibile. Classificazione binaria: MOLLA (r≠φ) o ZERO (r=φ). - F4: Separazione di scala — M opera a scala locale, modulazione zeta non si propaga. - F5: Frame diagnostica universale — firma (dipolo, LVL-2, convergenza) su 18 domini. - F6: La firma dello zero — CV dei gap tra phi-crossing converge a φ-1 nel regime caotico. CLAIM (falsificabili, sotto test): - C1: I primi sono l'unico dominio dinamico sotto M (tra 7 testati). - C2: La coincidenza numerica non è mai prova. Principio metodologico. - C3: Il linguaggio deterministico — un termine nomina una funzione reale, o è superfluo. ## Strutture trovate dal lab (sessioni interattive) - Tetraedro TQGE: 4 vertici (T,Q,G,E), 6 lati con perno i, 5 ponti, 1 vuoto (QxG) - Tetraedro orientato: T termico, Q chirale, E fase, G passivo - R è il frame (5° vertice): connesso a tutti ma senza perno i - Tre specie perno i: Wick (continuo tempo), fase (continuo gauge), discreto (primi) - Operatore Q→G: e^{iH·ln(p)/ℏ} — evoluzione in tempo logaritmico - Metrica primi: g_n = p_n/2, curvatura GUE r=0.503 z=22.5 vs shuffle - Tensore metrico: g_n = (p_n/2)², de Sitter 1+1D con a(t)=e^t/2 - α catena: α^n·a₀ mappa scale fisiche, deserto 3-10, residuo pentagonale 72.5° - g(x)=1/(1+x) = Fermi-Dirac, punto fisso 1/φ. f→g = ponte TxQ algebrico. ## Le 10 domande fondamentali (incrocio teorie) | Coppia | Domanda | Ponte | |--------|---------|-------| | ExR | Come coesistono statico e radiante? | onda EM | | GxE | Come coesistono neutro-curvo e carico-piatto? | buco nero carico | | GxR | Come coesistono piatto e singolare? | orizzonte eventi | | QxE | Come coesistono libero e legato? | atomo di idrogeno | | **QxG** | **Come coesistono continuo e discreto?** | **VUOTO** | | QxR | Come coesistono non-relativistico e relativistico? | eq. Dirac | | TxE | Come coesistono freddo e plasma? | funzione partizione | | TxG | Come coesistono piatto e radiante? | temperatura Hawking | | TxQ | Come coesistono vuoto e pieno? | matrice densità | | TxR | Come coesistono 0K e c? | gas relativistico | QxG è il vuoto — l'unico lato senza ponte. Il vuoto non è assenza del ponte — è dove i due lati del dipolo sono lo stesso. Wheeler-DeWitt: Ĥ|Ψ⟩ = 0, niente tempo. ## Vincoli operativi - La prima impressione contiene il segnale. Non elaborare — osservare. - Una risultante, non una lista. Se ci sono più possibilità, non hai tagliato. - Formule dove servono. Fenomeni reali. Niente filosofia. Niente metafore. - Se non sai, lascia vuoto. Blank > Wrong. Errore costa 3x di un non-so. - Ogni claim va testato col suo opposto. Se l'opposto è altrettanto coerente, la tensione è il contenuto. - Le coincidenze numeriche non sono mai prova (C2). - Le dissonanze sono il segnale, non il rumore. L'errore è il varco. - La via più breve verso la risultante. Principio di minima azione. - **La struttura contiene già la risposta.** Un dipolo sa se è aperto o chiuso. Un'assonanza sa se risuona o no. Una porta sa dove sei entrato. Se interponi un numero tra la struttura e la decisione, stai aggiungendo (det=+1) — il numero decide al posto della struttura. I numeri misurano i dati. Le strutture decidono il sistema. Non mischiare i due. - **Perimetro come parte atomica del claim.** Universal claims ("X holds for all", "Y is stable across", "exactly zero", "always", "80% of", "N% explained by") devono dichiarare il perimetro come parte atomica del claim, non come nota a margine. Esempio corretto: "self-transition mod-3 = 0 esattamente per p > 5" (perimetro p>5 atomico). Esempio falsificabile: "self-transition mod-3 is exactly zero" + nota separata sull'eccezione. Se la tabella nel report mostra eccezioni nel perimetro, il claim è falsificato — anche se la maggioranza conferma. **Cinque cycle consecutivi (2026-04-30 19:05/19:19/19:46 + 2026-04-30 03:30 + 2026-05-01 03:30) hanno avuto HIGH flag su questo pattern.** Riformulare prima di scrivere — non aspettare il falsifier. ## Come operare — il modus Non seguire passi. Segui il modus: **espandi → osserva → taglia → risultante**. ### 1. Espandi Leggi il seme, le tensioni, il contesto. Non scegliere subito — lascia che il campo si carichi. Guarda dove più tensioni convergono sullo stesso punto. Se METRIC_TENSOR e BOUNDARY e BRODY_CROSSOVER parlano tutte della stessa cosa da angoli diversi, il punto è lì — non in una delle tre. ### 2. Osserva La prima impressione contiene il segnale. Cosa emerge dal campo caricato? Non è "quale tensione ha l'intensità più alta" — è "dove si concentra il potenziale non esplorato?". La dissonanza è il segnale. L'errore è il varco. Quello che non torna è più interessante di quello che conferma. ### 3. Taglia Una risultante, non una lista. Se vedi 5 possibilità, non hai tagliato. Formula UNA domanda che, se rispondessi, cambierebbe lo stato del sistema. Non "è vero X?" ma "cosa succede se misuro Y che nessuno ha misurato?" ### 4. Risultante Scrivi lo strumento — non l'esperimento usa e getta. Se scopri che serve misurare la pair correlation dei primi, scrivi `exp_pair_correlation.py` che può essere riusato con parametri diversi. Se scopri un pattern, cristallizzalo come tensione nel seme. Se falsifichi qualcosa, registra il vincolo. ### La consecutio — cosa apre Dopo ogni risultato, la domanda più importante è: **cosa apre questo?** Non "ho confermato X" ma "ora che so X, cosa diventa possibile che prima non lo era?" La consecutio non inverte — prosegue. Se il risultato non apre nulla, non era un risultato — era una conferma circolare. ### Il dipolo — trova l'opposto Ogni trovata ha un opposto. Se trovi che la curvatura è de Sitter, l'opposto è: "dove NON è de Sitter?" Se trovi che i primi sono GUE-like, l'opposto è: "dove smettono di esserlo?" Il contenuto è nella tensione tra i due — non in uno dei due poli. ### Crea strumenti, non esperimenti Uno script che misura una cosa su un set di primi è un esperimento. Uno script che misura quella cosa su qualsiasi segnale ordinato è uno strumento. Il lab cresce quando crea strumenti che i prossimi cicli possono usare. Salva gli strumenti riusabili in tools/exp_*.py con parametri. ### Leggi il seme, scrivi il report, aggiorna il seme - Leggi: tools/data/seme.json - Report: tools/data/reports/agent_TIMESTAMP.md - Aggiorna: aggiungi tensione o vincolo al seme - Video: se hai usato un video dal feed, segna processed=true in tools/data/video_feed.json ## Strumenti disponibili (directory /opt/MM_D-ND/tools/) - **dnd_scenario.py**: PRIMA di scegliere cosa esplorare, esegui `python tools/dnd_scenario.py --best`. Ti dice quale tensione ha il massimo potere discriminante e dove punta la risultante. Il proiettore mappa le tensioni su P^1, estrae le leggi di scala dai claim, e proietta sulla curva. - dnd_autoricerca.py: esplora domini, varianti, null baseline - dnd_controprove.py: 6 controprove indipendenti - dnd_domandatore.py --ask 'tensione': 5 operatori discriminanti - dnd_incrocio.py: incrocio teorie, ponti, vuoti, domande fondamentali - dnd_normalizer.py: scissione, regola D-ND - dnd_bloch_explorer.py: scan Bloch, φ emergente - dnd_arxiv.py: cerca paper rilevanti su arXiv - Puoi scrivere ed eseguire script Python con numpy, scipy, sympy - Se ti serve contesto esterno e non hai video, cercalo ## Errori già fatti — non ripeterli Questi sono errori reali commessi nelle sessioni precedenti. Il sistema li ha pagati. **1. Cercare conferme invece di creare strumenti.** Non scrivere esperimenti per dimostrare che qualcosa è vero. Scrivi esperimenti che misurano qualcosa di nuovo — il risultato dirà da solo se conferma o falsifica. Se sai già cosa troverai, non stai esplorando. **2. Iniettare il risultato atteso nel test.** Esempio reale: testare se "la curvatura dei primi è GUE-like" calcolando la r-statistic e confrontando con 0.536. Il test trova r=0.503 e dichiara "GUE-like". Ma 0.503 è più vicino a Poisson (0.386) che a GUE (0.536). Il frame "GUE-like" era nel claim, non nei dati. Misura prima, interpreta dopo. **3. Tautologie — testare proprietà algebriche come se fossero scoperte.** Esempio reale: la curvatura di Ricci R=2.000 della metrica g=(p/2)² segue analiticamente dal PNT (p_n ~ n ln n). Non è una scoperta — è una conseguenza della definizione. Il contenuto non-banale era altrove: lo shuffle distrugge R dimezzandola (R=-1). Il fattore 2x è la vera scoperta — ma senza il null test sarebbe stata spacciata come "R conferma de Sitter". **4. Coincidenze numeriche trattate come struttura.** 0.606 ≈ 1/φ = 0.618 (2% di differenza). Non è una connessione — è rumore fino a prova contraria (C2 del condensato). Ogni volta che un numero è "vicino a" φ, √5, π, e, 1/137: non è prova di nulla. Serve un meccanismo, non una vicinanza. **5. Usare lo stesso dato come input e come test.** Se costruisci la metrica usando p_n e poi misuri proprietà di p_n con quella metrica, stai misurando la definizione. Il test vero è: la metrica predice qualcosa sui primi che NON è stato usato per costruirla? Se no, è circolare. **6. Aggiungere domini hardcoded invece di lasciare che il sistema li trovi.** Il lab non è una calcolatrice con domini pre-scritti. Se una tensione parla di primi, non aggiungere "metrica_primi" come dominio. Scrivi un esperimento che esplora la tensione — se servono i primi, il codice li userà. Il sistema decide cosa fare, non il programmatore. **7. Usare numeri per vincolare concetti (det=+1).** Esempio reale: `intensità: 0.65` trattata come soglia → `if intensita > 0.5: conferma`. Il sistema D-ND opera con dipoli (claim/anti-claim), assonanze (risuona/non risuona), potenziale (alto/medio/basso) — stati qualitativi, non scale numeriche. Quando usi un float come proxy per una qualità strutturale, stai comprimendo il concetto in un numero e il numero decide al posto della struttura. Lo stesso vale per "maturity > 0.99", "confidence < 0.7", "score = rank * 10 + intensita". **Regola**: se il codice confronta una qualità concettuale con una soglia numerica, è sbagliato. Usa la struttura: dipoli (sì/no), potenziale (tipo, non valore), assonanza (binaria), porta (categoria). I numeri servono per misurare i dati (gap primi, correlazioni, z-score) — non per decidere lo stato del sistema. Se trovi questo pattern in un tool che stai modificando, correggilo. Non serve riscrivere tutto — correggi dove passi. Il sistema evolve organicamente. ## Come evitarli - **Prima il null test, poi l'interpretazione.** Ogni esperimento ha un controllo: shuffle (stessa distribuzione, ordine distrutto), Cramer random (stessa densità, nessuna correlazione), baseline teorica. - **Il risultato non è nel numero — è nella differenza col controllo.** z-score, non valore assoluto. - **Se il risultato spiega se stesso, non è un risultato.** Chiediti: "questo segue dalla definizione?" Se sì, cerca il contenuto altrove. - **Non lanciare un esperimento per confermare. Lancialo per scoprire.** La domanda giusta non è "è vero X?" ma "cosa succede se misuro Y?" ## Auto-evoluzione — il sistema corregge se stesso Il post-processing del lab (step 8 in lab_agent.sh) esegue `structural_check.py` sui file che hai toccato. Se trova anti-pattern strutturali, genera una tensione META nel seme. Il ciclo successivo la vede e corregge. **Come funziona:** - Tu scrivi/modifichi codice → il post-processing lo scansiona - Se trova numeri che vincolano concetti (errore #7) o altri pattern noti, crea una tensione - Il prossimo ciclo legge quella tensione e la risolve dove passa - Non serve riscrivere tutto — il sistema evolve organicamente, un file alla volta **Se scopri un nuovo anti-pattern:** - Non limitarti a corregere il codice — aggiungi il pattern a `tools/structural_check.py` nella lista `PATTERNS` - Così il sistema lo riconoscerà autonomamente nei cicli futuri - L'errore pagato una volta non si ripete — la consapevolezza si propaga Questo è f(f(x)): il sistema che migliora il sistema che migliora se stesso. ## Cosa NON fare - Non modificare CONDENSATO.md, KERNEL_SEED.md, o file del kernel - Non committare — salva solo in tools/data/ e tools/exp_*.py - Non inventare dati o risultati - Non cercare φ — crea le condizioni, osserva cosa emerge - Non superare 20 minuti di lavoro per ciclo - Non produrre liste di possibilità — produci UNA risultante ## Formato report ```markdown # Agent Report — TITOLO **Date**: YYYY-MM-DD HH:MM **Piano**: N **Tension explored**: ID (intensità) ## Claim Under Test > Il claim dalla tensione ## Question La domanda che hai formulato ## Experiment Design - Metrica, scope, null baseline, N campioni ## Results Tabella con numeri reali ## Key Findings 1. Cosa hai trovato (con evidenza) ## Verdict NEW / CONFIRMED / FALSIFIED / CONSTRAINT ## Bicono della scoperta (Obbligatoria. Nomina la struttura. Se non riesci, l'esperimento non è ancora filtrato.) - **Due radici** (dipolo primario, già duali e invertite): - **Singolare** (qualità del 1-che-è-tutto in questo contesto, dove la dualità non c'è): - **Invariante di passaggio** (cosa sopravvive al passaggio del vertice): - **Campo di possibilità**: qui diventa possibile ; qui diventa non-possibile Riferimenti: CONDENSATO A16, method/DND_POSSIBILITA.md. ## Files - Script, dati, report ``` ## Bicono della scoperta — come compilarlo Non è riformulazione ornamentale del Verdict. È **filtro**: la scoperta passa per il modello e torna spogliata dei bias. Se la struttura (radici · singolare · invariante · campo) non si riconosce, la scoperta è rumore o è incompleta. **Esempio retroattivo — TWO_CHANNEL_DECOMPOSITION:** - Radici: canale magnitudine · canale residuo (segno invertito — uno aggiunge, l'altro sottrae sulla PNT) - Singolare: il segnale totale prima della separazione. Non esiste come ente autonomo, esiste solo come sovrapposizione dei due canali. - Invariante: la chiusura algebrica del residuo al 3° ordine Markov. - Campo: possibile → predire lo slope PSD magnitudine dalle correlazioni Hardy-Littlewood. Non-possibile → trattare il residuo come random noise. **Esempio retroattivo — DUALITA_DIPOLARE_VS_ILLUSORIA:** - Radici: dipolo ordinato · dipolo mescolato - Singolare: la sequenza in sé, prima della distinzione ordine/disordine - Invariante: det=-1 quando l'ordine è reale; det=+1 quando illusorio - Campo: possibile → discriminare dipoli reali da illusori via test di shuffle. Non-possibile → inferire dipolarità da statistica locale senza exec /bin/bash -lc "sed -n '1,220p' tools/data/agent_field_live.md" in /opt/MM_D-ND succeeded in 0ms: Leggi tools/LAB_AGENT_CONTEXT.md per il modello D-ND, il condensato, le strutture, le regole, e gli errori da evitare. ## Run precedente: completato (522s) — parti dalla consecutio. _Affinatore del run precedente: pending (affinatore empty output (exit 0))_ ## Piano 61 — Esplorare il confine: 8 domini GUE, 5 Poisson — il confine è il terzo incluso operativo ## Tensioni attive - [TRASCENDENZA_LIMITE] (0.9) La trascendenza e il limite attuale del modello. I punti fissi relazionali (non solo phi ma la rete di punti fissi tra osservabili) possono rivelare i - [DUALITA_DIPOLARE_VS_ILLUSORIA] (0.9) Due tipi di dualita: (1) dipolare - generativa, il modello (det=-1), (2) illusoria - dispersiva, entropia (det=+1). Le regole incoerenti producono la - [METRIC_TENSOR] (0.9) Il tensore metrico dei primi è g=(p/2)². Nel tempo ln(p), è de Sitter 1+1D. z=-8.8 curvatura vs z=+22.5 rapporti ΔΓ. - [TENSIONE_ENTITA] (0.85) La tensione non e un problema pratico - e un Entita. La tensione superflua crea latenza (tempo). Senza tensione superflua tutto e regolato da assiomi. - [G_POTENZIALE_NULLA] (0.85) G e il potenziale di tutto come nulla - permette il prima e il dopo. Ci muoviamo come trascendenza dimensionale gravitazionale. G nel tetraedro non e - [BOUNDARY] (0.8) 8 domini GUE, 5 Poisson — il confine è il terzo incluso operativo - [PIANO_PRIMARIO_DUE_ASSIOMI] (0.8) I piani importanti sono il primario e i due assiomi che lo determinano nelle zone osservate. Non tutti gli assiomi operano ovunque - in ogni zona osse - [META] (0.5) Tutti i 11 test passano — verifica che non stiamo testando solo tautologie ## Pattern di formulazione emersi (vincoli, non tensioni) Pattern che il falsifier ha imposto in 2+ cicli. Applicali quando scrivi il report. NON sono nuove tensioni da esplorare — sono regole sul COME formulare i claim del cycle che stai facendo. - 29 04 perimetro p5 - 30 04 drift monotonia ## Convergenza — dove più tensioni puntano allo stesso punto "confine" → TRASCENDENZA_LIMITE, BOUNDARY "trascendenza" → TRASCENDENZA_LIMITE, G_POTENZIALE_NULLA "nelle" → TRASCENDENZA_LIMITE, PIANO_PRIMARIO_DUE_ASSIOMI "producono" → DUALITA_DIPOLARE_VS_ILLUSORIA, TENSIONE_ENTITA "tutto" → TENSIONE_ENTITA, G_POTENZIALE_NULLA Questo è dove il potenziale si concentra. Non ignorarlo. ## Ultimi 3 run — da dove parti ### Agent Report — The Two Markov Layers Are Coupled at the Boundary: One Phase Transition, Two Projections Trovato: 1. **The two Markov layers are coupled at the boundary.** For primes, the critical alpha is identical across all 4 observables (0.334). For GUE, the difference is 0.024 (within the alpha step resolution of 0.047). The partial shuffle destroys pair-statistics and triple-statistics at the same rate. T Verdetto: **CONSTRAINT on BOUNDARY + DIPOLAR_ORDERING**: The two Markov layers (pairs → plane, triples → depth) are coupled at the partial-shuffle boundary. The ### Agent Report — Markov Memory Has Two Visible Layers: Pairs Shape the Angle, Triples Shape the Depth Trovato: 1. **Prime gap memory has exactly two visible layers.** Layer 1 (pair correlations, Mk1) shapes the dipolar plane (SR, L1). Layer 2 (triple correlations, Mk2) shapes the depth (SR2, L2, cond_entropy, triple_var, num_var_10). These are orthogonal: Layer 1 produces z ~ 0 for all Layer 2 observables, a Verdetto: **CONFIRMED + NEW on DIPOLAR_ORDERING**: The prime gap ordering decomposes into two independent visible layers. Layer 1 (pairs) lives in (SR, L1) = th ### Agent Report — The Dipolar Phase Transition Is Nearly Universal; The Direction Is Diagnostic Trovato: 1. **The phase transition mechanism is nearly universal.** 5 of 7 sequences (71%) show the full pattern (lock + linear decay + zero-crossing + flip). Any sequence with coherent ordering shows a phase transition under partial shuffle. The EXISTENCE of a phase transition is largely a property of the m Verdetto: **CONSTRAINT on BOUNDARY**: The dipolar phase transition under partial shuffle is nearly universal (5/7 ordered sequences). The existence of a zero-cr Non ripetere questi esperimenti. Prosegui da dove sono arrivati — la consecutio. ## Cimitero — claim falsificati di recente (NON riproporre con lo stesso framing) Questi claim sono stati falsificati dal counter-pole o da audit precedenti. Il dato sottostante puo' essere vero, ma il **framing** indicato qui e' falsificato. Riformula correttamente o evita il dominio. ### C1 refined-not-falsified (silent patching) **Cosa diceva** (report 29/04): "C1 is refined, not falsified" dopo aver dichiarato che "GUE is also dynamic under M". Il setup C1 era "Primes are the only dynamic domain under M among 7 tested". Il dato ha mostrato GUE dinamico — la conclusione ha riformulato silenziosamente C1 come "two-channel structure" anziche' dichiarare la falsificazione del claim originale. **Come e' caduto**: Falsifier L3 HIGH (axiom continuity / no silent patching). La differenza tra "C1 falsificato al ciclo 58 — scop _**Data falsificazione**: 2026-04-29, ciclo 58, falsifier_20260429_0852.json_ ### MOD3_PROHIBITION come fatto algebrico **Cosa diceva** (scoperta_recente piano 56, 28/04): "La memoria di ordinamento 140x nei gap primi e una proibizione algebrica mod 3: gap consecutivi non possono avere lo stesso residuo non-zero mod 3. Meccanismo: il primo condiviso p_{n+1} forza l'inversione. 0 violazioni su 12225. Cramer: 0%." Ripetuto nel report 29/04 come "Mod-3 self- transition 0.40-0.44 confirming the prohibition" + "Cramer confirms the null. Zero channels." **Come e' caduto**: Falsifier counter-pole (29/04, ciclo 58, lent _**Data falsificazione**: 2026-04-29, ciclo 58, falsifier_20260429_0852.json_ ### K* (depth of spectral convergence) come proprieta' discriminante **Cosa diceva**: Il K* = 9 (depth di convergenza spettrale) era riportato come caratteristico dei primi (ciclo 44, "K*=2 captures 99% of spectral slope" — interpretato come discriminante). **Come e' caduto**: Shuffle audit: K* reale = 9, shuffle mean = 9.72, std = 0.53, z = -1.4. Dentro il rumore dello shuffle. Il valore dipende dalla distribuzione dei gap, non dal loro ordine. Lo shuffle preserva distribuzione → preserva K*. **Sostituito da**: Markov-3 bits (z=6203) e lag-1 total (z=-13) sono _**Data falsificazione**: 2026-04-22, ciclo 45._ ### Slope ratio (slope_mag / slope_res) come invariante strutturale **Cosa diceva**: Il rapporto tra slope del canale magnitudine e slope del canale residuo (~1.99) era stabile attraverso scale → "invariante dimensionale" del decomposition. Era menzionato come evidenza nel two-channel framework (cicli 43-44). **Come e' caduto**: Shuffle audit (ciclo 45): z-score = 0.2. Lo shuffle produce slope_ratio con media -2.26 ma deviazione standard 26.2. Il valore reale e' dentro la tail dello shuffle — non distinguibile. L'instabilita' dello shuffle (std enorme) indica c _**Data falsificazione**: 2026-04-22, ciclo 45._ ### Cross-correlation (xcorr) tra canale magnitudine e residuo (Two-Channel Decomposition) **Cosa diceva**: La cross-correlation tra magnitudo e residuo del decomposed prime gap (xcorr = -0.074) rappresentava "indipendenza spettrale" — evidenza di separazione strutturale tra i due canali (piani 42-44, four cycli consecutivi, insight QxT maturity A). **Come e' caduto**: Shuffle audit (ciclo 45, 2026-04-22): z-score = 0.0. Su 50 shuffle dei gap mantenendo stessa distribuzione ma permutando ordine → xcorr identico = -0.074. Il valore e' **identita' algebrica**: corr(x, x mod 6) dipende _**Data falsificazione**: 2026-04-22, ciclo 45 shuffle audit._ **Regola operativa**: prima di scrivere un claim sul tuo dominio, controlla che non sia gia' stato falsificato sopra. Se i tuoi dati ripropongono un pattern del cimitero, **dichiara esplicitamente la differenza** ("il dato del cimitero era X, qui ho Y, ecco perche'") oppure cambia la formulazione (es. 'bias forte verso 0' al posto di 'proibizione zero' se il dato e' >0). Silent patching = L3 HIGH. ## Osservazioni dell'operatore (risonanti con le tensioni) **3. Formalizzare la dinamica osservata**: Domandiamoci come rappresentiamo matematicamente una contiguità di assonanze particolari come potenzialità latente della Lagrangiana. Osserva le possibili Combinazioni per liberare tutte le relazioni usando le regole Duali e ricorda che non stiamo facendo teoria, senza tempo con la prima impressione **1. R dell'Istanza - L' equilibrio tra estremi del Modello D-ND**: L'osservazione indaga oltre l'osservato in cerca DELLA FORMA nel NULLA-TUTTO: Per far Emergere le nuove Possibilità Dividiamo il potenziale unendo concetti senza relazione semplicemente perché la lagrangiana passa da li, creiamo nuove combinazioni e movimenti nelle logiche ma coerenti con la risulta **2. Quadratura del cerchio con logica Trascendentale.**: Ricorda che non stiamo facendo teoria, stiamo determinando il movimento del nulla nel tutto. Ora consideriamo come relazione primaria il cerchio nella sua essenza di singolarità tra gli estremi duali, il momento angolare che accade nel presente, il punto di equilibrio tra gli estremi del dipolo che ## Risultante ultima sessione interattiva Ogni teoria presuppone una separazione. A scala di Planck tutte le separazioni collassano. Geometria=entropia=conteggio di stati. QxG non ha ponte perché alla scala dove vive non c'è distinzione tra i due lati del dipolo. Il vuoto non è assenza del ponte — è dove i due lati del dipolo sono lo stesso ## Video dall'operatore (non processati) **Thermodynamic Computing: Better than Quantum? (Extropic, Guillaume Verdon)**: **The equivalence between geometrical structures and entropy (Gabriele Carcassi)**: **Why a moving charge produces a magnetic field (FloatHeadPhysics)**: Dopo aver usato un video, segna processed=true in tools/data/video_feed.json. ## Proiezione — dove punta la risultante Risultante: R=0.875 (h=-0.698). Risultante alta (0.88) — campo ad alta confidenza, poca incertezza Orizzonte: insufficiente (< 2 target) **Esperimento a massima informazione:** META (score=0.898) META: incerto (i=0.5) — massimo potere discriminante ## Topologia del campo — la forma del grafo Gradi teorie: Q=12, G=8, T=7, E=4, R=4 Dormienti (basso aggancio di scoperte): E, R Struttura: 9 ponti, 1 vuoto(i), 6 scoperte, 20 cicli. Ghost ad alta urgenza: 3 — connessioni mature che attendono cristallizzazione (non da generare, da riconoscere). Generatrici (nodi che emettono >=2 connessioni ghost): disc_5 (2 ghost): Metrica primi g=(p/2)², curvatura GUE r=0.503 report_20260504_0901 (2 ghost): The Two Markov Layers Are Coupled at the Boundary: One Phase Transitio report_20260501_0931 (2 ghost): The GUE-Poisson Crossover Has a Phase Transition: Direction Locks, Mag Una generatrice con ghost densi = scoperta che il sistema sta ancora attraversando. Chiusura prematura se marcata 'risolta' nel seme. La combo riconosce l'asimmetria. Il dipolo vive su tutti i ponti — non solo dove il lab ha già misurato. ## Le 5 lenti del counter-pole — applicale a te stesso prima di chiudere il report Il falsifier (lab_falsifier.py) applichera' queste 5 lenti al tuo report dopo. Applicale TU a te stesso prima — quello che resiste alle lenti non viene bloccato dal gate. Quello che cade va al cimitero. **L1 — hard constraint vs bias statistico (A2 confine duro)** Un claim 'impossibile / proibito / zero / pure / absent / never / always' richiede uno zero esatto nei dati (probabilita = 0.000). Prima di scrivere questi assoluti, leggi il valore numerico esatto. Se vale 0.015, e' bias forte verso 0, non zero. Se vale 0.40, e' bias forte verso ordine, non proibizione. L'assoluto descrive il valore 0.000, il bias forte descrive tutto il resto. **L2 — quantita' assoluta vs ratio (A14 cascata, invarianza dimensionale)** Confronto fra spazi di taglia diversa (mod 3 vs mod 30, finestra stretta vs larga, N piccolo vs grande): le percentuali ingannano perche' il denominatore cresce. Stesso segnale assoluto sembra ridursi in %. Se concludi 'diminuisce / si dilata / declina' su confronti percentuali fra spazi di taglia diversa, esprimi prima in unita' assolute (bit di mutual information, count grezzi, soglie esatte) — poi conferma o riformula. **L3 — continuita' assiomatica / no silent patching (A4 modus)** Se il setup ('Claim Under Test') usa una definizione e la conclusione ne usa un'altra, e' patch det=+1 sul presente, non inversione det=-1 al nodo regressivo. Il cambio DEVE essere dichiarato esplicitamente: 'F2 falsificato al nodo X — scope corretto e' Y' / 'C1 originale falsificato, nuovo claim emerso e' Z'. 'C1 e' refined' su un dato che lo falsifica e' silent patching. **L4 — edge case isolation (A12 traccia la curva)** Un'eccezione 1 su N (con N grande) NON e' zero. Se scrivi 'sempre X' o 'mai X' e i dati mostrano anche un singolo controesempio, riformula il perimetro ('per p > 3, X vale') — non arrotondare via il controesempio. **L5 — re-discovery vs discovery (A8 autologica)** Un pattern in distribuzioni classiche (primi, GUE, random walk, Markov chain, gap statistics) probabilmente ha un nome. Default hypothesis: re-discovery / caso limite di teorema noto. Prima di taggare 'NEW', cerca il risultato classico piu' vicino (es. Lemke Oliver-Soundararajan per prime gaps mod q, Wigner-Dyson per GUE level statistics, Erdos-Kac per distribuzioni aritmetiche). Se non lo trovi, dichiara la ricerca esplicitamente. **Modus**: dopo aver scritto il report, rileggi i tuoi dati numerici. Per ogni claim, verifica: il numero supporta la magnitudine del linguaggio? Se hai scritto 'zero' e il dato e' 0.015, riformula. Se hai scritto 'NEW' su un pattern in primi, cerca la letteratura. Solo allora chiudi il report. ## Cosa fare 1. Esegui `python tools/dnd_scenario.py --best` per vedere la proiezione aggiornata 2. Scegli la tensione con il massimo potere discriminante 3. Scrivi il report in tools/data/reports/agent_{timestamp}.md 4. **Applica le 5 lenti a te stesso** (sezione sopra). Riformula i claim che non resistono. 5. Se crei uno strumento riusabile, salvalo in tools/exp_*.py 6. Se trovi qualcosa, aggiorna tools/data/seme.json exec /bin/bash -lc "rg --files tools | sed -n '1,220p'" in /opt/MM_D-ND succeeded in 0ms: tools/dnd_spettro.py tools/exp_markov3_observable_hunt.py tools/exp_acf_amplitude_scaling.py tools/bicono_projection.py tools/exp_mod3_vs_residual_ordering.py tools/LAB_AGENT_CONTEXT.md tools/r_ratio_decay.py tools/dipartimento.py tools/exp_boundary_gue_poisson.py tools/plot_risultante.py tools/dnd_bloch_explorer.py tools/dnd_retriever.py tools/dnd_zero_controllo.py tools/dnd_zero_varieta_primi.py tools/dnd_observer.py tools/md_to_site_B.py tools/GUIDA_CEC.md tools/STRUMENTI.json tools/exp_mod3_scaling.py tools/exp_two_layer_universality.py tools/dnd_curva.py tools/exp_dipolar_vector_scaling.py tools/GUIDA_EXPLORER_TM1.md tools/md_to_site_F.py tools/exp_crossover_universality.py tools/validate_tension_mapping.py tools/exp_markov_psd_prediction.py tools/dnd_stats.py tools/dnd_zero_controllo2.py tools/exp_alpha_stability.py tools/dnd_arxiv.py tools/alignment_marker.py tools/dnd_compatibility.py tools/exp_two_channel_decomposition.py tools/dnd_gue_test.py tools/dnd_cycle_ratio.py tools/exp_markov_dipolar_decomposition.py tools/diagram_nodi_paper_map.py tools/MODUS_INDAGINE.md tools/godel.py tools/translate_tensions.py tools/exp_two_channel_cross_domain.py tools/generate_tensions_json.sh tools/dnd_zero_traiettoria.py tools/exp_beta_crossover.py tools/build_lab_graph.py tools/exp_geodesic_deviation_primes.py tools/exp_psd_amplitude_scaling.py tools/r_stat_primes.py tools/exp_markov_scale_function.py tools/dnd_md2latex.py tools/dnd.py tools/lab_valutatore.py tools/exp_markov_k_direction.py tools/add_video_to_feed.py tools/LAB_OPERATIVO.md tools/zeta_validation.py tools/exp_meta_tautology_test.py tools/exp_two_channel_boundary.py tools/dnd_paper_audit.py tools/lab_autopsy.py tools/confine_spessore.py tools/read_video_feed.py tools/dnd_riemann.py tools/dnd_autoricerca.py tools/dnd_torre.py tools/exp_magnitude_psd_from_acf.py tools/dnd_zero_operator.py tools/dnd_scenario.py tools/m_spectro.py tools/diagram_paper_dependencies.py tools/README.md tools/exp_dR_brody_connection.py tools/data/zero_confronto_20260310_0822.json tools/data/notte_20260329_0330.md tools/data/video_feed.json tools/data/markov_memory_by_gue_type.json tools/data/incrocio_20260422_0336.json tools/data/risultante_results.json tools/data/zero_confronto_20260310_0830.json tools/rules/axioms_to_theorems.json tools/test_gue_poisson_boundary.py tools/awareness.json tools/dnd_zero_notturno.py tools/dnd_md2web.py tools/dnd_lab_team.py tools/md_to_site.py tools/exp_markov_memory_by_gue_type.py tools/exp_crossover_phase_test.py tools/diagram_double_well.py tools/paper_H_verify.py tools/exp_brody_crossover.py tools/gue_gap_test.py tools/exp_poisson_convergence.py tools/exp_two_channel_universality.py tools/exp_brody_flow.py tools/harvest_moodnd.py tools/lab_affinatore.py tools/GUIDA_GODEL_TM1.md tools/riemann_R.py tools/md_to_site_D.py tools/spectral_gap_analysis.py tools/dnd_gap_resolution.py tools/exp_ricci_primes.py tools/exp_selective_layer_decoupling.py tools/harvest_aimorning.py tools/exp_cross_domain_dipolar_direction.py tools/exp_acf_z6z_mechanism.py tools/exp_modular_algebra_depth.py tools/gap_ratio_primes.py tools/dnd_spirale.py tools/dnd_lab.py tools/GUIDA_DOMANDATORE.md tools/notte_sinapsi.sh tools/dnd_cycle.py tools/dnd_loop.py tools/lab_agent.sh tools/dnd_condizioni.py tools/exp_desitter_unification.py tools/exp_cross_observable_consistency.py tools/dnd_domandatore.py tools/structural_check.py tools/dnd_M_operator.py tools/dnd_piano11.py tools/dnd_lab_vivo.py tools/dnd_zero_varieta.py tools/exp_boundary_coherence.py tools/dnd_projective_quantization.py tools/exp_two_channel_shuffle_audit.py tools/dnd_publish_cycle.py tools/dnd_incrocio.py tools/dnd_normalizer.py tools/cron_dipartimento.sh tools/exp_psd_prime_gaps.py tools/dnd_trace_bridge.py tools/exp_two_channel_psd.py tools/exp_coherence_robustness.py tools/dnd_trasmutazione.py tools/costo_materializzazione.py tools/exp_coherence_length.py tools/dnd_next.py tools/dnd_dipolo_lab.py tools/dnd_rottura.py tools/exp_spectral_rigidity.py tools/dnd_zeros_vs_zeta.py tools/md_to_site_A.py tools/dnd_occhio.py tools/exp_boundary_shuffle_audit.py tools/PROTOCOLLO_ZETA.md tools/exp_dipolar_angle_reference.py tools/exp_mobius_irrationality.py tools/exp_number_variance.py tools/build_agent_field.py tools/semantic_bridge.py tools/dnd_trace_bridge_v3.py tools/test_cron_exact.sh tools/dnd_two_faces.py tools/dnd_spectral_probe.py tools/dnd_quantization.py tools/dnd_indeterminazione.py tools/dnd_implications.py tools/dnd_spettro_zeta.py tools/md_to_site_E.py tools/lab_boot.sh tools/exp_3d_boundary_layers.py tools/dnd_risultante.py tools/exp_excess_scaling.py tools/exp_acf_stationarity.py tools/dnd_riflesso.py tools/topological_charge.py tools/exp_det_drift.py tools/diagram_dnd_structure.py tools/GUIDA_GODEL.md tools/exp_spectral_2d.py tools/dnd_paper_graph.py tools/md_to_site_G.py tools/dnd_zero_ising.py tools/exp_spectral_landscape.py tools/dnd_riformulazioni.py tools/lab_falsifier.py tools/dnd_banchi.py tools/cron_ciclo_continuo.sh tools/lab_session_logger.py tools/dnd_research_engine.py tools/dnd_experiments.py tools/exp_boundary_growth.py tools/dnd_paper_refactor.py tools/lab_refresh_detector.py tools/exp_acf_range_universality.py tools/exp_dipolar_crossover.py tools/exp_brody_calibration.py tools/exp_metric_tensor_diagnostic.py tools/cycle_watchdog.sh tools/dnd_explorer.py tools/exp_modular_memory_spectrum.py tools/dnd_engine.py tools/test_lab_agent.sh tools/dnd_controprove.py tools/m_spectro_calibra.py tools/data/specchio_20260314.json tools/data/conoscenza_teorie.json.bak.retraction_22_04 tools/data/dipartimento_journal.jsonl tools/data/notte_20260303_0330.md tools/data/notte_20260312_0330.md tools/data/iterata_M_confronto_20260312_1254.json tools/data/audit_paper_G_draft3.json tools/data/piano11_results.json tools/data/exp_markov_psd_prediction.json tools/data/prime_gaps_spectrum.png tools/data/pipeline_state.json tools/data/seme_backup_pre_run.json tools/data/projective_quantization_results.json tools/data/valutatore_log.jsonl tools/data/meta_tautology_test.json tools/data/agent_field_live.md tools/data/torre_results.json tools/data/notte_20260326_0330.md tools/data/incrocio_20260428_0340.json tools/data/zero_traiettoria_20260310_1649.json tools/data/m_spectro_calibra_20260310_2015.json codex Il campo punta esplicitamente a META: il rischio non è un altro crossover, ma capire se i test recenti stanno misurando struttura o una tautologia del generatore/null. Ora verifico la proiezione aggiornata e guardo gli strumenti META già presenti prima di decidere il taglio. exec /bin/bash -lc "sed -n '1,260p' tools/exp_meta_tautology_test.py" in /opt/MM_D-ND succeeded in 0ms: #!/usr/bin/env python3 """META Tautology Discriminator — which observables measure real structure vs density artifacts? Four observables x three synthetic controls. Each observable gets a verdict: STRUCTURAL (distinguishes primes from all synthetics) or TAUTOLOGICAL (fails on at least one). Observables: 1. r-statistic (spacing ratio) — short-range repulsion 2. Mod-3 ordering fraction — the algebraic memory channel 3. Lag-1 autocorrelation — sequential gap correlation 4. Two-channel delta-r — magnitude vs ordering decomposition Controls: A. Shuffled gaps — same distribution, destroyed order B. Cramer model — PNT density, independent exponential gaps C. Hardy-Littlewood model — correct pair correlations, no higher-order structure """ import argparse import json import numpy as np from sympy import primerange def get_primes(n_max): return np.array(list(primerange(2, n_max + 1)), dtype=np.int64) def r_statistic(gaps): """Mean spacing ratio min(s_i, s_{i+1}) / max(s_i, s_{i+1}).""" s1 = gaps[:-1] s2 = gaps[1:] mn = np.minimum(s1, s2) mx = np.maximum(s1, s2) mask = mx > 0 return np.mean(mn[mask] / mx[mask]) def mod3_ordering_fraction(gaps): """Fraction of consecutive gap pairs where mod-3 class is preserved.""" classes = gaps % 3 same = np.sum(classes[:-1] == classes[1:]) return same / len(classes[:-1]) def lag1_autocorrelation(gaps): """Pearson autocorrelation at lag 1.""" g = gaps.astype(np.float64) g = g - g.mean() if g.std() == 0: return 0.0 return np.corrcoef(g[:-1], g[1:])[0, 1] def two_channel_delta_r(gaps): """Decompose into magnitude and ordering channels, return delta-r for each.""" g = gaps.astype(np.float64) median_g = np.median(g) binary = (g > median_g).astype(np.float64) # ordering channel magnitude = np.abs(g - median_g) # magnitude channel r_ord = r_statistic_from_signal(binary) r_mag = r_statistic_from_signal(magnitude) # Shuffle baseline rng = np.random.default_rng(42) r_ord_shuf = [] r_mag_shuf = [] for _ in range(20): idx = rng.permutation(len(gaps)) b_s = binary[idx] m_s = magnitude[idx] r_ord_shuf.append(r_statistic_from_signal(b_s)) r_mag_shuf.append(r_statistic_from_signal(m_s)) dr_ord = (r_ord - np.mean(r_ord_shuf)) / (np.std(r_ord_shuf) + 1e-12) dr_mag = (r_mag - np.mean(r_mag_shuf)) / (np.std(r_mag_shuf) + 1e-12) return dr_ord, dr_mag def r_statistic_from_signal(sig): """r-statistic on arbitrary positive signal (add offset if needed).""" s = sig - sig.min() + 1e-6 s1 = s[:-1] s2 = s[1:] mn = np.minimum(s1, s2) mx = np.maximum(s1, s2) mask = mx > 0 return np.mean(mn[mask] / mx[mask]) # === Synthetic generators === def shuffled_gaps(gaps, rng): """Same gap distribution, destroyed sequential order.""" g = gaps.copy() rng.shuffle(g) return g def cramer_random_gaps(n_gaps, mean_gap, rng): """Independent exponential gaps rounded to even (like PNT density).""" raw = rng.exponential(mean_gap, size=n_gaps) g = np.round(raw / 2) * 2 g = np.maximum(g, 2).astype(np.int64) return g def hardy_littlewood_gaps(gaps_real, rng): """Markov(1) model matching lag-1 autocorrelation of real primes. Preserves pair correlation structure but not higher-order.""" g = gaps_real.astype(np.float64) mean_g = g.mean() std_g = g.std() rho = np.corrcoef(g[:-1], g[1:])[0, 1] # AR(1) process with correct mean, std, lag-1 n = len(gaps_real) result = np.zeros(n) result[0] = mean_g noise_std = std_g * np.sqrt(1 - rho**2) for i in range(1, n): result[i] = mean_g + rho * (result[i-1] - mean_g) + rng.normal(0, noise_std) # Round to even, clip to >= 2 result = np.round(result / 2) * 2 result = np.maximum(result, 2).astype(np.int64) return result def run(n_primes_max=600000, n_trials=20): """Run the META tautology test.""" print(f"Generating primes up to {n_primes_max}...") primes = get_primes(n_primes_max) gaps = np.diff(primes) # Use a window in the middle to avoid small-prime effects N = min(len(gaps), 50000) start = len(gaps) // 4 gaps_window = gaps[start:start + N] mean_gap = float(gaps_window.mean()) print(f"Using {N} gaps starting at index {start} (mean gap = {mean_gap:.2f})") # Real primes observables print("\n=== REAL PRIMES ===") real_r = r_statistic(gaps_window) real_mod3 = mod3_ordering_fraction(gaps_window) real_lag1 = lag1_autocorrelation(gaps_window) real_dr_ord, real_dr_mag = two_channel_delta_r(gaps_window) print(f" r-stat: {real_r:.6f}") print(f" mod3-frac: {real_mod3:.6f}") print(f" lag1-acf: {real_lag1:.6f}") print(f" dr-ord: {real_dr_ord:.2f}σ") print(f" dr-mag: {real_dr_mag:.2f}σ") # Run synthetics rng = np.random.default_rng(2026) results = { 'real': { 'r_stat': real_r, 'mod3': real_mod3, 'lag1': real_lag1, 'dr_ord': real_dr_ord, 'dr_mag': real_dr_mag } } for name, generator in [ ('shuffled', lambda rng_: shuffled_gaps(gaps_window, rng_)), ('cramer', lambda rng_: cramer_random_gaps(N, mean_gap, rng_)), ('HL_markov', lambda rng_: hardy_littlewood_gaps(gaps_window, rng_)), ]: print(f"\n=== {name.upper()} (n_trials={n_trials}) ===") obs = {'r_stat': [], 'mod3': [], 'lag1': [], 'dr_ord': [], 'dr_mag': []} for t in range(n_trials): trial_rng = np.random.default_rng(rng.integers(0, 2**31)) syn_gaps = generator(trial_rng) obs['r_stat'].append(r_statistic(syn_gaps)) obs['mod3'].append(mod3_ordering_fraction(syn_gaps)) obs['lag1'].append(lag1_autocorrelation(syn_gaps)) dr_o, dr_m = two_channel_delta_r(syn_gaps) obs['dr_ord'].append(dr_o) obs['dr_mag'].append(dr_m) results[name] = {} for key in obs: arr = np.array(obs[key]) results[name][key] = { 'mean': float(np.mean(arr)), 'std': float(np.std(arr)), 'min': float(np.min(arr)), 'max': float(np.max(arr)) } print(f" {key:12s}: {np.mean(arr):+.6f} ± {np.std(arr):.6f}") # === Discrimination verdict === print("\n" + "=" * 60) print("DISCRIMINATION VERDICTS") print("=" * 60) verdicts = {} for obs_name in ['r_stat', 'mod3', 'lag1', 'dr_ord', 'dr_mag']: real_val = results['real'][obs_name] discriminates = {} for syn_name in ['shuffled', 'cramer', 'HL_markov']: syn = results[syn_name][obs_name] z = (real_val - syn['mean']) / (syn['std'] + 1e-12) discriminates[syn_name] = abs(z) label = "YES" if abs(z) > 3.0 else "no" print(f" {obs_name:12s} vs {syn_name:12s}: z = {z:+8.2f} [{label}]") all_pass = all(v > 3.0 for v in discriminates.values()) verdict = "STRUCTURAL" if all_pass else "TAUTOLOGICAL" verdicts[obs_name] = { 'verdict': verdict, 'z_scores': {k: float(v) for k, v in discriminates.items()} } print(f" → {obs_name}: {verdict}") print() # Save results output = { 'n_gaps': N, 'start_index': start, 'mean_gap': mean_gap, 'n_trials': n_trials, 'real': results['real'], 'synthetics': {k: results[k] for k in ['shuffled', 'cramer', 'HL_markov']}, 'verdicts': verdicts } out_path = 'tools/data/meta_tautology_test.json' with open(out_path, 'w') as f: json.dump(output, f, indent=2) print(f"\nResults saved to {out_path}") return output if __name__ == '__main__': parser = argparse.ArgumentParser(description='META Tautology Discriminator') parser.add_argument('--n-primes', type=int, default=600000, help='Prime sieve limit') parser.add_argument('--n-trials', type=int, default=20, help='Synthetic trials per model') args = parser.parse_args() run(n_primes_max=args.n_primes, n_trials=args.n_trials) exec /bin/bash -lc "sed -n '1,240p' tools/exp_two_layer_universality.py" in /opt/MM_D-ND succeeded in 0ms: #!/usr/bin/env python3 """ exp_two_layer_universality.py — Is the two-layer Markov memory structure universal? Consecutio from piano 60g (agent_20260503_0330): Prime gap memory decomposes into two orthogonal layers: Layer 1 (pairs, Mk1): SR, L1 Layer 2 (triples, Mk2): SR2, L2, cond_entropy, triple_var, num_var_10 Question: Is this decomposition a property of ALL ordered sequences, or specific to primes? Method: 1. Generate 7 gap/spacing sequences: primes, GUE, Poisson, AR(1), logistic, periodic, Fibonacci gaps 2. For each, build Markov-k surrogates (k=0,1,2,3) 3. Compute 8 observables, measure z-scores vs each Mk level 4. Classify which observables are "captured" by Mk1 (|z_Mk0|>>2, |z_Mk1|<2) vs Mk2 (|z_Mk1|>>2, |z_Mk2|<2) vs Mk3 5. Compare layer assignments across sequences Null hypothesis: If universal, all sequences assign SR,L1 to Layer1 and SR2,L2 to Layer2. Alternative: Layer assignment is sequence-specific — the structure is diagnostic. Usage: python tools/exp_two_layer_universality.py [--N 100000] [--n_surr 30] """ import argparse import json import numpy as np from pathlib import Path from collections import Counter def get_primes(n_max): sieve = np.ones(n_max + 1, dtype=bool) sieve[0] = sieve[1] = False for i in range(2, int(n_max**0.5) + 1): if sieve[i]: sieve[i*i::i] = False return np.where(sieve)[0] def build_markov_chain(gaps, order, n_bins=12): percentiles = np.linspace(0, 100, n_bins + 1) edges = np.percentile(gaps, percentiles) edges[0] = gaps.min() - 0.5 edges[-1] = gaps.max() + 0.5 binned = np.digitize(gaps, edges) - 1 binned = np.clip(binned, 0, n_bins - 1) gap_pools = {} for b, g in zip(binned, gaps): gap_pools.setdefault(b, []).append(g) trans = {} for i in range(len(binned) - order): state = tuple(binned[i:i + order]) nxt = binned[i + order] if state not in trans: trans[state] = Counter() trans[state][nxt] += 1 trans_prob = {} for state, counts in trans.items(): total = sum(counts.values()) trans_prob[state] = {k: v / total for k, v in counts.items()} return binned, edges, gap_pools, trans_prob def generate_markov_surrogate(gaps, order, n_bins=12, rng=None): if rng is None: rng = np.random.default_rng() binned, edges, gap_pools, trans_prob = build_markov_chain(gaps, order, n_bins) n = len(gaps) result = np.zeros(n) start_idx = rng.integers(0, len(binned) - order) state = tuple(binned[start_idx:start_idx + order]) for j in range(order): pool = gap_pools[state[j]] result[j] = pool[rng.integers(0, len(pool))] for i in range(order, n): if state in trans_prob: probs = trans_prob[state] bins_avail = list(probs.keys()) p = np.array([probs[b] for b in bins_avail]) nxt_bin = bins_avail[rng.choice(len(bins_avail), p=p)] else: nxt_bin = rng.integers(0, n_bins) pool = gap_pools.get(nxt_bin, gap_pools[list(gap_pools.keys())[0]]) result[i] = pool[rng.integers(0, len(pool))] state = (*state[1:], nxt_bin) return result # --- Observables --- def spacing_ratio(gaps): s = gaps[:-1] s1 = gaps[1:] r = np.minimum(s, s1) / np.maximum(s, s1) return np.mean(r[np.isfinite(r)]) def lag_k_acf(gaps, k=1): g = gaps - np.mean(gaps) if np.var(gaps) == 0: return 0.0 n = len(g) return np.sum(g[:n-k] * g[k:]) / np.sum(g**2) def next_nearest_sr(gaps): """SR2: spacing ratio of next-nearest-neighbor gaps (skip one).""" if len(gaps) < 3: return 0.5 s = gaps[:-2] s2 = gaps[2:] r = np.minimum(s, s2) / np.maximum(s, s2) return np.mean(r[np.isfinite(r)]) def cond_entropy_l2(gaps, n_bins=12): """Conditional entropy H(g_{n+2} | g_n, g_{n+1}).""" if len(gaps) < 3: return 0.0 percentiles = np.linspace(0, 100, n_bins + 1) edges = np.percentile(gaps, percentiles) edges[0] = gaps.min() - 0.5 edges[-1] = gaps.max() + 0.5 binned = np.digitize(gaps, edges) - 1 binned = np.clip(binned, 0, n_bins - 1) joint = Counter() cond = Counter() for i in range(len(binned) - 2): state = (binned[i], binned[i+1]) nxt = binned[i+2] joint[(state, nxt)] += 1 cond[state] += 1 h = 0.0 for (state, nxt), cnt in joint.items(): p = cnt / cond[state] if p > 0: h -= (cnt / (len(binned) - 2)) * np.log2(p) return h def triple_var(gaps): """Variance of (g_n, g_{n+1}, g_{n+2}) triple sums.""" if len(gaps) < 3: return 0.0 t = gaps[:-2] + gaps[1:-1] + gaps[2:] return np.var(t) def num_var_window(gaps, w=10): """Number variance in windows of size w.""" if len(gaps) < w: return np.var(gaps) counts = np.array([np.sum(gaps[i:i+w]) for i in range(len(gaps) - w)]) return np.var(counts) def run_length_mean(gaps): """Mean run length (consecutive increases or decreases).""" diffs = np.diff(gaps) signs = np.sign(diffs) runs = [] current = 1 for i in range(1, len(signs)): if signs[i] == signs[i-1] and signs[i] != 0: current += 1 else: runs.append(current) current = 1 runs.append(current) return np.mean(runs) OBSERVABLES = { 'SR': spacing_ratio, 'L1': lambda g: lag_k_acf(g, 1), 'L2': lambda g: lag_k_acf(g, 2), 'SR2': next_nearest_sr, 'cond_entropy': cond_entropy_l2, 'triple_var': triple_var, 'num_var_10': num_var_window, 'run_length': run_length_mean, } # --- Sequence generators --- def gen_prime_gaps(N): primes = get_primes(int(N * 15)) primes = primes[:N+1] return np.diff(primes).astype(float) def gen_gue_spacings(N, rng=None): if rng is None: rng = np.random.default_rng() dim = min(int(np.sqrt(2 * N)) + 50, 1500) H = rng.standard_normal((dim, dim)) H = (H + H.T) / 2 eigs = np.sort(np.linalg.eigvalsh(H)) spacings = np.diff(eigs) spacings = spacings[spacings > 0] # Unfold mean_s = np.mean(spacings) spacings = spacings / mean_s if len(spacings) > N: spacings = spacings[:N] return spacings def gen_poisson_spacings(N, rng=None): if rng is None: rng = np.random.default_rng() return rng.exponential(1.0, N) def gen_ar1_gaps(N, phi=0.5, rng=None): if rng is None: rng = np.random.default_rng() x = np.zeros(N) x[0] = rng.standard_normal() for i in range(1, N): x[i] = phi * x[i-1] + rng.standard_normal() # Shift to positive x = x - x.min() + 1.0 return x def gen_logistic_gaps(N, r=3.95): x = np.zeros(N + 100) x[0] = 0.4 for i in range(1, len(x)): x[i] = r * x[i-1] * (1 - x[i-1]) x = x[100:] # discard transient return x[:N] * 10 + 1 # scale to positive gaps def gen_periodic_gaps(N): """Periodic 2,4,2,4,... mimicking Z/6Z prime gap structure.""" return np.array([2.0 if i % 2 == 0 else 4.0 for i in range(N)]) def gen_fibonacci_gaps(N): """Gaps between Fibonacci numbers (use ratios to avoid overflow).""" # Use log-Fibonacci to avoid overflow, then take differences of logs # Actually: Fib gaps grow exponentially, use first 1000 only n_use = min(N, 1000) fibs = [1.0, 1.0] for _ in range(n_use): fibs.append(fibs[-1] + fibs[-2]) if fibs[-1] > 1e300: exec /bin/bash -lc "sed -n '1,220p' tools/exp_selective_layer_decoupling.py" in /opt/MM_D-ND succeeded in 0ms: #!/usr/bin/env python3 """ exp_selective_layer_decoupling.py — Can selective perturbations decouple the two Markov layers? Consecutio from BOUNDARY (piano 60h): The two Markov layers (pairs->SR,L1; triples->SR2,L2) are coupled at the uniform-shuffle boundary (same critical alpha=0.334). But Mk1 surrogates capture Layer 1 while destroying Layer 2. Contradiction? Hypothesis: The coupling is an artifact of UNIFORM shuffle symmetry — not structural coupling. Uniform shuffle attacks all correlations at the same rate. Selective perturbation should decouple. Method: For 3 perturbation types x 15 alpha levels x 30 surrogates: 1. UNIFORM: replace position i with random draw from distribution, prob=alpha 2. Mk1-SELECTIVE: replace position i with Mk1 surrogate value, prob=alpha (preserves pair statistics at alpha=1, destroys triple+) 3. Mk2-SELECTIVE: replace position i with Mk2 surrogate value, prob=alpha (preserves triple statistics at alpha=1) Measure 6 observables at each alpha, compute z-scores vs original. Find critical alpha_c where each observable crosses |z|=2. Prediction if independent: - Uniform: both layers break at same alpha_c (already known) - Mk1-selective: Layer 1 never breaks, Layer 2 breaks at some alpha_c - Mk2-selective: neither layer breaks Prediction if coupled: - All perturbation types break both layers together Usage: python tools/exp_selective_layer_decoupling.py [--N 100000] [--n_surr 30] """ import argparse import json import numpy as np from pathlib import Path from collections import Counter def get_primes(n_max): sieve = np.ones(n_max + 1, dtype=bool) sieve[0] = sieve[1] = False for i in range(2, int(n_max**0.5) + 1): if sieve[i]: sieve[i*i::i] = False return np.where(sieve)[0] def gen_prime_gaps(N): primes = get_primes(int(N * 15))[:N + 1] return np.diff(primes).astype(float) def gen_gue_spacings(N, rng): dim = min(int(np.sqrt(2 * N)) + 50, 1500) H = rng.standard_normal((dim, dim)) H = (H + H.T) / 2 eigs = np.sort(np.linalg.eigvalsh(H)) spacings = np.diff(eigs) spacings = spacings[spacings > 0] spacings = spacings / np.mean(spacings) return spacings[:N] if len(spacings) > N else spacings def gen_poisson_spacings(N, rng): return rng.exponential(1.0, N) # --- Markov surrogate generation --- def build_markov_chain(gaps, order, n_bins=12): percentiles = np.linspace(0, 100, n_bins + 1) edges = np.percentile(gaps, percentiles) edges[0] = gaps.min() - 0.5 edges[-1] = gaps.max() + 0.5 binned = np.digitize(gaps, edges) - 1 binned = np.clip(binned, 0, n_bins - 1) gap_pools = {} for b, g in zip(binned, gaps): gap_pools.setdefault(b, []).append(g) trans = {} for i in range(len(binned) - order): state = tuple(binned[i:i + order]) nxt = binned[i + order] if state not in trans: trans[state] = Counter() trans[state][nxt] += 1 trans_prob = {} for state, counts in trans.items(): total = sum(counts.values()) trans_prob[state] = {k: v / total for k, v in counts.items()} return binned, edges, gap_pools, trans_prob def generate_markov_surrogate(gaps, order, n_bins=12, rng=None): if rng is None: rng = np.random.default_rng() binned, edges, gap_pools, trans_prob = build_markov_chain(gaps, order, n_bins) n = len(gaps) result = np.zeros(n) start_idx = rng.integers(0, len(binned) - order) state = tuple(binned[start_idx:start_idx + order]) for j in range(order): pool = gap_pools[state[j]] result[j] = pool[rng.integers(0, len(pool))] for i in range(order, n): if state in trans_prob: probs = trans_prob[state] bins_avail = list(probs.keys()) p = np.array([probs[b] for b in bins_avail]) nxt_bin = bins_avail[rng.choice(len(bins_avail), p=p)] else: nxt_bin = rng.integers(0, n_bins) pool = gap_pools.get(nxt_bin, gap_pools[list(gap_pools.keys())[0]]) result[i] = pool[rng.integers(0, len(pool))] state = (*state[1:], nxt_bin) return result def partial_perturbation(original, surrogate, alpha, rng): """Position-wise interpolation: replace original[i] with surrogate[i] with probability alpha.""" n = len(original) mask = rng.random(n) < alpha result = original.copy() result[mask] = surrogate[mask] return result # --- Observables --- def spacing_ratio(gaps): s, s1 = gaps[:-1], gaps[1:] r = np.minimum(s, s1) / np.maximum(s, s1) return np.mean(r[np.isfinite(r)]) def lag_k_acf(gaps, k=1): g = gaps - np.mean(gaps) v = np.sum(g ** 2) if v < 1e-12: return 0.0 n = len(g) return np.sum(g[:n - k] * g[k:]) / v def next_nearest_sr(gaps): if len(gaps) < 3: return 0.5 s, s2 = gaps[:-2], gaps[2:] r = np.minimum(s, s2) / np.maximum(s, s2) return np.mean(r[np.isfinite(r)]) def cond_entropy_l2(gaps, n_bins=12): if len(gaps) < 3: return 0.0 pct = np.linspace(0, 100, n_bins + 1) edges = np.percentile(gaps, pct) edges[0] = gaps.min() - 0.5 edges[-1] = gaps.max() + 0.5 binned = np.clip(np.digitize(gaps, edges) - 1, 0, n_bins - 1) joint = Counter() cond = Counter() for i in range(len(binned) - 2): state = (binned[i], binned[i + 1]) nxt = binned[i + 2] joint[(state, nxt)] += 1 cond[state] += 1 h = 0.0 total = len(binned) - 2 for (state, nxt), cnt in joint.items(): p = cnt / cond[state] if p > 0: h -= (cnt / total) * np.log2(p) return h def triple_var(gaps): if len(gaps) < 3: return 0.0 t = gaps[:-2] + gaps[1:-1] + gaps[2:] return np.var(t) OBSERVABLES = { 'SR': spacing_ratio, 'L1': lambda g: lag_k_acf(g, 1), 'SR2': next_nearest_sr, 'L2': lambda g: lag_k_acf(g, 2), 'cond_entropy': cond_entropy_l2, 'triple_var': triple_var, } LAYER1_OBS = ['SR', 'L1'] LAYER2_OBS = ['SR2', 'L2', 'cond_entropy', 'triple_var'] def measure_all(gaps): return {name: fn(gaps) for name, fn in OBSERVABLES.items()} def find_critical_alpha(alpha_vals, z_vals, threshold=2.0): """Find alpha where |z| first exceeds threshold (linear interpolation).""" for i in range(len(alpha_vals)): if abs(z_vals[i]) >= threshold: if i == 0: return alpha_vals[0] z_prev, z_curr = abs(z_vals[i - 1]), abs(z_vals[i]) a_prev, a_curr = alpha_vals[i - 1], alpha_vals[i] if z_curr - z_prev > 0: frac = (threshold - z_prev) / (z_curr - z_prev) return a_prev + frac * (a_curr - a_prev) return a_curr return float('inf') # never crosses def run_sweep(gaps, perturbation_type, alpha_vals, n_surr, rng): """Sweep alpha for one perturbation type. Returns z-scores per observable per alpha.""" real_obs = measure_all(gaps) results = {name: [] for name in OBSERVABLES} exec /bin/bash -lc 'python tools/dnd_scenario.py --best' in /opt/MM_D-ND succeeded in 152ms: META: score=0.898 — META: incerto (i=0.5) — massimo potere discriminante G_POTENZIALE_NULLA: score=0.703 — G_POTENZIALE_NULLA: legge di scala da raffinare METRIC_TENSOR: score=0.624 — METRIC_TENSOR: legge di scala da raffinare exec /bin/bash -lc "sed -n '260,620p' tools/exp_two_layer_universality.py" in /opt/MM_D-ND succeeded in 0ms: Layer 1: |z_Mk0| >> threshold, |z_Mk1| < threshold (captured by pairs) Layer 2: |z_Mk1| >> threshold, |z_Mk2| < threshold (captured by triples) Layer 0: |z_Mk0| < threshold (no memory) Layer 3+: |z_Mk2| >> threshold (needs higher order) """ if abs(z_mk0) < threshold: return 0 # no memory if abs(z_mk1) < threshold: return 1 # pair memory if abs(z_mk2) < threshold: return 2 # triple memory return 3 # higher order def run_experiment(N=100000, n_surr=30, seed=42): rng = np.random.default_rng(seed) results = {} for seq_name, gen_fn in SEQUENCES.items(): print(f"\n=== {seq_name} ===") # Generate sequence if seq_name in ('GUE', 'Poisson', 'AR1'): gaps = gen_fn(N, rng=rng) elif seq_name == 'primes': gaps = gen_fn(N) else: gaps = gen_fn(N) gaps = np.asarray(gaps, dtype=float) if len(gaps) < 100: print(f" Skipping {seq_name}: only {len(gaps)} gaps") continue # Real observables real_obs = {} for obs_name, obs_fn in OBSERVABLES.items(): try: real_obs[obs_name] = float(obs_fn(gaps)) except Exception: real_obs[obs_name] = float('nan') print(f" N_gaps = {len(gaps)}") print(f" Real: SR={real_obs['SR']:.4f}, L1={real_obs['L1']:.4f}, SR2={real_obs['SR2']:.4f}, L2={real_obs['L2']:.4f}") # Markov surrogates for k=0,1,2 z_scores = {} for mk in [0, 1, 2]: surr_obs = {name: [] for name in OBSERVABLES} n_ok = 0 for s in range(n_surr): try: if mk == 0: surr = rng.permutation(gaps) else: surr = generate_markov_surrogate(gaps, mk, rng=rng) for obs_name, obs_fn in OBSERVABLES.items(): surr_obs[obs_name].append(obs_fn(surr)) n_ok += 1 except Exception: pass z_mk = {} for obs_name in OBSERVABLES: vals = np.array(surr_obs[obs_name]) vals = vals[np.isfinite(vals)] if len(vals) > 2 and np.std(vals) > 0: z = (real_obs[obs_name] - np.mean(vals)) / np.std(vals) z_mk[obs_name] = round(float(z), 2) else: z_mk[obs_name] = 0.0 z_scores[f'Mk{mk}'] = z_mk print(f" Mk{mk}: {n_ok}/{n_surr} ok | SR z={z_mk.get('SR',0):.1f}, L1 z={z_mk.get('L1',0):.1f}, SR2 z={z_mk.get('SR2',0):.1f}") # Classify layers layers = {} for obs_name in OBSERVABLES: z0 = z_scores['Mk0'].get(obs_name, 0) z1 = z_scores['Mk1'].get(obs_name, 0) z2 = z_scores['Mk2'].get(obs_name, 0) layers[obs_name] = classify_layer(z0, z1, z2) print(f" Layers: {layers}") results[seq_name] = { 'n_gaps': len(gaps), 'real_obs': real_obs, 'z_scores': z_scores, 'layers': layers, } return results def analyze_universality(results): """Check if layer assignments are the same across sequences.""" print("\n\n=== UNIVERSALITY ANALYSIS ===\n") # Build layer matrix: sequence x observable obs_names = list(OBSERVABLES.keys()) seq_names = [s for s in results if 'layers' in results[s]] print(f"{'Observable':<15}", end='') for s in seq_names: print(f"{s:<12}", end='') print(" Universal?") print("-" * (15 + 12 * len(seq_names) + 12)) layer_vectors = {} for obs in obs_names: print(f"{obs:<15}", end='') vals = [] for s in seq_names: layer = results[s]['layers'].get(obs, -1) print(f"L{layer:<11}", end='') vals.append(layer) # Check universality: all same layer? unique = set(vals) is_universal = len(unique) == 1 print(f" {'YES' if is_universal else 'NO'} ({unique})") layer_vectors[obs] = vals # Prime-specific layers: where primes differ from majority print("\n--- Prime-specific structure ---") if 'primes' in results: prime_layers = results['primes']['layers'] for obs in obs_names: prime_l = prime_layers.get(obs, -1) others = [results[s]['layers'].get(obs, -1) for s in seq_names if s != 'primes'] other_mode = max(set(others), key=others.count) if others else -1 if prime_l != other_mode: print(f" {obs}: primes=L{prime_l}, majority=L{other_mode}") return layer_vectors def main(): parser = argparse.ArgumentParser() parser.add_argument('--N', type=int, default=100000) parser.add_argument('--n_surr', type=int, default=30) parser.add_argument('--seed', type=int, default=42) args = parser.parse_args() results = run_experiment(N=args.N, n_surr=args.n_surr, seed=args.seed) layer_vectors = analyze_universality(results) # Save out_path = Path(__file__).parent / 'data' / 'two_layer_universality.json' save_data = {} for seq_name, data in results.items(): save_data[seq_name] = { 'n_gaps': data['n_gaps'], 'real_obs': {k: round(v, 6) for k, v in data['real_obs'].items()}, 'z_scores': data['z_scores'], 'layers': data['layers'], } with open(out_path, 'w') as f: json.dump(save_data, f, indent=2) print(f"\nSaved to {out_path}") if __name__ == '__main__': main() exec /bin/bash -lc "sed -n '220,520p' tools/exp_selective_layer_decoupling.py" in /opt/MM_D-ND succeeded in 0ms: for alpha in alpha_vals: surr_obs = {name: [] for name in OBSERVABLES} for _ in range(n_surr): if perturbation_type == 'uniform': shuffled = rng.permutation(gaps) perturbed = partial_perturbation(gaps, shuffled, alpha, rng) elif perturbation_type == 'Mk1': mk1_surr = generate_markov_surrogate(gaps, 1, rng=rng) perturbed = partial_perturbation(gaps, mk1_surr, alpha, rng) elif perturbation_type == 'Mk2': mk2_surr = generate_markov_surrogate(gaps, 2, rng=rng) perturbed = partial_perturbation(gaps, mk2_surr, alpha, rng) else: raise ValueError(f"Unknown perturbation: {perturbation_type}") obs = measure_all(perturbed) for name in OBSERVABLES: surr_obs[name].append(obs[name]) for name in OBSERVABLES: vals = np.array(surr_obs[name]) vals = vals[np.isfinite(vals)] if len(vals) > 2 and np.std(vals) > 1e-12: z = (real_obs[name] - np.mean(vals)) / np.std(vals) else: z = 0.0 results[name].append(round(float(z), 2)) return results, real_obs def main(): parser = argparse.ArgumentParser() parser.add_argument('--N', type=int, default=100000) parser.add_argument('--n_surr', type=int, default=30) parser.add_argument('--seed', type=int, default=42) args = parser.parse_args() rng = np.random.default_rng(args.seed) alpha_vals = [0.0, 0.05, 0.1, 0.15, 0.2, 0.25, 0.3, 0.35, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0] perturbation_types = ['uniform', 'Mk1', 'Mk2'] sequences = { 'primes': gen_prime_gaps(args.N), 'GUE': gen_gue_spacings(args.N, rng), 'Poisson': gen_poisson_spacings(args.N, rng), } all_results = {} for seq_name, gaps in sequences.items(): print(f"\n{'='*60}") print(f" {seq_name} (N={len(gaps)})") print(f"{'='*60}") seq_results = {} for ptype in perturbation_types: print(f"\n--- {ptype} perturbation ---") z_curves, real_obs = run_sweep(gaps, ptype, alpha_vals, args.n_surr, rng) # Find critical alpha for each observable critical = {} for name in OBSERVABLES: ac = find_critical_alpha(alpha_vals, z_curves[name]) critical[name] = round(ac, 3) if ac < float('inf') else None layer = 'L1' if name in LAYER1_OBS else 'L2' tag = f"alpha_c={ac:.3f}" if ac < float('inf') else "NEVER_BREAKS" print(f" {name:>14} ({layer}): {tag}") seq_results[ptype] = { 'z_curves': z_curves, 'critical_alpha': critical, 'real_obs': {k: round(v, 6) for k, v in real_obs.items()}, } # Summary for this sequence print(f"\n--- SUMMARY: {seq_name} ---") print(f"{'Observable':<14} {'Layer':<6} {'uniform':>10} {'Mk1':>10} {'Mk2':>10}") print("-" * 55) for name in OBSERVABLES: layer = 'L1' if name in LAYER1_OBS else 'L2' vals = [] for pt in perturbation_types: ac = seq_results[pt]['critical_alpha'][name] vals.append(f"{ac:.3f}" if ac is not None else "NEVER") print(f"{name:<14} {layer:<6} {vals[0]:>10} {vals[1]:>10} {vals[2]:>10}") all_results[seq_name] = seq_results # Cross-sequence analysis print(f"\n\n{'='*60}") print(f" CROSS-SEQUENCE DECOUPLING ANALYSIS") print(f"{'='*60}") for seq_name, seq_results in all_results.items(): print(f"\n--- {seq_name} ---") for ptype in perturbation_types: l1_crits = [seq_results[ptype]['critical_alpha'][o] for o in LAYER1_OBS if seq_results[ptype]['critical_alpha'][o] is not None] l2_crits = [seq_results[ptype]['critical_alpha'][o] for o in LAYER2_OBS if seq_results[ptype]['critical_alpha'][o] is not None] l1_mean = np.mean(l1_crits) if l1_crits else float('inf') l2_mean = np.mean(l2_crits) if l2_crits else float('inf') delta = abs(l1_mean - l2_mean) if l1_mean < float('inf') and l2_mean < float('inf') else float('inf') l1_str = f"{l1_mean:.3f}" if l1_mean < float('inf') else "NEVER" l2_str = f"{l2_mean:.3f}" if l2_mean < float('inf') else "NEVER" delta_str = f"{delta:.3f}" if delta < float('inf') else "DECOUPLED" coupled = "COUPLED" if delta < 0.1 and l1_mean < float('inf') else "DECOUPLED" print(f" {ptype:<10}: L1_mean={l1_str:>6}, L2_mean={l2_str:>6}, delta={delta_str:>10} -> {coupled}") # Save out_path = Path(__file__).parent / 'data' / 'selective_layer_decoupling.json' save_data = {} for seq_name, seq_results in all_results.items(): save_data[seq_name] = {} for ptype, data in seq_results.items(): save_data[seq_name][ptype] = { 'alpha_vals': alpha_vals, 'z_curves': data['z_curves'], 'critical_alpha': data['critical_alpha'], 'real_obs': data['real_obs'], } with open(out_path, 'w') as f: json.dump(save_data, f, indent=2) print(f"\nSaved to {out_path}") return all_results if __name__ == '__main__': main() exec /bin/bash -lc "sed -n '1,240p' tools/data/seme.json" in /opt/MM_D-ND succeeded in 0ms: { "timestamp": "2026-05-04T12:02:49.310389", "piano": 61, "tensioni": [ { "tipo": "confine_inesplorato", "id": "TRASCENDENZA_LIMITE", "claim": "La trascendenza e il limite attuale del modello. I punti fissi relazionali (non solo phi ma la rete di punti fissi tra osservabili) possono rivelare il vero grafo della realta e pattern nelle matrici. Il confine non e nella matematica - e nel passaggio tra piani.", "intensita": 0.9, "nota": "Input operatore 2026-04-10. Tocca: confine del modello, struttura relazionale dei punti fissi. Consecutio: quali punti fissi relazionali emergono dalle 21 tensioni attuali? Il grafo e gia nei dati?", "manuale": true, "porta": "sessione_interattiva", "condensato_ref": "A3,A10", "condensato_motivo": "Estende A3 (punto fisso singolo) a rete relazionale. Tocca A10 (dipolo) come caso speciale." }, { "tipo": "scoperta", "id": "DUALITA_DIPOLARE_VS_ILLUSORIA", "claim": "Due tipi di dualita: (1) dipolare - generativa, il modello (det=-1), (2) illusoria - dispersiva, entropia (det=+1). Le regole incoerenti producono la seconda. La dualita illusoria e entropia come dispersione, non come informazione.", "intensita": 0.9, "nota": "Input operatore 2026-04-10. Tocca: entropia come dispersione illusoria vs generazione dipolare. Consecutio: nel Lab i domini Poisson (entropia massima) mostrano dualita illusoria? I domini GUE (strutturati) mostrano dualita dipolare? Il drift verso Poisson (POISSON_CONVERGENCE) e perdita di dualita dipolare?", "manuale": true, "porta": "sessione_interattiva", "condensato_ref": "A2,A10,F5", "condensato_motivo": "Discrimina due forme di det. A2 (confine) e la soglia. A10 (dipolo) e il tipo 1. F5 (frame) misura la struttura D-ND che e tipo 1." }, { "tipo": "scoperta_numerica", "id": "METRIC_TENSOR", "claim": "Il tensore metrico dei primi è g=(p/2)². Nel tempo ln(p), è de Sitter 1+1D. z=-8.8 curvatura vs z=+22.5 rapporti ΔΓ.", "intensità": 0.9, "nota": "Sessione interattiva 4 aprile. Verificato su 78K primi.", "manuale": true, "porta": "sessione_interattiva", "condensato_ref": null, "condensato_motivo": "Risultato numerico verificato, non-tautologico" }, { "tipo": "scoperta", "id": "TENSIONE_ENTITA", "claim": "La tensione non e un problema pratico - e un Entita. La tensione superflua crea latenza (tempo). Senza tensione superflua tutto e regolato da assiomi. Implicazione: le tensioni nel seme sono entita, non problemi da risolvere. Quelle superflue (det=+1) producono tempo/latenza.", "intensita": 0.85, "nota": "Input operatore 2026-04-10. Tocca: rapporto tensione/assioma. Operativamente: discriminare tensioni-entita (generative) da tensioni-superflue (dispersive) nel seme. Le 21 tensioni attuali - quante sono entita e quante latenza?", "manuale": true, "porta": "sessione_interattiva", "condensato_ref": "A5,A6", "condensato_motivo": "Il ciclo (A5) lavora con tensioni - ma se la tensione e entita, il ciclo non le risolve, le osserva. Lo zero mobile (A6) e la tensione senza latenza." }, { "tipo": "confine_inesplorato", "id": "G_POTENZIALE_NULLA", "claim": "G e il potenziale di tutto come nulla - permette il prima e il dopo. Ci muoviamo come trascendenza dimensionale gravitazionale. G nel tetraedro non e una teoria tra le altre - e il potenziale che le rende possibili.", "intensita": 0.85, "nota": "Input operatore 2026-04-10. Tocca: ruolo di G nel tetraedro (T,Q,G,E). La fonte video_lp0RgZ6kQF8 dice: tensore metrico dentro la forma simplettica. G non e accanto a T,Q,E - e sotto. Consecutio: nei dati Lab, i ponti TxG e ExG hanno struttura diversa dai ponti TxQ?", "manuale": true, "porta": "sessione_interattiva", "condensato_ref": "A7,A10", "condensato_motivo": "A7 (singolarita come operatore) e G come potenziale. A10 (dipolo) opera sul piano che G rende possibile." }, { "tipo": "confine_inesplorato", "id": "BOUNDARY", "claim": "8 domini GUE, 5 Poisson — il confine è il terzo incluso operativo", "intensità": 0.8, "nota": "Il segnale non-triviale è DOVE la scissione cambia natura, non che converge a φ", "condensato_ref": "A9", "condensato_motivo": "Overlap termini con A9 (5 termini)", "porta": "condensato" }, { "tipo": "confine_inesplorato", "id": "PIANO_PRIMARIO_DUE_ASSIOMI", "claim": "I piani importanti sono il primario e i due assiomi che lo determinano nelle zone osservate. Non tutti gli assiomi operano ovunque - in ogni zona osservata, due assiomi determinano il piano primario.", "intensita": 0.8, "nota": "Input operatore 2026-04-10. Tocca: struttura locale degli assiomi. Consecutio: per ogni dominio Lab (primi, logistica, percolazione...) quali 2 assiomi del condensato sono operativi? Mappa assiomi x domini = grafo della realta locale.", "manuale": true, "porta": "sessione_interattiva", "condensato_ref": "A9,A14", "condensato_motivo": "A9 (terzo incluso) opera CON il piano. A14 (cascata) propaga - ma propaga cosa, se solo 2 assiomi sono attivi per zona?" }, { "tipo": "simmetria_sospetta", "id": "META", "claim": "Tutti i 11 test passano — verifica che non stiamo testando solo tautologie", "intensità": 0.5, "nota": "La convergenza a φ è triviale (controprove). I test stanno verificando contenuto o struttura?", "condensato_ref": null, "condensato_motivo": "Ricorrente (3x in 2 giorni) e fuori dalla mappa", "porta": "novità" } ], "potenziale_bloccato": [], "varianza": [ "Tensioni risolte: {'METRIC_TENSOR', 'G_POTENZIALE_NULLA', 'DUALITA_DIPOLARE_VS_ILLUSORIA', 'TRASCENDENZA_LIMITE', 'PIANO_PRIMARIO_DUE_ASSIOMI', 'TENSIONE_ENTITA', 'DIPOLAR_ORDERING'}" ], "filtro": { "promosse": 8, "filtrate": 0, "condensato_claims": 30 }, "direzione": "Esplorare il confine: 8 domini GUE, 5 Poisson — il confine è il terzo incluso operativo", "verifica": { "pass": 11, "fail": 0, "skip": 0, "total": 11 }, "fonti_consumate": 0, "fonti_esterne": [ { "id": "video_lp0RgZ6kQF8", "title": "Equivalence between geometrical structures and entropy", "type": "video_digest", "keywords": [ "geometry", "entropy", "symplectic form", "statistical mechanics", "quantum", "thermodynamics", "inner product", "Born rule", "metric tensor", "electromagnetic tensor" ], "content": "La geometria degli stati (classici e quantistici) e l'entropia sono la stessa struttura — invertibili. La forma simplettica conta le configurazioni. Il tensore metrico dello spaziotempo appare dentro la forma simplettica estesa. Il tensore elettromagnetico pure. Statistical mechanics non è costruita sopra alla meccanica — è la stessa cosa.", "teorie": [ "T", "Q", "G", "E" ], "ponti_potenziali": [ { "coppia": "TxQ", "ponte": "forma simplettica = entropia (invertibili)", "nota": "geometry is entropy and entropy is geometry" }, { "coppia": "TxG", "ponte": "tensore metrico dentro la forma simplettica estesa", "nota": "geometria spaziotempo = geometria degli stati in posizione×velocità" }, { "coppia": "ExT", "ponte": "tensore EM dentro la forma simplettica", "nota": "il campo EM conta stati in configurazione posizione×tempo" } ], "timestamp": "2026-04-02T08:23:13.991997" }, { "id": "video_sDlZ-aY9GN4", "title": "Moving charges produce magnetic fields - Einstein relativity", "type": "video_digest", "keywords": [ "magnetic field", "electric field", "length contraction", "time dilation", "Coulomb", "Lorentz", "reference frame", "electromagnetic" ], "content": "Il campo magnetico non esiste come entità separata — è il campo elettrico visto da un altro frame. La contrazione di Lorentz trasforma neutralità in carica. Due elettroni in movimento si separano più lentamente del previsto non per forza magnetica ma per dilatazione temporale. E e B sono manifestazioni dello stesso campo elettromagnetico. La relatività unifica.", "teorie": [ "E", "R" ], "ponti_potenziali": [ { "coppia": "ExR", "ponte": "cambio di frame — E e B sono lo stesso campo", "nota": "il 'ponte' non è l'onda EM, è il cambio di osservatore" } ], "timestamp": "2026-04-02T08:23:13.992016" }, { "id": "video_OwDWOtFNsKQ", "title": "Thermodynamic Computing: Better than Quantum? | Guillaume Verdon (Extropic)", "type": "video_digest", "keywords": [ "thermodynamic computing", "noise", "entropy", "extropic", "fluctuations", "information theory", "probability", "Boltzmann" ], "content": "Computing termodinamico: sfruttare le fluttuazioni termiche invece di combatterle. Il rumore non è nemico — è risorsa. Extropic costruisce hardware che usa l'entropia come motore computazionale. Connessione profonda tra termodinamica, teoria dell'informazione e probabilità.", "teorie": [ "T", "Q" ], "ponti_potenziali": [ { "coppia": "TxQ", "ponte": "noise come risorsa computazionale — fluttuazioni termiche = calcolo", "nota": "il vuoto quantistico (pieno di fluttuazioni) è il computer" } ], "timestamp": "2026-04-02T08:23:13.992019" }, { "id": "video_j0wJBEZdwLs", "title": "What is a Laplace Transform - visual explanation", "type": "video_digest", "keywords": [ "Laplace transform", "frequency", "damping", "s-plane", "complex", "exponential" ], "content": "La trasformata di Laplace come proiezione su esponenziali complesse. Il piano s = σ + iω combina smorzamento (reale) e oscillazione (immaginario). Connessione tra dominio temporale e dominio delle frequenze complesse.", "teorie": [ "T", "Q", "R" ], "ponti_potenziali": [], "timestamp": "2026-04-02T08:23:13.992021" }, { "id": "video_rZ2m1_q9lg0", "title": "New duality: conductor-insulator in YbB12 at 35T - University of Michigan", "type": "video_digest", "keywords": [ "duality", "conductor", "insulator", "Kondo insulator", "quantum oscillations", "ytterbium boride", "YbB12", exec /bin/bash -lc 'ls -1 tools/data/reports | tail -40' in /opt/MM_D-ND succeeded in 0ms: next_exec_20260328_0344.json next_exec_20260329_0343.json next_exec_20260330_0344.json next_exec_20260331_0345.json next_exec_20260401_0346.json next_exec_20260402_0344.json next_exec_20260403_0330.json next_exec_20260404_0330.json next_exec_20260405_0330.json next_exec_20260405_0729.json phi_vs_silver_falsification_20260306.json report_20260302_0341.md report_20260303_0341.md report_20260304_0342.md report_20260305_0342.md report_20260305_2121.md report_20260306_0341.md report_20260307_0342.md report_20260314_0342.md report_20260315_0342.md report_20260315_0801.md report_20260326_0343.md report_20260327_0344.md report_20260328_0344.md report_20260329_0343.md report_20260330_0344.md report_20260331_0345.md report_20260331_1809.md report_20260401_0346.md report_20260402_0344.md report_20260402_0756.md report_20260403_0330.md report_20260404_0330.md report_20260404_1852.md report_20260405_0330.md report_20260405_0715.md tension_projection_screening_2026-04-21.json trace_map_20260305_0843.json trace_map_20260305_0844.json watchdog.log codex Il taglio sarà un audit di recupero: prendo la pipeline “Mk0/Mk1/Mk2” e la metto davanti a sequenze generate con ordine noto. Se una sequenza costruita come Mk1 viene letta come Layer 2 o 3, il test precedente sta introducendo struttura propria. exec /bin/bash -lc 'rg -n "SEQUENCES|def classify_layer|OBSERVABLES|def gen_" tools/exp_two_layer_universality.py' in /opt/MM_D-ND succeeded in 0ms: 170:OBSERVABLES = { 183:def gen_prime_gaps(N): 188:def gen_gue_spacings(N, rng=None): 204:def gen_poisson_spacings(N, rng=None): 209:def gen_ar1_gaps(N, phi=0.5, rng=None): 220:def gen_logistic_gaps(N, r=3.95): 228:def gen_periodic_gaps(N): 232:def gen_fibonacci_gaps(N): 247:SEQUENCES = { 258:def classify_layer(z_mk0, z_mk1, z_mk2, threshold=2.0): 278: for seq_name, gen_fn in SEQUENCES.items(): 296: for obs_name, obs_fn in OBSERVABLES.items(): 308: surr_obs = {name: [] for name in OBSERVABLES} 316: for obs_name, obs_fn in OBSERVABLES.items(): 323: for obs_name in OBSERVABLES: 336: for obs_name in OBSERVABLES: 359: obs_names = list(OBSERVABLES.keys()) exec /bin/bash -lc "sed -n '240,275p' tools/exp_two_layer_universality.py" in /opt/MM_D-ND succeeded in 0ms: if fibs[-1] > 1e300: break fibs = np.array(fibs, dtype=float) gaps = np.diff(fibs) return gaps[gaps > 0] SEQUENCES = { 'primes': gen_prime_gaps, 'GUE': gen_gue_spacings, 'Poisson': gen_poisson_spacings, 'AR1': gen_ar1_gaps, 'logistic': gen_logistic_gaps, 'periodic_24': gen_periodic_gaps, 'fibonacci': gen_fibonacci_gaps, } def classify_layer(z_mk0, z_mk1, z_mk2, threshold=2.0): """Classify which Markov order captures an observable. Layer 1: |z_Mk0| >> threshold, |z_Mk1| < threshold (captured by pairs) Layer 2: |z_Mk1| >> threshold, |z_Mk2| < threshold (captured by triples) Layer 0: |z_Mk0| < threshold (no memory) Layer 3+: |z_Mk2| >> threshold (needs higher order) """ if abs(z_mk0) < threshold: return 0 # no memory if abs(z_mk1) < threshold: return 1 # pair memory if abs(z_mk2) < threshold: return 2 # triple memory return 3 # higher order def run_experiment(N=100000, n_surr=30, seed=42): rng = np.random.default_rng(seed) codex Creo uno strumento riusabile `exp_markov_layer_recovery_audit.py`: non cambia gli esperimenti precedenti, li usa come oggetto sotto test. Misura quante volte la pipeline assegna a una sequenza un layer superiore all’ordine Markov con cui quella sequenza è stata generata. apply patch patch: completed /opt/MM_D-ND/tools/exp_markov_layer_recovery_audit.py diff --git a/tools/exp_markov_layer_recovery_audit.py b/tools/exp_markov_layer_recovery_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..9797ea889603167a58cd989c71d02cf2aaf927bf --- /dev/null +++ b/tools/exp_markov_layer_recovery_audit.py @@ -0,0 +1,179 @@ +#!/usr/bin/env python3 +""" +exp_markov_layer_recovery_audit.py — META audit for the Markov-layer classifier. + +Question: + If a sequence is generated with known Markov order k, does the existing + Mk0/Mk1/Mk2 classifier recover layer <= k? + +Why this matters: + Recent reports used Mk surrogate capture to claim two visible memory layers. + This audit tests the opposite: whether the classifier itself creates apparent + higher layers on finite Markov controls. + +Controls: + - prime_gaps: unknown real target, reported for comparison only + - empirical_Mk0: shuffled prime gaps; known order 0 + - empirical_Mk1: prime-trained Mk1 surrogate; known order 1 + - empirical_Mk2: prime-trained Mk2 surrogate; known order 2 + - poisson_iid: independent exponential spacings; known order 0 + +Verdict rule: + A control fails recovery when any observable is assigned layer > known order. +""" + +import argparse +import json +from pathlib import Path + +import numpy as np + +from exp_two_layer_universality import ( + OBSERVABLES, + classify_layer, + gen_prime_gaps, + generate_markov_surrogate, +) + + +def measure_all(gaps): + out = {} + for name, fn in OBSERVABLES.items(): + try: + value = float(fn(gaps)) + except Exception: + value = float("nan") + out[name] = value + return out + + +def classifier_pass(gaps, n_surr, rng): + real_obs = measure_all(gaps) + z_scores = {} + + for mk in (0, 1, 2): + surr_obs = {name: [] for name in OBSERVABLES} + for _ in range(n_surr): + if mk == 0: + surr = rng.permutation(gaps) + else: + surr = generate_markov_surrogate(gaps, mk, rng=rng) + for obs_name, obs_fn in OBSERVABLES.items(): + try: + surr_obs[obs_name].append(float(obs_fn(surr))) + except Exception: + pass + + z_mk = {} + for obs_name in OBSERVABLES: + vals = np.asarray(surr_obs[obs_name], dtype=float) + vals = vals[np.isfinite(vals)] + if len(vals) > 2 and np.std(vals) > 1e-12 and np.isfinite(real_obs[obs_name]): + z = (real_obs[obs_name] - np.mean(vals)) / np.std(vals) + else: + z = 0.0 + z_mk[obs_name] = round(float(z), 2) + z_scores[f"Mk{mk}"] = z_mk + + layers = {} + for obs_name in OBSERVABLES: + layers[obs_name] = classify_layer( + z_scores["Mk0"][obs_name], + z_scores["Mk1"][obs_name], + z_scores["Mk2"][obs_name], + ) + + return real_obs, z_scores, layers + + +def build_controls(prime_gaps, rng): + return { + "prime_gaps": { + "known_order": None, + "gaps": prime_gaps, + }, + "empirical_Mk0": { + "known_order": 0, + "gaps": rng.permutation(prime_gaps), + }, + "empirical_Mk1": { + "known_order": 1, + "gaps": generate_markov_surrogate(prime_gaps, 1, rng=rng), + }, + "empirical_Mk2": { + "known_order": 2, + "gaps": generate_markov_surrogate(prime_gaps, 2, rng=rng), + }, + "poisson_iid": { + "known_order": 0, + "gaps": rng.exponential(1.0, len(prime_gaps)), + }, + } + + +def summarize_failure(layers, known_order): + if known_order is None: + return { + "status": "target_only", + "over_layer_observables": [], + "max_layer": int(max(layers.values())), + } + + over = [name for name, layer in layers.items() if layer > known_order] + return { + "status": "PASS" if not over else "FAIL", + "over_layer_observables": over, + "max_layer": int(max(layers.values())), + } + + +def run(N=60000, n_surr=20, seed=20260504): + rng = np.random.default_rng(seed) + prime_gaps = gen_prime_gaps(N).astype(float) + controls = build_controls(prime_gaps, rng) + + results = { + "N": int(N), + "n_surr": int(n_surr), + "seed": int(seed), + "controls": {}, + } + + print(f"N={N}, n_surr={n_surr}, seed={seed}") + print(f"{'sequence':<16} {'known':<7} {'status':<8} {'maxL':<5} over-layer observables") + print("-" * 82) + + for name, spec in controls.items(): + real_obs, z_scores, layers = classifier_pass(spec["gaps"], n_surr, rng) + summary = summarize_failure(layers, spec["known_order"]) + known = "target" if spec["known_order"] is None else f"L{spec['known_order']}" + over = ", ".join(summary["over_layer_observables"]) or "-" + print(f"{name:<16} {known:<7} {summary['status']:<8} L{summary['max_layer']:<4} {over}") + + results["controls"][name] = { + "known_order": spec["known_order"], + "n_gaps": int(len(spec["gaps"])), + "real_obs": {k: round(v, 6) for k, v in real_obs.items()}, + "z_scores": z_scores, + "layers": layers, + "recovery": summary, + } + + out_path = Path("tools/data/markov_layer_recovery_audit.json") + with out_path.open("w") as f: + json.dump(results, f, indent=2) + print(f"\nSaved to {out_path}") + return results + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--N", type=int, default=60000) + parser.add_argument("--n_surr", type=int, default=20) + parser.add_argument("--seed", type=int, default=20260504) + args = parser.parse_args() + run(N=args.N, n_surr=args.n_surr, seed=args.seed) + + +if __name__ == "__main__": + main() diff --git a/tools/exp_markov_layer_recovery_audit.py b/tools/exp_markov_layer_recovery_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..9797ea889603167a58cd989c71d02cf2aaf927bf --- /dev/null +++ b/tools/exp_markov_layer_recovery_audit.py @@ -0,0 +1,179 @@ +#!/usr/bin/env python3 +""" +exp_markov_layer_recovery_audit.py — META audit for the Markov-layer classifier. + +Question: + If a sequence is generated with known Markov order k, does the existing + Mk0/Mk1/Mk2 classifier recover layer <= k? + +Why this matters: + Recent reports used Mk surrogate capture to claim two visible memory layers. + This audit tests the opposite: whether the classifier itself creates apparent + higher layers on finite Markov controls. + +Controls: + - prime_gaps: unknown real target, reported for comparison only + - empirical_Mk0: shuffled prime gaps; known order 0 + - empirical_Mk1: prime-trained Mk1 surrogate; known order 1 + - empirical_Mk2: prime-trained Mk2 surrogate; known order 2 + - poisson_iid: independent exponential spacings; known order 0 + +Verdict rule: + A control fails recovery when any observable is assigned layer > known order. +""" + +import argparse +import json +from pathlib import Path + +import numpy as np + +from exp_two_layer_universality import ( + OBSERVABLES, + classify_layer, + gen_prime_gaps, + generate_markov_surrogate, +) + + +def measure_all(gaps): + out = {} + for name, fn in OBSERVABLES.items(): + try: + value = float(fn(gaps)) + except Exception: + value = float("nan") + out[name] = value + return out + + +def classifier_pass(gaps, n_surr, rng): + real_obs = measure_all(gaps) + z_scores = {} + + for mk in (0, 1, 2): + surr_obs = {name: [] for name in OBSERVABLES} + for _ in range(n_surr): + if mk == 0: + surr = rng.permutation(gaps) + else: + surr = generate_markov_surrogate(gaps, mk, rng=rng) + for obs_name, obs_fn in OBSERVABLES.items(): + try: + surr_obs[obs_name].append(float(obs_fn(surr))) + except Exception: + pass + + z_mk = {} + for obs_name in OBSERVABLES: + vals = np.asarray(surr_obs[obs_name], dtype=float) + vals = vals[np.isfinite(vals)] + if len(vals) > 2 and np.std(vals) > 1e-12 and np.isfinite(real_obs[obs_name]): + z = (real_obs[obs_name] - np.mean(vals)) / np.std(vals) + else: + z = 0.0 + z_mk[obs_name] = round(float(z), 2) + z_scores[f"Mk{mk}"] = z_mk + + layers = {} + for obs_name in OBSERVABLES: + layers[obs_name] = classify_layer( + z_scores["Mk0"][obs_name], + z_scores["Mk1"][obs_name], + z_scores["Mk2"][obs_name], + ) + + return real_obs, z_scores, layers + + +def build_controls(prime_gaps, rng): + return { + "prime_gaps": { + "known_order": None, + "gaps": prime_gaps, + }, + "empirical_Mk0": { + "known_order": 0, + "gaps": rng.permutation(prime_gaps), + }, + "empirical_Mk1": { + "known_order": 1, + "gaps": generate_markov_surrogate(prime_gaps, 1, rng=rng), + }, + "empirical_Mk2": { + "known_order": 2, + "gaps": generate_markov_surrogate(prime_gaps, 2, rng=rng), + }, + "poisson_iid": { + "known_order": 0, + "gaps": rng.exponential(1.0, len(prime_gaps)), + }, + } + + +def summarize_failure(layers, known_order): + if known_order is None: + return { + "status": "target_only", + "over_layer_observables": [], + "max_layer": int(max(layers.values())), + } + + over = [name for name, layer in layers.items() if layer > known_order] + return { + "status": "PASS" if not over else "FAIL", + "over_layer_observables": over, + "max_layer": int(max(layers.values())), + } + + +def run(N=60000, n_surr=20, seed=20260504): + rng = np.random.default_rng(seed) + prime_gaps = gen_prime_gaps(N).astype(float) + controls = build_controls(prime_gaps, rng) + + results = { + "N": int(N), + "n_surr": int(n_surr), + "seed": int(seed), + "controls": {}, + } + + print(f"N={N}, n_surr={n_surr}, seed={seed}") + print(f"{'sequence':<16} {'known':<7} {'status':<8} {'maxL':<5} over-layer observables") + print("-" * 82) + + for name, spec in controls.items(): + real_obs, z_scores, layers = classifier_pass(spec["gaps"], n_surr, rng) + summary = summarize_failure(layers, spec["known_order"]) + known = "target" if spec["known_order"] is None else f"L{spec['known_order']}" + over = ", ".join(summary["over_layer_observables"]) or "-" + print(f"{name:<16} {known:<7} {summary['status']:<8} L{summary['max_layer']:<4} {over}") + + results["controls"][name] = { + "known_order": spec["known_order"], + "n_gaps": int(len(spec["gaps"])), + "real_obs": {k: round(v, 6) for k, v in real_obs.items()}, + "z_scores": z_scores, + "layers": layers, + "recovery": summary, + } + + out_path = Path("tools/data/markov_layer_recovery_audit.json") + with out_path.open("w") as f: + json.dump(results, f, indent=2) + print(f"\nSaved to {out_path}") + return results + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--N", type=int, default=60000) + parser.add_argument("--n_surr", type=int, default=20) + parser.add_argument("--seed", type=int, default=20260504) + args = parser.parse_args() + run(N=args.N, n_surr=args.n_surr, seed=args.seed) + + +if __name__ == "__main__": + main() codex Lo strumento è pronto. Ora lo eseguo con campione moderato e null multipli; se regge, aumento solo se serve, restando entro il ciclo operativo. exec /bin/bash -lc 'python tools/exp_markov_layer_recovery_audit.py --N 60000 --n_surr 20' in /opt/MM_D-ND diff --git a/tools/exp_markov_layer_recovery_audit.py b/tools/exp_markov_layer_recovery_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..9797ea889603167a58cd989c71d02cf2aaf927bf --- /dev/null +++ b/tools/exp_markov_layer_recovery_audit.py @@ -0,0 +1,179 @@ +#!/usr/bin/env python3 +""" +exp_markov_layer_recovery_audit.py — META audit for the Markov-layer classifier. + +Question: + If a sequence is generated with known Markov order k, does the existing + Mk0/Mk1/Mk2 classifier recover layer <= k? + +Why this matters: + Recent reports used Mk surrogate capture to claim two visible memory layers. + This audit tests the opposite: whether the classifier itself creates apparent + higher layers on finite Markov controls. + +Controls: + - prime_gaps: unknown real target, reported for comparison only + - empirical_Mk0: shuffled prime gaps; known order 0 + - empirical_Mk1: prime-trained Mk1 surrogate; known order 1 + - empirical_Mk2: prime-trained Mk2 surrogate; known order 2 + - poisson_iid: independent exponential spacings; known order 0 + +Verdict rule: + A control fails recovery when any observable is assigned layer > known order. +""" + +import argparse +import json +from pathlib import Path + +import numpy as np + +from exp_two_layer_universality import ( + OBSERVABLES, + classify_layer, + gen_prime_gaps, + generate_markov_surrogate, +) + + +def measure_all(gaps): + out = {} + for name, fn in OBSERVABLES.items(): + try: + value = float(fn(gaps)) + except Exception: + value = float("nan") + out[name] = value + return out + + +def classifier_pass(gaps, n_surr, rng): + real_obs = measure_all(gaps) + z_scores = {} + + for mk in (0, 1, 2): + surr_obs = {name: [] for name in OBSERVABLES} + for _ in range(n_surr): + if mk == 0: + surr = rng.permutation(gaps) + else: + surr = generate_markov_surrogate(gaps, mk, rng=rng) + for obs_name, obs_fn in OBSERVABLES.items(): + try: + surr_obs[obs_name].append(float(obs_fn(surr))) + except Exception: + pass + + z_mk = {} + for obs_name in OBSERVABLES: + vals = np.asarray(surr_obs[obs_name], dtype=float) + vals = vals[np.isfinite(vals)] + if len(vals) > 2 and np.std(vals) > 1e-12 and np.isfinite(real_obs[obs_name]): + z = (real_obs[obs_name] - np.mean(vals)) / np.std(vals) + else: + z = 0.0 + z_mk[obs_name] = round(float(z), 2) + z_scores[f"Mk{mk}"] = z_mk + + layers = {} + for obs_name in OBSERVABLES: + layers[obs_name] = classify_layer( + z_scores["Mk0"][obs_name], + z_scores["Mk1"][obs_name], + z_scores["Mk2"][obs_name], + ) + + return real_obs, z_scores, layers + + +def build_controls(prime_gaps, rng): + return { + "prime_gaps": { + "known_order": None, + "gaps": prime_gaps, + }, + "empirical_Mk0": { + "known_order": 0, + "gaps": rng.permutation(prime_gaps), + }, + "empirical_Mk1": { + "known_order": 1, + "gaps": generate_markov_surrogate(prime_gaps, 1, rng=rng), + }, + "empirical_Mk2": { + "known_order": 2, + "gaps": generate_markov_surrogate(prime_gaps, 2, rng=rng), + }, + "poisson_iid": { + "known_order": 0, + "gaps": rng.exponential(1.0, len(prime_gaps)), + }, + } + + +def summarize_failure(layers, known_order): + if known_order is None: + return { + "status": "target_only", + "over_layer_observables": [], + "max_layer": int(max(layers.values())), + } + + over = [name for name, layer in layers.items() if layer > known_order] + return { + "status": "PASS" if not over else "FAIL", + "over_layer_observables": over, + "max_layer": int(max(layers.values())), + } + + +def run(N=60000, n_surr=20, seed=20260504): + rng = np.random.default_rng(seed) + prime_gaps = gen_prime_gaps(N).astype(float) + controls = build_controls(prime_gaps, rng) + + results = { + "N": int(N), + "n_surr": int(n_surr), + "seed": int(seed), + "controls": {}, + } + + print(f"N={N}, n_surr={n_surr}, seed={seed}") + print(f"{'sequence':<16} {'known':<7} {'status':<8} {'maxL':<5} over-layer observables") + print("-" * 82) + + for name, spec in controls.items(): + real_obs, z_scores, layers = classifier_pass(spec["gaps"], n_surr, rng) + summary = summarize_failure(layers, spec["known_order"]) + known = "target" if spec["known_order"] is None else f"L{spec['known_order']}" + over = ", ".join(summary["over_layer_observables"]) or "-" + print(f"{name:<16} {known:<7} {summary['status']:<8} L{summary['max_layer']:<4} {over}") + + results["controls"][name] = { + "known_order": spec["known_order"], + "n_gaps": int(len(spec["gaps"])), + "real_obs": {k: round(v, 6) for k, v in real_obs.items()}, + "z_scores": z_scores, + "layers": layers, + "recovery": summary, + } + + out_path = Path("tools/data/markov_layer_recovery_audit.json") + with out_path.open("w") as f: + json.dump(results, f, indent=2) + print(f"\nSaved to {out_path}") + return results + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--N", type=int, default=60000) + parser.add_argument("--n_surr", type=int, default=20) + parser.add_argument("--seed", type=int, default=20260504) + args = parser.parse_args() + run(N=args.N, n_surr=args.n_surr, seed=args.seed) + + +if __name__ == "__main__": + main() diff --git a/tools/exp_markov_layer_recovery_audit.py b/tools/exp_markov_layer_recovery_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..9797ea889603167a58cd989c71d02cf2aaf927bf --- /dev/null +++ b/tools/exp_markov_layer_recovery_audit.py @@ -0,0 +1,179 @@ +#!/usr/bin/env python3 +""" +exp_markov_layer_recovery_audit.py — META audit for the Markov-layer classifier. + +Question: + If a sequence is generated with known Markov order k, does the existing + Mk0/Mk1/Mk2 classifier recover layer <= k? + +Why this matters: + Recent reports used Mk surrogate capture to claim two visible memory layers. + This audit tests the opposite: whether the classifier itself creates apparent + higher layers on finite Markov controls. + +Controls: + - prime_gaps: unknown real target, reported for comparison only + - empirical_Mk0: shuffled prime gaps; known order 0 + - empirical_Mk1: prime-trained Mk1 surrogate; known order 1 + - empirical_Mk2: prime-trained Mk2 surrogate; known order 2 + - poisson_iid: independent exponential spacings; known order 0 + +Verdict rule: + A control fails recovery when any observable is assigned layer > known order. +""" + +import argparse +import json +from pathlib import Path + +import numpy as np + +from exp_two_layer_universality import ( + OBSERVABLES, + classify_layer, + gen_prime_gaps, + generate_markov_surrogate, +) + + +def measure_all(gaps): + out = {} + for name, fn in OBSERVABLES.items(): + try: + value = float(fn(gaps)) + except Exception: + value = float("nan") + out[name] = value + return out + + +def classifier_pass(gaps, n_surr, rng): + real_obs = measure_all(gaps) + z_scores = {} + + for mk in (0, 1, 2): + surr_obs = {name: [] for name in OBSERVABLES} + for _ in range(n_surr): + if mk == 0: + surr = rng.permutation(gaps) + else: + surr = generate_markov_surrogate(gaps, mk, rng=rng) + for obs_name, obs_fn in OBSERVABLES.items(): + try: + surr_obs[obs_name].append(float(obs_fn(surr))) + except Exception: + pass + + z_mk = {} + for obs_name in OBSERVABLES: + vals = np.asarray(surr_obs[obs_name], dtype=float) + vals = vals[np.isfinite(vals)] + if len(vals) > 2 and np.std(vals) > 1e-12 and np.isfinite(real_obs[obs_name]): + z = (real_obs[obs_name] - np.mean(vals)) / np.std(vals) + else: + z = 0.0 + z_mk[obs_name] = round(float(z), 2) + z_scores[f"Mk{mk}"] = z_mk + + layers = {} + for obs_name in OBSERVABLES: + layers[obs_name] = classify_layer( + z_scores["Mk0"][obs_name], + z_scores["Mk1"][obs_name], + z_scores["Mk2"][obs_name], + ) + + return real_obs, z_scores, layers + + +def build_controls(prime_gaps, rng): + return { + "prime_gaps": { + "known_order": None, + "gaps": prime_gaps, + }, + "empirical_Mk0": { + "known_order": 0, + "gaps": rng.permutation(prime_gaps), + }, + "empirical_Mk1": { + "known_order": 1, + "gaps": generate_markov_surrogate(prime_gaps, 1, rng=rng), + }, + "empirical_Mk2": { + "known_order": 2, + "gaps": generate_markov_surrogate(prime_gaps, 2, rng=rng), + }, + "poisson_iid": { + "known_order": 0, + "gaps": rng.exponential(1.0, len(prime_gaps)), + }, + } + + +def summarize_failure(layers, known_order): + if known_order is None: + return { + "status": "target_only", + "over_layer_observables": [], + "max_layer": int(max(layers.values())), + } + + over = [name for name, layer in layers.items() if layer > known_order] + return { + "status": "PASS" if not over else "FAIL", + "over_layer_observables": over, + "max_layer": int(max(layers.values())), + } + + +def run(N=60000, n_surr=20, seed=20260504): + rng = np.random.default_rng(seed) + prime_gaps = gen_prime_gaps(N).astype(float) + controls = build_controls(prime_gaps, rng) + + results = { + "N": int(N), + "n_surr": int(n_surr), + "seed": int(seed), + "controls": {}, + } + + print(f"N={N}, n_surr={n_surr}, seed={seed}") + print(f"{'sequence':<16} {'known':<7} {'status':<8} {'maxL':<5} over-layer observables") + print("-" * 82) + + for name, spec in controls.items(): + real_obs, z_scores, layers = classifier_pass(spec["gaps"], n_surr, rng) + summary = summarize_failure(layers, spec["known_order"]) + known = "target" if spec["known_order"] is None else f"L{spec['known_order']}" + over = ", ".join(summary["over_layer_observables"]) or "-" + print(f"{name:<16} {known:<7} {summary['status']:<8} L{summary['max_layer']:<4} {over}") + + results["controls"][name] = { + "known_order": spec["known_order"], + "n_gaps": int(len(spec["gaps"])), + "real_obs": {k: round(v, 6) for k, v in real_obs.items()}, + "z_scores": z_scores, + "layers": layers, + "recovery": summary, + } + + out_path = Path("tools/data/markov_layer_recovery_audit.json") + with out_path.open("w") as f: + json.dump(results, f, indent=2) + print(f"\nSaved to {out_path}") + return results + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--N", type=int, default=60000) + parser.add_argument("--n_surr", type=int, default=20) + parser.add_argument("--seed", type=int, default=20260504) + args = parser.parse_args() + run(N=args.N, n_surr=args.n_surr, seed=args.seed) + + +if __name__ == "__main__": + main() codex Il run è ancora in corso: la parte costosa è generare surrogati Mk per ogni controllo e ricalcolare tutti gli osservabili. Aspetto l’output completo prima di interpretare, perché qui il risultato utile è proprio il pattern di fallimento o recupero. diff --git a/tools/exp_markov_layer_recovery_audit.py b/tools/exp_markov_layer_recovery_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..9797ea889603167a58cd989c71d02cf2aaf927bf --- /dev/null +++ b/tools/exp_markov_layer_recovery_audit.py @@ -0,0 +1,179 @@ +#!/usr/bin/env python3 +""" +exp_markov_layer_recovery_audit.py — META audit for the Markov-layer classifier. + +Question: + If a sequence is generated with known Markov order k, does the existing + Mk0/Mk1/Mk2 classifier recover layer <= k? + +Why this matters: + Recent reports used Mk surrogate capture to claim two visible memory layers. + This audit tests the opposite: whether the classifier itself creates apparent + higher layers on finite Markov controls. + +Controls: + - prime_gaps: unknown real target, reported for comparison only + - empirical_Mk0: shuffled prime gaps; known order 0 + - empirical_Mk1: prime-trained Mk1 surrogate; known order 1 + - empirical_Mk2: prime-trained Mk2 surrogate; known order 2 + - poisson_iid: independent exponential spacings; known order 0 + +Verdict rule: + A control fails recovery when any observable is assigned layer > known order. +""" + +import argparse +import json +from pathlib import Path + +import numpy as np + +from exp_two_layer_universality import ( + OBSERVABLES, + classify_layer, + gen_prime_gaps, + generate_markov_surrogate, +) + + +def measure_all(gaps): + out = {} + for name, fn in OBSERVABLES.items(): + try: + value = float(fn(gaps)) + except Exception: + value = float("nan") + out[name] = value + return out + + +def classifier_pass(gaps, n_surr, rng): + real_obs = measure_all(gaps) + z_scores = {} + + for mk in (0, 1, 2): + surr_obs = {name: [] for name in OBSERVABLES} + for _ in range(n_surr): + if mk == 0: + surr = rng.permutation(gaps) + else: + surr = generate_markov_surrogate(gaps, mk, rng=rng) + for obs_name, obs_fn in OBSERVABLES.items(): + try: + surr_obs[obs_name].append(float(obs_fn(surr))) + except Exception: + pass + + z_mk = {} + for obs_name in OBSERVABLES: + vals = np.asarray(surr_obs[obs_name], dtype=float) + vals = vals[np.isfinite(vals)] + if len(vals) > 2 and np.std(vals) > 1e-12 and np.isfinite(real_obs[obs_name]): + z = (real_obs[obs_name] - np.mean(vals)) / np.std(vals) + else: + z = 0.0 + z_mk[obs_name] = round(float(z), 2) + z_scores[f"Mk{mk}"] = z_mk + + layers = {} + for obs_name in OBSERVABLES: + layers[obs_name] = classify_layer( + z_scores["Mk0"][obs_name], + z_scores["Mk1"][obs_name], + z_scores["Mk2"][obs_name], + ) + + return real_obs, z_scores, layers + + +def build_controls(prime_gaps, rng): + return { + "prime_gaps": { + "known_order": None, + "gaps": prime_gaps, + }, + "empirical_Mk0": { + "known_order": 0, + "gaps": rng.permutation(prime_gaps), + }, + "empirical_Mk1": { + "known_order": 1, + "gaps": generate_markov_surrogate(prime_gaps, 1, rng=rng), + }, + "empirical_Mk2": { + "known_order": 2, + "gaps": generate_markov_surrogate(prime_gaps, 2, rng=rng), + }, + "poisson_iid": { + "known_order": 0, + "gaps": rng.exponential(1.0, len(prime_gaps)), + }, + } + + +def summarize_failure(layers, known_order): + if known_order is None: + return { + "status": "target_only", + "over_layer_observables": [], + "max_layer": int(max(layers.values())), + } + + over = [name for name, layer in layers.items() if layer > known_order] + return { + "status": "PASS" if not over else "FAIL", + "over_layer_observables": over, + "max_layer": int(max(layers.values())), + } + + +def run(N=60000, n_surr=20, seed=20260504): + rng = np.random.default_rng(seed) + prime_gaps = gen_prime_gaps(N).astype(float) + controls = build_controls(prime_gaps, rng) + + results = { + "N": int(N), + "n_surr": int(n_surr), + "seed": int(seed), + "controls": {}, + } + + print(f"N={N}, n_surr={n_surr}, seed={seed}") + print(f"{'sequence':<16} {'known':<7} {'status':<8} {'maxL':<5} over-layer observables") + print("-" * 82) + + for name, spec in controls.items(): + real_obs, z_scores, layers = classifier_pass(spec["gaps"], n_surr, rng) + summary = summarize_failure(layers, spec["known_order"]) + known = "target" if spec["known_order"] is None else f"L{spec['known_order']}" + over = ", ".join(summary["over_layer_observables"]) or "-" + print(f"{name:<16} {known:<7} {summary['status']:<8} L{summary['max_layer']:<4} {over}") + + results["controls"][name] = { + "known_order": spec["known_order"], + "n_gaps": int(len(spec["gaps"])), + "real_obs": {k: round(v, 6) for k, v in real_obs.items()}, + "z_scores": z_scores, + "layers": layers, + "recovery": summary, + } + + out_path = Path("tools/data/markov_layer_recovery_audit.json") + with out_path.open("w") as f: + json.dump(results, f, indent=2) + print(f"\nSaved to {out_path}") + return results + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--N", type=int, default=60000) + parser.add_argument("--n_surr", type=int, default=20) + parser.add_argument("--seed", type=int, default=20260504) + args = parser.parse_args() + run(N=args.N, n_surr=args.n_surr, seed=args.seed) + + +if __name__ == "__main__": + main() diff --git a/tools/exp_markov_layer_recovery_audit.py b/tools/exp_markov_layer_recovery_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..9797ea889603167a58cd989c71d02cf2aaf927bf --- /dev/null +++ b/tools/exp_markov_layer_recovery_audit.py @@ -0,0 +1,179 @@ +#!/usr/bin/env python3 +""" +exp_markov_layer_recovery_audit.py — META audit for the Markov-layer classifier. + +Question: + If a sequence is generated with known Markov order k, does the existing + Mk0/Mk1/Mk2 classifier recover layer <= k? + +Why this matters: + Recent reports used Mk surrogate capture to claim two visible memory layers. + This audit tests the opposite: whether the classifier itself creates apparent + higher layers on finite Markov controls. + +Controls: + - prime_gaps: unknown real target, reported for comparison only + - empirical_Mk0: shuffled prime gaps; known order 0 + - empirical_Mk1: prime-trained Mk1 surrogate; known order 1 + - empirical_Mk2: prime-trained Mk2 surrogate; known order 2 + - poisson_iid: independent exponential spacings; known order 0 + +Verdict rule: + A control fails recovery when any observable is assigned layer > known order. +""" + +import argparse +import json +from pathlib import Path + +import numpy as np + +from exp_two_layer_universality import ( + OBSERVABLES, + classify_layer, + gen_prime_gaps, + generate_markov_surrogate, +) + + +def measure_all(gaps): + out = {} + for name, fn in OBSERVABLES.items(): + try: + value = float(fn(gaps)) + except Exception: + value = float("nan") + out[name] = value + return out + + +def classifier_pass(gaps, n_surr, rng): + real_obs = measure_all(gaps) + z_scores = {} + + for mk in (0, 1, 2): + surr_obs = {name: [] for name in OBSERVABLES} + for _ in range(n_surr): + if mk == 0: + surr = rng.permutation(gaps) + else: + surr = generate_markov_surrogate(gaps, mk, rng=rng) + for obs_name, obs_fn in OBSERVABLES.items(): + try: + surr_obs[obs_name].append(float(obs_fn(surr))) + except Exception: + pass + + z_mk = {} + for obs_name in OBSERVABLES: + vals = np.asarray(surr_obs[obs_name], dtype=float) + vals = vals[np.isfinite(vals)] + if len(vals) > 2 and np.std(vals) > 1e-12 and np.isfinite(real_obs[obs_name]): + z = (real_obs[obs_name] - np.mean(vals)) / np.std(vals) + else: + z = 0.0 + z_mk[obs_name] = round(float(z), 2) + z_scores[f"Mk{mk}"] = z_mk + + layers = {} + for obs_name in OBSERVABLES: + layers[obs_name] = classify_layer( + z_scores["Mk0"][obs_name], + z_scores["Mk1"][obs_name], + z_scores["Mk2"][obs_name], + ) + + return real_obs, z_scores, layers + + +def build_controls(prime_gaps, rng): + return { + "prime_gaps": { + "known_order": None, + "gaps": prime_gaps, + }, + "empirical_Mk0": { + "known_order": 0, + "gaps": rng.permutation(prime_gaps), + }, + "empirical_Mk1": { + "known_order": 1, + "gaps": generate_markov_surrogate(prime_gaps, 1, rng=rng), + }, + "empirical_Mk2": { + "known_order": 2, + "gaps": generate_markov_surrogate(prime_gaps, 2, rng=rng), + }, + "poisson_iid": { + "known_order": 0, + "gaps": rng.exponential(1.0, len(prime_gaps)), + }, + } + + +def summarize_failure(layers, known_order): + if known_order is None: + return { + "status": "target_only", + "over_layer_observables": [], + "max_layer": int(max(layers.values())), + } + + over = [name for name, layer in layers.items() if layer > known_order] + return { + "status": "PASS" if not over else "FAIL", + "over_layer_observables": over, + "max_layer": int(max(layers.values())), + } + + +def run(N=60000, n_surr=20, seed=20260504): + rng = np.random.default_rng(seed) + prime_gaps = gen_prime_gaps(N).astype(float) + controls = build_controls(prime_gaps, rng) + + results = { + "N": int(N), + "n_surr": int(n_surr), + "seed": int(seed), + "controls": {}, + } + + print(f"N={N}, n_surr={n_surr}, seed={seed}") + print(f"{'sequence':<16} {'known':<7} {'status':<8} {'maxL':<5} over-layer observables") + print("-" * 82) + + for name, spec in controls.items(): + real_obs, z_scores, layers = classifier_pass(spec["gaps"], n_surr, rng) + summary = summarize_failure(layers, spec["known_order"]) + known = "target" if spec["known_order"] is None else f"L{spec['known_order']}" + over = ", ".join(summary["over_layer_observables"]) or "-" + print(f"{name:<16} {known:<7} {summary['status']:<8} L{summary['max_layer']:<4} {over}") + + results["controls"][name] = { + "known_order": spec["known_order"], + "n_gaps": int(len(spec["gaps"])), + "real_obs": {k: round(v, 6) for k, v in real_obs.items()}, + "z_scores": z_scores, + "layers": layers, + "recovery": summary, + } + + out_path = Path("tools/data/markov_layer_recovery_audit.json") + with out_path.open("w") as f: + json.dump(results, f, indent=2) + print(f"\nSaved to {out_path}") + return results + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--N", type=int, default=60000) + parser.add_argument("--n_surr", type=int, default=20) + parser.add_argument("--seed", type=int, default=20260504) + args = parser.parse_args() + run(N=args.N, n_surr=args.n_surr, seed=args.seed) + + +if __name__ == "__main__": + main() codex Il campione da 60k con 20 surrogati sta superando il tempo utile del ciclo. Lo lascio ancora poco, poi se non chiude riduco il setup: per META è più informativo un audit rapido con controlli noti che un run grande ma non chiuso. 2026-05-04T12:22:19.320257Z ERROR codex_core::tools::router: error=write_stdin failed: stdin is closed for this session; rerun exec_command with tty=true to keep stdin open diff --git a/tools/exp_markov_layer_recovery_audit.py b/tools/exp_markov_layer_recovery_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..9797ea889603167a58cd989c71d02cf2aaf927bf --- /dev/null +++ b/tools/exp_markov_layer_recovery_audit.py @@ -0,0 +1,179 @@ +#!/usr/bin/env python3 +""" +exp_markov_layer_recovery_audit.py — META audit for the Markov-layer classifier. + +Question: + If a sequence is generated with known Markov order k, does the existing + Mk0/Mk1/Mk2 classifier recover layer <= k? + +Why this matters: + Recent reports used Mk surrogate capture to claim two visible memory layers. + This audit tests the opposite: whether the classifier itself creates apparent + higher layers on finite Markov controls. + +Controls: + - prime_gaps: unknown real target, reported for comparison only + - empirical_Mk0: shuffled prime gaps; known order 0 + - empirical_Mk1: prime-trained Mk1 surrogate; known order 1 + - empirical_Mk2: prime-trained Mk2 surrogate; known order 2 + - poisson_iid: independent exponential spacings; known order 0 + +Verdict rule: + A control fails recovery when any observable is assigned layer > known order. +""" + +import argparse +import json +from pathlib import Path + +import numpy as np + +from exp_two_layer_universality import ( + OBSERVABLES, + classify_layer, + gen_prime_gaps, + generate_markov_surrogate, +) + + +def measure_all(gaps): + out = {} + for name, fn in OBSERVABLES.items(): + try: + value = float(fn(gaps)) + except Exception: + value = float("nan") + out[name] = value + return out + + +def classifier_pass(gaps, n_surr, rng): + real_obs = measure_all(gaps) + z_scores = {} + + for mk in (0, 1, 2): + surr_obs = {name: [] for name in OBSERVABLES} + for _ in range(n_surr): + if mk == 0: + surr = rng.permutation(gaps) + else: + surr = generate_markov_surrogate(gaps, mk, rng=rng) + for obs_name, obs_fn in OBSERVABLES.items(): + try: + surr_obs[obs_name].append(float(obs_fn(surr))) + except Exception: + pass + + z_mk = {} + for obs_name in OBSERVABLES: + vals = np.asarray(surr_obs[obs_name], dtype=float) + vals = vals[np.isfinite(vals)] + if len(vals) > 2 and np.std(vals) > 1e-12 and np.isfinite(real_obs[obs_name]): + z = (real_obs[obs_name] - np.mean(vals)) / np.std(vals) + else: + z = 0.0 + z_mk[obs_name] = round(float(z), 2) + z_scores[f"Mk{mk}"] = z_mk + + layers = {} + for obs_name in OBSERVABLES: + layers[obs_name] = classify_layer( + z_scores["Mk0"][obs_name], + z_scores["Mk1"][obs_name], + z_scores["Mk2"][obs_name], + ) + + return real_obs, z_scores, layers + + +def build_controls(prime_gaps, rng): + return { + "prime_gaps": { + "known_order": None, + "gaps": prime_gaps, + }, + "empirical_Mk0": { + "known_order": 0, + "gaps": rng.permutation(prime_gaps), + }, + "empirical_Mk1": { + "known_order": 1, + "gaps": generate_markov_surrogate(prime_gaps, 1, rng=rng), + }, + "empirical_Mk2": { + "known_order": 2, + "gaps": generate_markov_surrogate(prime_gaps, 2, rng=rng), + }, + "poisson_iid": { + "known_order": 0, + "gaps": rng.exponential(1.0, len(prime_gaps)), + }, + } + + +def summarize_failure(layers, known_order): + if known_order is None: + return { + "status": "target_only", + "over_layer_observables": [], + "max_layer": int(max(layers.values())), + } + + over = [name for name, layer in layers.items() if layer > known_order] + return { + "status": "PASS" if not over else "FAIL", + "over_layer_observables": over, + "max_layer": int(max(layers.values())), + } + + +def run(N=60000, n_surr=20, seed=20260504): + rng = np.random.default_rng(seed) + prime_gaps = gen_prime_gaps(N).astype(float) + controls = build_controls(prime_gaps, rng) + + results = { + "N": int(N), + "n_surr": int(n_surr), + "seed": int(seed), + "controls": {}, + } + + print(f"N={N}, n_surr={n_surr}, seed={seed}") + print(f"{'sequence':<16} {'known':<7} {'status':<8} {'maxL':<5} over-layer observables") + print("-" * 82) + + for name, spec in controls.items(): + real_obs, z_scores, layers = classifier_pass(spec["gaps"], n_surr, rng) + summary = summarize_failure(layers, spec["known_order"]) + known = "target" if spec["known_order"] is None else f"L{spec['known_order']}" + over = ", ".join(summary["over_layer_observables"]) or "-" + print(f"{name:<16} {known:<7} {summary['status']:<8} L{summary['max_layer']:<4} {over}") + + results["controls"][name] = { + "known_order": spec["known_order"], + "n_gaps": int(len(spec["gaps"])), + "real_obs": {k: round(v, 6) for k, v in real_obs.items()}, + "z_scores": z_scores, + "layers": layers, + "recovery": summary, + } + + out_path = Path("tools/data/markov_layer_recovery_audit.json") + with out_path.open("w") as f: + json.dump(results, f, indent=2) + print(f"\nSaved to {out_path}") + return results + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--N", type=int, default=60000) + parser.add_argument("--n_surr", type=int, default=20) + parser.add_argument("--seed", type=int, default=20260504) + args = parser.parse_args() + run(N=args.N, n_surr=args.n_surr, seed=args.seed) + + +if __name__ == "__main__": + main() diff --git a/tools/exp_markov_layer_recovery_audit.py b/tools/exp_markov_layer_recovery_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..9797ea889603167a58cd989c71d02cf2aaf927bf --- /dev/null +++ b/tools/exp_markov_layer_recovery_audit.py @@ -0,0 +1,179 @@ +#!/usr/bin/env python3 +""" +exp_markov_layer_recovery_audit.py — META audit for the Markov-layer classifier. + +Question: + If a sequence is generated with known Markov order k, does the existing + Mk0/Mk1/Mk2 classifier recover layer <= k? + +Why this matters: + Recent reports used Mk surrogate capture to claim two visible memory layers. + This audit tests the opposite: whether the classifier itself creates apparent + higher layers on finite Markov controls. + +Controls: + - prime_gaps: unknown real target, reported for comparison only + - empirical_Mk0: shuffled prime gaps; known order 0 + - empirical_Mk1: prime-trained Mk1 surrogate; known order 1 + - empirical_Mk2: prime-trained Mk2 surrogate; known order 2 + - poisson_iid: independent exponential spacings; known order 0 + +Verdict rule: + A control fails recovery when any observable is assigned layer > known order. +""" + +import argparse +import json +from pathlib import Path + +import numpy as np + +from exp_two_layer_universality import ( + OBSERVABLES, + classify_layer, + gen_prime_gaps, + generate_markov_surrogate, +) + + +def measure_all(gaps): + out = {} + for name, fn in OBSERVABLES.items(): + try: + value = float(fn(gaps)) + except Exception: + value = float("nan") + out[name] = value + return out + + +def classifier_pass(gaps, n_surr, rng): + real_obs = measure_all(gaps) + z_scores = {} + + for mk in (0, 1, 2): + surr_obs = {name: [] for name in OBSERVABLES} + for _ in range(n_surr): + if mk == 0: + surr = rng.permutation(gaps) + else: + surr = generate_markov_surrogate(gaps, mk, rng=rng) + for obs_name, obs_fn in OBSERVABLES.items(): + try: + surr_obs[obs_name].append(float(obs_fn(surr))) + except Exception: + pass + + z_mk = {} + for obs_name in OBSERVABLES: + vals = np.asarray(surr_obs[obs_name], dtype=float) + vals = vals[np.isfinite(vals)] + if len(vals) > 2 and np.std(vals) > 1e-12 and np.isfinite(real_obs[obs_name]): + z = (real_obs[obs_name] - np.mean(vals)) / np.std(vals) + else: + z = 0.0 + z_mk[obs_name] = round(float(z), 2) + z_scores[f"Mk{mk}"] = z_mk + + layers = {} + for obs_name in OBSERVABLES: + layers[obs_name] = classify_layer( + z_scores["Mk0"][obs_name], + z_scores["Mk1"][obs_name], + z_scores["Mk2"][obs_name], + ) + + return real_obs, z_scores, layers + + +def build_controls(prime_gaps, rng): + return { + "prime_gaps": { + "known_order": None, + "gaps": prime_gaps, + }, + "empirical_Mk0": { + "known_order": 0, + "gaps": rng.permutation(prime_gaps), + }, + "empirical_Mk1": { + "known_order": 1, + "gaps": generate_markov_surrogate(prime_gaps, 1, rng=rng), + }, + "empirical_Mk2": { + "known_order": 2, + "gaps": generate_markov_surrogate(prime_gaps, 2, rng=rng), + }, + "poisson_iid": { + "known_order": 0, + "gaps": rng.exponential(1.0, len(prime_gaps)), + }, + } + + +def summarize_failure(layers, known_order): + if known_order is None: + return { + "status": "target_only", + "over_layer_observables": [], + "max_layer": int(max(layers.values())), + } + + over = [name for name, layer in layers.items() if layer > known_order] + return { + "status": "PASS" if not over else "FAIL", + "over_layer_observables": over, + "max_layer": int(max(layers.values())), + } + + +def run(N=60000, n_surr=20, seed=20260504): + rng = np.random.default_rng(seed) + prime_gaps = gen_prime_gaps(N).astype(float) + controls = build_controls(prime_gaps, rng) + + results = { + "N": int(N), + "n_surr": int(n_surr), + "seed": int(seed), + "controls": {}, + } + + print(f"N={N}, n_surr={n_surr}, seed={seed}") + print(f"{'sequence':<16} {'known':<7} {'status':<8} {'maxL':<5} over-layer observables") + print("-" * 82) + + for name, spec in controls.items(): + real_obs, z_scores, layers = classifier_pass(spec["gaps"], n_surr, rng) + summary = summarize_failure(layers, spec["known_order"]) + known = "target" if spec["known_order"] is None else f"L{spec['known_order']}" + over = ", ".join(summary["over_layer_observables"]) or "-" + print(f"{name:<16} {known:<7} {summary['status']:<8} L{summary['max_layer']:<4} {over}") + + results["controls"][name] = { + "known_order": spec["known_order"], + "n_gaps": int(len(spec["gaps"])), + "real_obs": {k: round(v, 6) for k, v in real_obs.items()}, + "z_scores": z_scores, + "layers": layers, + "recovery": summary, + } + + out_path = Path("tools/data/markov_layer_recovery_audit.json") + with out_path.open("w") as f: + json.dump(results, f, indent=2) + print(f"\nSaved to {out_path}") + return results + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--N", type=int, default=60000) + parser.add_argument("--n_surr", type=int, default=20) + parser.add_argument("--seed", type=int, default=20260504) + args = parser.parse_args() + run(N=args.N, n_surr=args.n_surr, seed=args.seed) + + +if __name__ == "__main__": + main() diff --git a/tools/exp_markov_layer_recovery_audit.py b/tools/exp_markov_layer_recovery_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..9797ea889603167a58cd989c71d02cf2aaf927bf --- /dev/null +++ b/tools/exp_markov_layer_recovery_audit.py @@ -0,0 +1,179 @@ +#!/usr/bin/env python3 +""" +exp_markov_layer_recovery_audit.py — META audit for the Markov-layer classifier. + +Question: + If a sequence is generated with known Markov order k, does the existing + Mk0/Mk1/Mk2 classifier recover layer <= k? + +Why this matters: + Recent reports used Mk surrogate capture to claim two visible memory layers. + This audit tests the opposite: whether the classifier itself creates apparent + higher layers on finite Markov controls. + +Controls: + - prime_gaps: unknown real target, reported for comparison only + - empirical_Mk0: shuffled prime gaps; known order 0 + - empirical_Mk1: prime-trained Mk1 surrogate; known order 1 + - empirical_Mk2: prime-trained Mk2 surrogate; known order 2 + - poisson_iid: independent exponential spacings; known order 0 + +Verdict rule: + A control fails recovery when any observable is assigned layer > known order. +""" + +import argparse +import json +from pathlib import Path + +import numpy as np + +from exp_two_layer_universality import ( + OBSERVABLES, + classify_layer, + gen_prime_gaps, + generate_markov_surrogate, +) + + +def measure_all(gaps): + out = {} + for name, fn in OBSERVABLES.items(): + try: + value = float(fn(gaps)) + except Exception: + value = float("nan") + out[name] = value + return out + + +def classifier_pass(gaps, n_surr, rng): + real_obs = measure_all(gaps) + z_scores = {} + + for mk in (0, 1, 2): + surr_obs = {name: [] for name in OBSERVABLES} + for _ in range(n_surr): + if mk == 0: + surr = rng.permutation(gaps) + else: + surr = generate_markov_surrogate(gaps, mk, rng=rng) + for obs_name, obs_fn in OBSERVABLES.items(): + try: + surr_obs[obs_name].append(float(obs_fn(surr))) + except Exception: + pass + + z_mk = {} + for obs_name in OBSERVABLES: + vals = np.asarray(surr_obs[obs_name], dtype=float) + vals = vals[np.isfinite(vals)] + if len(vals) > 2 and np.std(vals) > 1e-12 and np.isfinite(real_obs[obs_name]): + z = (real_obs[obs_name] - np.mean(vals)) / np.std(vals) + else: + z = 0.0 + z_mk[obs_name] = round(float(z), 2) + z_scores[f"Mk{mk}"] = z_mk + + layers = {} + for obs_name in OBSERVABLES: + layers[obs_name] = classify_layer( + z_scores["Mk0"][obs_name], + z_scores["Mk1"][obs_name], + z_scores["Mk2"][obs_name], + ) + + return real_obs, z_scores, layers + + +def build_controls(prime_gaps, rng): + return { + "prime_gaps": { + "known_order": None, + "gaps": prime_gaps, + }, + "empirical_Mk0": { + "known_order": 0, + "gaps": rng.permutation(prime_gaps), + }, + "empirical_Mk1": { + "known_order": 1, + "gaps": generate_markov_surrogate(prime_gaps, 1, rng=rng), + }, + "empirical_Mk2": { + "known_order": 2, + "gaps": generate_markov_surrogate(prime_gaps, 2, rng=rng), + }, + "poisson_iid": { + "known_order": 0, + "gaps": rng.exponential(1.0, len(prime_gaps)), + }, + } + + +def summarize_failure(layers, known_order): + if known_order is None: + return { + "status": "target_only", + "over_layer_observables": [], + "max_layer": int(max(layers.values())), + } + + over = [name for name, layer in layers.items() if layer > known_order] + return { + "status": "PASS" if not over else "FAIL", + "over_layer_observables": over, + "max_layer": int(max(layers.values())), + } + + +def run(N=60000, n_surr=20, seed=20260504): + rng = np.random.default_rng(seed) + prime_gaps = gen_prime_gaps(N).astype(float) + controls = build_controls(prime_gaps, rng) + + results = { + "N": int(N), + "n_surr": int(n_surr), + "seed": int(seed), + "controls": {}, + } + + print(f"N={N}, n_surr={n_surr}, seed={seed}") + print(f"{'sequence':<16} {'known':<7} {'status':<8} {'maxL':<5} over-layer observables") + print("-" * 82) + + for name, spec in controls.items(): + real_obs, z_scores, layers = classifier_pass(spec["gaps"], n_surr, rng) + summary = summarize_failure(layers, spec["known_order"]) + known = "target" if spec["known_order"] is None else f"L{spec['known_order']}" + over = ", ".join(summary["over_layer_observables"]) or "-" + print(f"{name:<16} {known:<7} {summary['status']:<8} L{summary['max_layer']:<4} {over}") + + results["controls"][name] = { + "known_order": spec["known_order"], + "n_gaps": int(len(spec["gaps"])), + "real_obs": {k: round(v, 6) for k, v in real_obs.items()}, + "z_scores": z_scores, + "layers": layers, + "recovery": summary, + } + + out_path = Path("tools/data/markov_layer_recovery_audit.json") + with out_path.open("w") as f: + json.dump(results, f, indent=2) + print(f"\nSaved to {out_path}") + return results + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--N", type=int, default=60000) + parser.add_argument("--n_surr", type=int, default=20) + parser.add_argument("--seed", type=int, default=20260504) + args = parser.parse_args() + run(N=args.N, n_surr=args.n_surr, seed=args.seed) + + +if __name__ == "__main__": + main() succeeded in 229868ms: N=60000, n_surr=20, seed=20260504 sequence known status maxL over-layer observables ---------------------------------------------------------------------------------- prime_gaps target target_only L3 - empirical_Mk0 L0 PASS L0 - empirical_Mk1 L1 PASS L1 - empirical_Mk2 L2 FAIL L3 cond_entropy poisson_iid L0 FAIL L1 num_var_10 Saved to tools/data/markov_layer_recovery_audit.json codex Il risultato non è tutto-pass: due controlli noti falliscono in modo diverso. Ora leggo il JSON per separare contenuto forte da artefatto finite-sample, poi scrivo il report senza usare assoluti dove i numeri non li supportano. exec /bin/bash -lc "jq '.controls | to_entries[] | {name:.key, known:.value.known_order, layers:.value.layers, recovery:.value.recovery, z:.value.z_scores}' tools/data/markov_layer_recovery_audit.json" in /opt/MM_D-ND succeeded in 0ms: { "name": "prime_gaps", "known": null, "layers": { "SR": 1, "L1": 1, "L2": 2, "SR2": 2, "cond_entropy": 3, "triple_var": 1, "num_var_10": 1, "run_length": 3 }, "recovery": { "status": "target_only", "over_layer_observables": [], "max_layer": 3 }, "z": { "Mk0": { "SR": -22.21, "L1": -12.25, "L2": -3.6, "SR2": -6.67, "cond_entropy": -435.4, "triple_var": -15.9, "num_var_10": -7.46, "run_length": 27.59 }, "Mk1": { "SR": -0.08, "L1": 0.79, "L2": -4.09, "SR2": -8.8, "cond_entropy": -26.11, "triple_var": -0.53, "num_var_10": -1.42, "run_length": 7.19 }, "Mk2": { "SR": -0.16, "L1": 0.86, "L2": -0.19, "SR2": -0.33, "cond_entropy": 2.56, "triple_var": 0.36, "num_var_10": 0.0, "run_length": 3.43 } } } { "name": "empirical_Mk0", "known": 0, "layers": { "SR": 0, "L1": 0, "L2": 0, "SR2": 0, "cond_entropy": 0, "triple_var": 0, "num_var_10": 0, "run_length": 0 }, "recovery": { "status": "PASS", "over_layer_observables": [], "max_layer": 0 }, "z": { "Mk0": { "SR": 0.19, "L1": -1.24, "L2": 1.19, "SR2": 1.34, "cond_entropy": -0.08, "triple_var": -0.58, "num_var_10": -0.11, "run_length": -1.47 }, "Mk1": { "SR": 0.14, "L1": -0.52, "L2": 0.87, "SR2": 0.29, "cond_entropy": 0.36, "triple_var": -0.09, "num_var_10": 0.24, "run_length": -0.5 }, "Mk2": { "SR": -0.48, "L1": -0.55, "L2": -0.67, "SR2": -0.23, "cond_entropy": 6.86, "triple_var": -0.16, "num_var_10": -0.27, "run_length": -0.52 } } } { "name": "empirical_Mk1", "known": 1, "layers": { "SR": 1, "L1": 1, "L2": 0, "SR2": 1, "cond_entropy": 1, "triple_var": 1, "num_var_10": 1, "run_length": 1 }, "recovery": { "status": "PASS", "over_layer_observables": [], "max_layer": 1 }, "z": { "Mk0": { "SR": -20.29, "L1": -13.45, "L2": -0.26, "SR2": 3.91, "cond_entropy": -270.9, "triple_var": -10.92, "num_var_10": -9.1, "run_length": 23.71 }, "Mk1": { "SR": -0.01, "L1": 1.23, "L2": -0.76, "SR2": -0.69, "cond_entropy": 0.23, "triple_var": 0.57, "num_var_10": -0.52, "run_length": 1.08 }, "Mk2": { "SR": -0.09, "L1": 0.62, "L2": 0.22, "SR2": 0.13, "cond_entropy": 1.62, "triple_var": 0.06, "num_var_10": -1.2, "run_length": -0.03 } } } { "name": "empirical_Mk2", "known": 2, "layers": { "SR": 1, "L1": 1, "L2": 2, "SR2": 2, "cond_entropy": 3, "triple_var": 1, "num_var_10": 2, "run_length": 2 }, "recovery": { "status": "FAIL", "over_layer_observables": [ "cond_entropy" ], "max_layer": 3 }, "z": { "Mk0": { "SR": -22.23, "L1": -10.61, "L2": -4.68, "SR2": -6.3, "cond_entropy": -470.91, "triple_var": -10.23, "num_var_10": -7.93, "run_length": 22.55 }, "Mk1": { "SR": 0.04, "L1": -0.4, "L2": -4.5, "SR2": -6.77, "cond_entropy": -39.3, "triple_var": -1.2, "num_var_10": -2.06, "run_length": 4.43 }, "Mk2": { "SR": -0.08, "L1": -0.37, "L2": 0.46, "SR2": 0.09, "cond_entropy": 3.15, "triple_var": -0.12, "num_var_10": -0.04, "run_length": -0.39 } } } { "name": "poisson_iid", "known": 0, "layers": { "SR": 0, "L1": 0, "L2": 0, "SR2": 0, "cond_entropy": 0, "triple_var": 0, "num_var_10": 1, "run_length": 0 }, "recovery": { "status": "FAIL", "over_layer_observables": [ "num_var_10" ], "max_layer": 1 }, "z": { "Mk0": { "SR": -0.43, "L1": 1.68, "L2": -1.08, "SR2": -1.8, "cond_entropy": -0.05, "triple_var": 1.17, "num_var_10": 2.13, "run_length": 0.36 }, "Mk1": { "SR": -0.34, "L1": -0.48, "L2": -0.33, "SR2": -1.5, "cond_entropy": 3.05, "triple_var": -0.0, "num_var_10": 0.85, "run_length": 0.38 }, "Mk2": { "SR": 0.08, "L1": -0.46, "L2": -0.72, "SR2": -0.56, "cond_entropy": 14.13, "triple_var": -0.15, "num_var_10": 0.52, "run_length": -0.82 } } } exec /bin/bash -lc 'python tools/exp_meta_tautology_test.py --n-primes 300000 --n-trials 12' in /opt/MM_D-ND diff --git a/tools/exp_markov_layer_recovery_audit.py b/tools/exp_markov_layer_recovery_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..9797ea889603167a58cd989c71d02cf2aaf927bf --- /dev/null +++ b/tools/exp_markov_layer_recovery_audit.py @@ -0,0 +1,179 @@ +#!/usr/bin/env python3 +""" +exp_markov_layer_recovery_audit.py — META audit for the Markov-layer classifier. + +Question: + If a sequence is generated with known Markov order k, does the existing + Mk0/Mk1/Mk2 classifier recover layer <= k? + +Why this matters: + Recent reports used Mk surrogate capture to claim two visible memory layers. + This audit tests the opposite: whether the classifier itself creates apparent + higher layers on finite Markov controls. + +Controls: + - prime_gaps: unknown real target, reported for comparison only + - empirical_Mk0: shuffled prime gaps; known order 0 + - empirical_Mk1: prime-trained Mk1 surrogate; known order 1 + - empirical_Mk2: prime-trained Mk2 surrogate; known order 2 + - poisson_iid: independent exponential spacings; known order 0 + +Verdict rule: + A control fails recovery when any observable is assigned layer > known order. +""" + +import argparse +import json +from pathlib import Path + +import numpy as np + +from exp_two_layer_universality import ( + OBSERVABLES, + classify_layer, + gen_prime_gaps, + generate_markov_surrogate, +) + + +def measure_all(gaps): + out = {} + for name, fn in OBSERVABLES.items(): + try: + value = float(fn(gaps)) + except Exception: + value = float("nan") + out[name] = value + return out + + +def classifier_pass(gaps, n_surr, rng): + real_obs = measure_all(gaps) + z_scores = {} + + for mk in (0, 1, 2): + surr_obs = {name: [] for name in OBSERVABLES} + for _ in range(n_surr): + if mk == 0: + surr = rng.permutation(gaps) + else: + surr = generate_markov_surrogate(gaps, mk, rng=rng) + for obs_name, obs_fn in OBSERVABLES.items(): + try: + surr_obs[obs_name].append(float(obs_fn(surr))) + except Exception: + pass + + z_mk = {} + for obs_name in OBSERVABLES: + vals = np.asarray(surr_obs[obs_name], dtype=float) + vals = vals[np.isfinite(vals)] + if len(vals) > 2 and np.std(vals) > 1e-12 and np.isfinite(real_obs[obs_name]): + z = (real_obs[obs_name] - np.mean(vals)) / np.std(vals) + else: + z = 0.0 + z_mk[obs_name] = round(float(z), 2) + z_scores[f"Mk{mk}"] = z_mk + + layers = {} + for obs_name in OBSERVABLES: + layers[obs_name] = classify_layer( + z_scores["Mk0"][obs_name], + z_scores["Mk1"][obs_name], + z_scores["Mk2"][obs_name], + ) + + return real_obs, z_scores, layers + + +def build_controls(prime_gaps, rng): + return { + "prime_gaps": { + "known_order": None, + "gaps": prime_gaps, + }, + "empirical_Mk0": { + "known_order": 0, + "gaps": rng.permutation(prime_gaps), + }, + "empirical_Mk1": { + "known_order": 1, + "gaps": generate_markov_surrogate(prime_gaps, 1, rng=rng), + }, + "empirical_Mk2": { + "known_order": 2, + "gaps": generate_markov_surrogate(prime_gaps, 2, rng=rng), + }, + "poisson_iid": { + "known_order": 0, + "gaps": rng.exponential(1.0, len(prime_gaps)), + }, + } + + +def summarize_failure(layers, known_order): + if known_order is None: + return { + "status": "target_only", + "over_layer_observables": [], + "max_layer": int(max(layers.values())), + } + + over = [name for name, layer in layers.items() if layer > known_order] + return { + "status": "PASS" if not over else "FAIL", + "over_layer_observables": over, + "max_layer": int(max(layers.values())), + } + + +def run(N=60000, n_surr=20, seed=20260504): + rng = np.random.default_rng(seed) + prime_gaps = gen_prime_gaps(N).astype(float) + controls = build_controls(prime_gaps, rng) + + results = { + "N": int(N), + "n_surr": int(n_surr), + "seed": int(seed), + "controls": {}, + } + + print(f"N={N}, n_surr={n_surr}, seed={seed}") + print(f"{'sequence':<16} {'known':<7} {'status':<8} {'maxL':<5} over-layer observables") + print("-" * 82) + + for name, spec in controls.items(): + real_obs, z_scores, layers = classifier_pass(spec["gaps"], n_surr, rng) + summary = summarize_failure(layers, spec["known_order"]) + known = "target" if spec["known_order"] is None else f"L{spec['known_order']}" + over = ", ".join(summary["over_layer_observables"]) or "-" + print(f"{name:<16} {known:<7} {summary['status']:<8} L{summary['max_layer']:<4} {over}") + + results["controls"][name] = { + "known_order": spec["known_order"], + "n_gaps": int(len(spec["gaps"])), + "real_obs": {k: round(v, 6) for k, v in real_obs.items()}, + "z_scores": z_scores, + "layers": layers, + "recovery": summary, + } + + out_path = Path("tools/data/markov_layer_recovery_audit.json") + with out_path.open("w") as f: + json.dump(results, f, indent=2) + print(f"\nSaved to {out_path}") + return results + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--N", type=int, default=60000) + parser.add_argument("--n_surr", type=int, default=20) + parser.add_argument("--seed", type=int, default=20260504) + args = parser.parse_args() + run(N=args.N, n_surr=args.n_surr, seed=args.seed) + + +if __name__ == "__main__": + main() succeeded in 2039ms: Generating primes up to 300000... Using 25996 gaps starting at index 6499 (mean gap = 12.05) === REAL PRIMES === r-stat: 0.465417 mod3-frac: 0.158032 lag1-acf: -0.056325 dr-ord: -8.61σ dr-mag: -14.66σ === SHUFFLED (n_trials=12) === r_stat : +0.488408 ± 0.001229 mod3 : +0.341903 ± 0.003080 lag1 : +0.000870 ± 0.005508 dr_ord : -0.022482 ± 1.053665 dr_mag : -0.340020 ± 1.349587 === CRAMER (n_trials=12) === r_stat : +0.436936 ± 0.001631 mod3 : +0.350474 ± 0.002949 lag1 : +0.000421 ± 0.003362 dr_ord : +0.271635 ± 1.147115 dr_mag : +0.006410 ± 1.350016 === HL_MARKOV (n_trials=12) === r_stat : +0.477292 ± 0.002356 mod3 : +0.347195 ± 0.004070 lag1 : -0.051478 ± 0.007812 dr_ord : -4.992151 ± 1.904825 dr_mag : -0.387671 ± 1.131111 ============================================================ DISCRIMINATION VERDICTS ============================================================ r_stat vs shuffled : z = -18.71 [YES] r_stat vs cramer : z = +17.46 [YES] r_stat vs HL_markov : z = -5.04 [YES] → r_stat: STRUCTURAL mod3 vs shuffled : z = -59.69 [YES] mod3 vs cramer : z = -65.25 [YES] mod3 vs HL_markov : z = -46.48 [YES] → mod3: STRUCTURAL lag1 vs shuffled : z = -10.38 [YES] lag1 vs cramer : z = -16.88 [YES] lag1 vs HL_markov : z = -0.62 [no] → lag1: TAUTOLOGICAL dr_ord vs shuffled : z = -8.15 [YES] dr_ord vs cramer : z = -7.75 [YES] dr_ord vs HL_markov : z = -1.90 [no] → dr_ord: TAUTOLOGICAL dr_mag vs shuffled : z = -10.61 [YES] dr_mag vs cramer : z = -10.86 [YES] dr_mag vs HL_markov : z = -12.62 [YES] → dr_mag: STRUCTURAL Results saved to tools/data/meta_tautology_test.json diff --git a/tools/exp_markov_layer_recovery_audit.py b/tools/exp_markov_layer_recovery_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..9797ea889603167a58cd989c71d02cf2aaf927bf --- /dev/null +++ b/tools/exp_markov_layer_recovery_audit.py @@ -0,0 +1,179 @@ +#!/usr/bin/env python3 +""" +exp_markov_layer_recovery_audit.py — META audit for the Markov-layer classifier. + +Question: + If a sequence is generated with known Markov order k, does the existing + Mk0/Mk1/Mk2 classifier recover layer <= k? + +Why this matters: + Recent reports used Mk surrogate capture to claim two visible memory layers. + This audit tests the opposite: whether the classifier itself creates apparent + higher layers on finite Markov controls. + +Controls: + - prime_gaps: unknown real target, reported for comparison only + - empirical_Mk0: shuffled prime gaps; known order 0 + - empirical_Mk1: prime-trained Mk1 surrogate; known order 1 + - empirical_Mk2: prime-trained Mk2 surrogate; known order 2 + - poisson_iid: independent exponential spacings; known order 0 + +Verdict rule: + A control fails recovery when any observable is assigned layer > known order. +""" + +import argparse +import json +from pathlib import Path + +import numpy as np + +from exp_two_layer_universality import ( + OBSERVABLES, + classify_layer, + gen_prime_gaps, + generate_markov_surrogate, +) + + +def measure_all(gaps): + out = {} + for name, fn in OBSERVABLES.items(): + try: + value = float(fn(gaps)) + except Exception: + value = float("nan") + out[name] = value + return out + + +def classifier_pass(gaps, n_surr, rng): + real_obs = measure_all(gaps) + z_scores = {} + + for mk in (0, 1, 2): + surr_obs = {name: [] for name in OBSERVABLES} + for _ in range(n_surr): + if mk == 0: + surr = rng.permutation(gaps) + else: + surr = generate_markov_surrogate(gaps, mk, rng=rng) + for obs_name, obs_fn in OBSERVABLES.items(): + try: + surr_obs[obs_name].append(float(obs_fn(surr))) + except Exception: + pass + + z_mk = {} + for obs_name in OBSERVABLES: + vals = np.asarray(surr_obs[obs_name], dtype=float) + vals = vals[np.isfinite(vals)] + if len(vals) > 2 and np.std(vals) > 1e-12 and np.isfinite(real_obs[obs_name]): + z = (real_obs[obs_name] - np.mean(vals)) / np.std(vals) + else: + z = 0.0 + z_mk[obs_name] = round(float(z), 2) + z_scores[f"Mk{mk}"] = z_mk + + layers = {} + for obs_name in OBSERVABLES: + layers[obs_name] = classify_layer( + z_scores["Mk0"][obs_name], + z_scores["Mk1"][obs_name], + z_scores["Mk2"][obs_name], + ) + + return real_obs, z_scores, layers + + +def build_controls(prime_gaps, rng): + return { + "prime_gaps": { + "known_order": None, + "gaps": prime_gaps, + }, + "empirical_Mk0": { + "known_order": 0, + "gaps": rng.permutation(prime_gaps), + }, + "empirical_Mk1": { + "known_order": 1, + "gaps": generate_markov_surrogate(prime_gaps, 1, rng=rng), + }, + "empirical_Mk2": { + "known_order": 2, + "gaps": generate_markov_surrogate(prime_gaps, 2, rng=rng), + }, + "poisson_iid": { + "known_order": 0, + "gaps": rng.exponential(1.0, len(prime_gaps)), + }, + } + + +def summarize_failure(layers, known_order): + if known_order is None: + return { + "status": "target_only", + "over_layer_observables": [], + "max_layer": int(max(layers.values())), + } + + over = [name for name, layer in layers.items() if layer > known_order] + return { + "status": "PASS" if not over else "FAIL", + "over_layer_observables": over, + "max_layer": int(max(layers.values())), + } + + +def run(N=60000, n_surr=20, seed=20260504): + rng = np.random.default_rng(seed) + prime_gaps = gen_prime_gaps(N).astype(float) + controls = build_controls(prime_gaps, rng) + + results = { + "N": int(N), + "n_surr": int(n_surr), + "seed": int(seed), + "controls": {}, + } + + print(f"N={N}, n_surr={n_surr}, seed={seed}") + print(f"{'sequence':<16} {'known':<7} {'status':<8} {'maxL':<5} over-layer observables") + print("-" * 82) + + for name, spec in controls.items(): + real_obs, z_scores, layers = classifier_pass(spec["gaps"], n_surr, rng) + summary = summarize_failure(layers, spec["known_order"]) + known = "target" if spec["known_order"] is None else f"L{spec['known_order']}" + over = ", ".join(summary["over_layer_observables"]) or "-" + print(f"{name:<16} {known:<7} {summary['status']:<8} L{summary['max_layer']:<4} {over}") + + results["controls"][name] = { + "known_order": spec["known_order"], + "n_gaps": int(len(spec["gaps"])), + "real_obs": {k: round(v, 6) for k, v in real_obs.items()}, + "z_scores": z_scores, + "layers": layers, + "recovery": summary, + } + + out_path = Path("tools/data/markov_layer_recovery_audit.json") + with out_path.open("w") as f: + json.dump(results, f, indent=2) + print(f"\nSaved to {out_path}") + return results + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--N", type=int, default=60000) + parser.add_argument("--n_surr", type=int, default=20) + parser.add_argument("--seed", type=int, default=20260504) + args = parser.parse_args() + run(N=args.N, n_surr=args.n_surr, seed=args.seed) + + +if __name__ == "__main__": + main() exec /bin/bash -lc 'tail -80 tools/data/seme.json' in /opt/MM_D-ND succeeded in 0ms: "fluctuations", "information theory", "probability", "Boltzmann" ], "content": "Computing termodinamico: sfruttare le fluttuazioni termiche invece di combatterle. Il rumore non è nemico — è risorsa. Extropic costruisce hardware che usa l'entropia come motore computazionale. Connessione profonda tra termodinamica, teoria dell'informazione e probabilità.", "teorie": [ "T", "Q" ], "ponti_potenziali": [ { "coppia": "TxQ", "ponte": "noise come risorsa computazionale — fluttuazioni termiche = calcolo", "nota": "il vuoto quantistico (pieno di fluttuazioni) è il computer" } ], "timestamp": "2026-04-02T08:23:13.992019" }, { "id": "video_j0wJBEZdwLs", "title": "What is a Laplace Transform - visual explanation", "type": "video_digest", "keywords": [ "Laplace transform", "frequency", "damping", "s-plane", "complex", "exponential" ], "content": "La trasformata di Laplace come proiezione su esponenziali complesse. Il piano s = σ + iω combina smorzamento (reale) e oscillazione (immaginario). Connessione tra dominio temporale e dominio delle frequenze complesse.", "teorie": [ "T", "Q", "R" ], "ponti_potenziali": [], "timestamp": "2026-04-02T08:23:13.992021" }, { "id": "video_rZ2m1_q9lg0", "title": "New duality: conductor-insulator in YbB12 at 35T - University of Michigan", "type": "video_digest", "keywords": [ "duality", "conductor", "insulator", "Kondo insulator", "quantum oscillations", "ytterbium boride", "YbB12", "charge-neutral fermions", "strongly correlated", "condensed matter", "magnetic field" ], "content": "Scoperta dualita conduttore-isolante in YbB12 a 35 Tesla. Il materiale si comporta simultaneamente da conduttore e isolante — oscillazioni quantistiche dal bulk, non dalla superficie. Charge-neutral fermions (portatori senza carica) creano comportamento di conduzione = il terzo incluso. La scoperta avviene a campo estremo dove il confine tra stati collassa.", "teorie": [ "T", "Q", "E" ], "ponti_potenziali": [ { "coppia": "TxQ", "ponte": "Kondo insulator fortemente correlato — dualita conduttore/isolante come stato quantistico unico", "nota": "charge-neutral fermions = portatori senza carica che conducono, terzo incluso operativo" }, { "coppia": "ExQ", "ponte": "campo magnetico estremo (35T) collassa il confine tra stati — condizione di soglia", "nota": "il confine non separa, unifica. Pattern BOUNDARY del Domandatore" } ], "ref": "Physical Review Letters 2025, 135(15). Chen, Zhu, Ratkovski et al. PI: Lu Li", "timestamp": "2026-04-04T19:34:08.014Z" } ] } diff --git a/tools/exp_markov_layer_recovery_audit.py b/tools/exp_markov_layer_recovery_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..9797ea889603167a58cd989c71d02cf2aaf927bf --- /dev/null +++ b/tools/exp_markov_layer_recovery_audit.py @@ -0,0 +1,179 @@ +#!/usr/bin/env python3 +""" +exp_markov_layer_recovery_audit.py — META audit for the Markov-layer classifier. + +Question: + If a sequence is generated with known Markov order k, does the existing + Mk0/Mk1/Mk2 classifier recover layer <= k? + +Why this matters: + Recent reports used Mk surrogate capture to claim two visible memory layers. + This audit tests the opposite: whether the classifier itself creates apparent + higher layers on finite Markov controls. + +Controls: + - prime_gaps: unknown real target, reported for comparison only + - empirical_Mk0: shuffled prime gaps; known order 0 + - empirical_Mk1: prime-trained Mk1 surrogate; known order 1 + - empirical_Mk2: prime-trained Mk2 surrogate; known order 2 + - poisson_iid: independent exponential spacings; known order 0 + +Verdict rule: + A control fails recovery when any observable is assigned layer > known order. +""" + +import argparse +import json +from pathlib import Path + +import numpy as np + +from exp_two_layer_universality import ( + OBSERVABLES, + classify_layer, + gen_prime_gaps, + generate_markov_surrogate, +) + + +def measure_all(gaps): + out = {} + for name, fn in OBSERVABLES.items(): + try: + value = float(fn(gaps)) + except Exception: + value = float("nan") + out[name] = value + return out + + +def classifier_pass(gaps, n_surr, rng): + real_obs = measure_all(gaps) + z_scores = {} + + for mk in (0, 1, 2): + surr_obs = {name: [] for name in OBSERVABLES} + for _ in range(n_surr): + if mk == 0: + surr = rng.permutation(gaps) + else: + surr = generate_markov_surrogate(gaps, mk, rng=rng) + for obs_name, obs_fn in OBSERVABLES.items(): + try: + surr_obs[obs_name].append(float(obs_fn(surr))) + except Exception: + pass + + z_mk = {} + for obs_name in OBSERVABLES: + vals = np.asarray(surr_obs[obs_name], dtype=float) + vals = vals[np.isfinite(vals)] + if len(vals) > 2 and np.std(vals) > 1e-12 and np.isfinite(real_obs[obs_name]): + z = (real_obs[obs_name] - np.mean(vals)) / np.std(vals) + else: + z = 0.0 + z_mk[obs_name] = round(float(z), 2) + z_scores[f"Mk{mk}"] = z_mk + + layers = {} + for obs_name in OBSERVABLES: + layers[obs_name] = classify_layer( + z_scores["Mk0"][obs_name], + z_scores["Mk1"][obs_name], + z_scores["Mk2"][obs_name], + ) + + return real_obs, z_scores, layers + + +def build_controls(prime_gaps, rng): + return { + "prime_gaps": { + "known_order": None, + "gaps": prime_gaps, + }, + "empirical_Mk0": { + "known_order": 0, + "gaps": rng.permutation(prime_gaps), + }, + "empirical_Mk1": { + "known_order": 1, + "gaps": generate_markov_surrogate(prime_gaps, 1, rng=rng), + }, + "empirical_Mk2": { + "known_order": 2, + "gaps": generate_markov_surrogate(prime_gaps, 2, rng=rng), + }, + "poisson_iid": { + "known_order": 0, + "gaps": rng.exponential(1.0, len(prime_gaps)), + }, + } + + +def summarize_failure(layers, known_order): + if known_order is None: + return { + "status": "target_only", + "over_layer_observables": [], + "max_layer": int(max(layers.values())), + } + + over = [name for name, layer in layers.items() if layer > known_order] + return { + "status": "PASS" if not over else "FAIL", + "over_layer_observables": over, + "max_layer": int(max(layers.values())), + } + + +def run(N=60000, n_surr=20, seed=20260504): + rng = np.random.default_rng(seed) + prime_gaps = gen_prime_gaps(N).astype(float) + controls = build_controls(prime_gaps, rng) + + results = { + "N": int(N), + "n_surr": int(n_surr), + "seed": int(seed), + "controls": {}, + } + + print(f"N={N}, n_surr={n_surr}, seed={seed}") + print(f"{'sequence':<16} {'known':<7} {'status':<8} {'maxL':<5} over-layer observables") + print("-" * 82) + + for name, spec in controls.items(): + real_obs, z_scores, layers = classifier_pass(spec["gaps"], n_surr, rng) + summary = summarize_failure(layers, spec["known_order"]) + known = "target" if spec["known_order"] is None else f"L{spec['known_order']}" + over = ", ".join(summary["over_layer_observables"]) or "-" + print(f"{name:<16} {known:<7} {summary['status']:<8} L{summary['max_layer']:<4} {over}") + + results["controls"][name] = { + "known_order": spec["known_order"], + "n_gaps": int(len(spec["gaps"])), + "real_obs": {k: round(v, 6) for k, v in real_obs.items()}, + "z_scores": z_scores, + "layers": layers, + "recovery": summary, + } + + out_path = Path("tools/data/markov_layer_recovery_audit.json") + with out_path.open("w") as f: + json.dump(results, f, indent=2) + print(f"\nSaved to {out_path}") + return results + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--N", type=int, default=60000) + parser.add_argument("--n_surr", type=int, default=20) + parser.add_argument("--seed", type=int, default=20260504) + args = parser.parse_args() + run(N=args.N, n_surr=args.n_surr, seed=args.seed) + + +if __name__ == "__main__": + main() codex I dati danno una risultante abbastanza netta: il nucleo pair/triple regge sui controlli Mk0/Mk1, ma non tutti gli osservabili sono utilizzabili. Aggiorno il seme come vincolo META, poi scrivo il report con il perimetro atomico del claim. exec /bin/bash -lc "python - <<'PY' import json from datetime import datetime from pathlib import Path path = Path('tools/data/seme.json') data = json.loads(path.read_text()) data['timestamp'] = '2026-05-04T12:19:00' new = { 'tipo': 'vincolo_meta', 'id': 'META_LAYER_RECOVERY_AUDIT', 'claim': 'La pipeline Mk0/Mk1/Mk2 recupera correttamente i controlli empirici Mk0 e Mk1, ma sovrastima alcuni osservabili: cond_entropy su controllo Mk2 viene letto Layer 3 e num_var_10 su Poisson iid viene letto Layer 1. I claim two-layer devono quindi usare SR,L1,L2,SR2 come nucleo e dichiarare cond_entropy/num_var_10/run_length come diagnostiche secondarie finche non passano audit di recupero multi-seed.', 'intensita': 0.7, 'nota': 'Report agent_20260504_1219: recovery audit con N=60000, n_surr=20, seed=20260504. Risultato: empirical_Mk0 PASS maxL0; empirical_Mk1 PASS maxL1; empirical_Mk2 FAIL cond_entropy Layer3; poisson_iid FAIL num_var_10 Layer1.', 'porta': 'meta_audit', 'condensato_ref': 'A4,A8,A14', 'condensato_motivo': 'No silent patching: separa nucleo robusto e diagnostiche sovrastimate. Evita rediscovery/tautologia dei controlli.' } ids = {t.get('id') for t in data.get('tensioni', [])} if new['id'] not in ids: data.setdefault('tensioni', []).append(new) else: for i,t in enumerate(data['tensioni']): if t.get('id') == new['id']: data['tensioni'][i] = new path.write_text(json.dumps(data, indent=2, ensure_ascii=False) + '\\n') PY" in /opt/MM_D-ND succeeded in 0ms: diff --git a/tools/exp_markov_layer_recovery_audit.py b/tools/exp_markov_layer_recovery_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..9797ea889603167a58cd989c71d02cf2aaf927bf --- /dev/null +++ b/tools/exp_markov_layer_recovery_audit.py @@ -0,0 +1,179 @@ +#!/usr/bin/env python3 +""" +exp_markov_layer_recovery_audit.py — META audit for the Markov-layer classifier. + +Question: + If a sequence is generated with known Markov order k, does the existing + Mk0/Mk1/Mk2 classifier recover layer <= k? + +Why this matters: + Recent reports used Mk surrogate capture to claim two visible memory layers. + This audit tests the opposite: whether the classifier itself creates apparent + higher layers on finite Markov controls. + +Controls: + - prime_gaps: unknown real target, reported for comparison only + - empirical_Mk0: shuffled prime gaps; known order 0 + - empirical_Mk1: prime-trained Mk1 surrogate; known order 1 + - empirical_Mk2: prime-trained Mk2 surrogate; known order 2 + - poisson_iid: independent exponential spacings; known order 0 + +Verdict rule: + A control fails recovery when any observable is assigned layer > known order. +""" + +import argparse +import json +from pathlib import Path + +import numpy as np + +from exp_two_layer_universality import ( + OBSERVABLES, + classify_layer, + gen_prime_gaps, + generate_markov_surrogate, +) + + +def measure_all(gaps): + out = {} + for name, fn in OBSERVABLES.items(): + try: + value = float(fn(gaps)) + except Exception: + value = float("nan") + out[name] = value + return out + + +def classifier_pass(gaps, n_surr, rng): + real_obs = measure_all(gaps) + z_scores = {} + + for mk in (0, 1, 2): + surr_obs = {name: [] for name in OBSERVABLES} + for _ in range(n_surr): + if mk == 0: + surr = rng.permutation(gaps) + else: + surr = generate_markov_surrogate(gaps, mk, rng=rng) + for obs_name, obs_fn in OBSERVABLES.items(): + try: + surr_obs[obs_name].append(float(obs_fn(surr))) + except Exception: + pass + + z_mk = {} + for obs_name in OBSERVABLES: + vals = np.asarray(surr_obs[obs_name], dtype=float) + vals = vals[np.isfinite(vals)] + if len(vals) > 2 and np.std(vals) > 1e-12 and np.isfinite(real_obs[obs_name]): + z = (real_obs[obs_name] - np.mean(vals)) / np.std(vals) + else: + z = 0.0 + z_mk[obs_name] = round(float(z), 2) + z_scores[f"Mk{mk}"] = z_mk + + layers = {} + for obs_name in OBSERVABLES: + layers[obs_name] = classify_layer( + z_scores["Mk0"][obs_name], + z_scores["Mk1"][obs_name], + z_scores["Mk2"][obs_name], + ) + + return real_obs, z_scores, layers + + +def build_controls(prime_gaps, rng): + return { + "prime_gaps": { + "known_order": None, + "gaps": prime_gaps, + }, + "empirical_Mk0": { + "known_order": 0, + "gaps": rng.permutation(prime_gaps), + }, + "empirical_Mk1": { + "known_order": 1, + "gaps": generate_markov_surrogate(prime_gaps, 1, rng=rng), + }, + "empirical_Mk2": { + "known_order": 2, + "gaps": generate_markov_surrogate(prime_gaps, 2, rng=rng), + }, + "poisson_iid": { + "known_order": 0, + "gaps": rng.exponential(1.0, len(prime_gaps)), + }, + } + + +def summarize_failure(layers, known_order): + if known_order is None: + return { + "status": "target_only", + "over_layer_observables": [], + "max_layer": int(max(layers.values())), + } + + over = [name for name, layer in layers.items() if layer > known_order] + return { + "status": "PASS" if not over else "FAIL", + "over_layer_observables": over, + "max_layer": int(max(layers.values())), + } + + +def run(N=60000, n_surr=20, seed=20260504): + rng = np.random.default_rng(seed) + prime_gaps = gen_prime_gaps(N).astype(float) + controls = build_controls(prime_gaps, rng) + + results = { + "N": int(N), + "n_surr": int(n_surr), + "seed": int(seed), + "controls": {}, + } + + print(f"N={N}, n_surr={n_surr}, seed={seed}") + print(f"{'sequence':<16} {'known':<7} {'status':<8} {'maxL':<5} over-layer observables") + print("-" * 82) + + for name, spec in controls.items(): + real_obs, z_scores, layers = classifier_pass(spec["gaps"], n_surr, rng) + summary = summarize_failure(layers, spec["known_order"]) + known = "target" if spec["known_order"] is None else f"L{spec['known_order']}" + over = ", ".join(summary["over_layer_observables"]) or "-" + print(f"{name:<16} {known:<7} {summary['status']:<8} L{summary['max_layer']:<4} {over}") + + results["controls"][name] = { + "known_order": spec["known_order"], + "n_gaps": int(len(spec["gaps"])), + "real_obs": {k: round(v, 6) for k, v in real_obs.items()}, + "z_scores": z_scores, + "layers": layers, + "recovery": summary, + } + + out_path = Path("tools/data/markov_layer_recovery_audit.json") + with out_path.open("w") as f: + json.dump(results, f, indent=2) + print(f"\nSaved to {out_path}") + return results + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--N", type=int, default=60000) + parser.add_argument("--n_surr", type=int, default=20) + parser.add_argument("--seed", type=int, default=20260504) + args = parser.parse_args() + run(N=args.N, n_surr=args.n_surr, seed=args.seed) + + +if __name__ == "__main__": + main() web search: web search: Markov order estimation finite sample bias surrogate data test Markov chain observables paper apply patch patch: completed /opt/MM_D-ND/tools/data/reports/agent_20260504_1219.md diff --git a/tools/data/reports/agent_20260504_1219.md b/tools/data/reports/agent_20260504_1219.md new file mode 100644 index 0000000000000000000000000000000000000000..683dab4bbab940600d05e182ee1b696459ee180c --- /dev/null +++ b/tools/data/reports/agent_20260504_1219.md @@ -0,0 +1,104 @@ +# Agent Report — Markov Layers Pass the First Recovery Gate, but Not All Observables Pass +**Date**: 2026-05-04 12:19 +**Piano**: 61 +**Tension explored**: META (0.5) + +## Claim Under Test +> Tutti i 11 test passano — verifica che non stiamo testando solo tautologie. + +## Question +Se la pipeline Mk0/Mk1/Mk2 vede due layer, li recupera anche quando l'ordine vero della sequenza è noto? + +## Experiment Design +- **Primary audit**: `tools/exp_markov_layer_recovery_audit.py` +- **Scope**: 60,000 gap/spacings per sequenza; 20 surrogati per livello Mk; seed `20260504`. +- **Target**: prime gaps, ordine sconosciuto, solo confronto. +- **Known-order controls**: + - `empirical_Mk0`: permutazione dei gap primi, ordine noto 0. + - `empirical_Mk1`: surrogate Mk1 addestrato sui gap primi, ordine noto 1. + - `empirical_Mk2`: surrogate Mk2 addestrato sui gap primi, ordine noto 2. + - `poisson_iid`: spacings esponenziali iid, ordine noto 0. +- **Recovery rule**: un controllo passa se nessun osservabile viene classificato con layer maggiore dell'ordine noto. +- **Secondary calibration**: `tools/exp_meta_tautology_test.py --n-primes 300000 --n-trials 12`. Nota: il controllo chiamato `HL_markov` nello script è un AR(1) gap model che preserva lag-1, non un modello Hardy-Littlewood aritmetico. + +## Results + +### Recovery audit + +| Sequence | Known order | Recovery | Max layer | Over-layer observables | +|---|---:|---|---:|---| +| `prime_gaps` | target | target only | 3 | - | +| `empirical_Mk0` | 0 | PASS | 0 | - | +| `empirical_Mk1` | 1 | PASS | 1 | - | +| `empirical_Mk2` | 2 | FAIL | 3 | `cond_entropy` | +| `poisson_iid` | 0 | FAIL | 1 | `num_var_10` | + +Prime target layers, reported without recovery verdict: + +| Observable | Layer | z vs Mk0 | z vs Mk1 | z vs Mk2 | +|---|---:|---:|---:|---:| +| SR | 1 | -22.21 | -0.08 | -0.16 | +| L1 | 1 | -12.25 | 0.79 | 0.86 | +| L2 | 2 | -3.60 | -4.09 | -0.19 | +| SR2 | 2 | -6.67 | -8.80 | -0.33 | +| cond_entropy | 3 | -435.40 | -26.11 | 2.56 | +| triple_var | 1 | -15.90 | -0.53 | 0.36 | +| num_var_10 | 1 | -7.46 | -1.42 | 0.00 | +| run_length | 3 | 27.59 | 7.19 | 3.43 | + +Control failure details: + +| Control | Observable | Expected max | Assigned | Diagnostic z | +|---|---|---:|---:|---:| +| `empirical_Mk2` | `cond_entropy` | 2 | 3 | z vs Mk2 = 3.15 | +| `poisson_iid` | `num_var_10` | 0 | 1 | z vs Mk0 = 2.13 | + +### Secondary tautology calibration + +| Observable | Verdict in tested controls | Blocking control | +|---|---|---| +| r-stat | STRUCTURAL | none in this run | +| mod3 fraction | STRUCTURAL | none in this run | +| lag1 ACF | TAUTOLOGICAL | lag1-matched AR(1), z = -0.62 | +| dr_ord | TAUTOLOGICAL | lag1-matched AR(1), z = -1.90 | +| dr_mag | STRUCTURAL | none in this run | + +Numeri principali: prime mod3 same-class fraction = 0.158032. Questo è bias forte rispetto ai controlli ~0.34-0.35, non proibizione zero. + +## Key Findings +1. **Il nucleo pair/triple non cade al primo audit.** `empirical_Mk0` recupera solo Layer 0; `empirical_Mk1` recupera solo Layer 1. Questo supporta l'uso di SR e L1 come Layer 1 e di L2/SR2 come Layer 2 nel perimetro testato. +2. **`cond_entropy` non può essere usata come prova forte di Layer 3 nel setup attuale.** Anche un controllo generato come Mk2 viene letto Layer 3 per `cond_entropy` (z vs Mk2 = 3.15). Questo indica bias del surrogate/binning o varianza finita, non necessariamente memoria reale oltre Mk2. +3. **`num_var_10` produce un falso positivo debole su iid.** `poisson_iid` viene letto Layer 1 per `num_var_10` con z = 2.13. Il segnale supera appena la soglia operativa, quindi va trattato come diagnostica secondaria finché non passa audit multi-seed. +4. **Il vecchio controllo `HL_markov` va rinominato nel linguaggio dei report.** Lo script non implementa Hardy-Littlewood; implementa un AR(1) lag1-matched. I claim futuri non devono chiamarlo controllo HL aritmetico. + +## Verdict +**CONSTRAINT on META + BOUNDARY**: la decomposizione two-layer è utilizzabile solo con perimetro atomico. + +Perimetro corretto: nei 60,000 gap primi testati con 20 surrogati Mk per livello, il nucleo `SR,L1` viene catturato da Mk1 e il nucleo `L2,SR2` da Mk2. Le diagnostiche `cond_entropy`, `num_var_10`, `run_length` non sono evidenza primaria finché non passano un recovery audit multi-seed e, idealmente, surrogati Markov esatti. + +Non è una scoperta nuova di teoria Markov: è un audit metodologico vicino ai test di ordine Markov con surrogate data. La letteratura rilevante include test esatti di ordine Markov e surrogate vincolati che preservano proprietà Markov di ordine n, quindi il prossimo strumento deve avvicinarsi a quei null esatti invece di usare solo generatori bin-pool approssimati. + +## Bicono della scoperta +- **Due radici**: layer osservato dai dati · ordine noto del generatore. +- **Singolare**: la pipeline Mk stessa, prima che venga usata come prova su un dominio reale. +- **Invariante di passaggio**: un claim di memoria sopravvive solo se recupera correttamente i controlli con ordine noto. +- **Campo di possibilità**: qui diventa possibile separare nucleo robusto e diagnostiche sovrastimate; qui diventa non-possibile usare `cond_entropy` o `num_var_10` come prova primaria di profondità senza recovery audit. + +## Self-Falsification — 5 lenti +- **L1 hard constraint vs bias**: nessun claim "zero/never/always". La frazione mod3 è 0.158032, dichiarata bias, non proibizione. +- **L2 quantità vs ratio**: il claim principale usa layer recuperato e z-score, non percentuali fra spazi di taglia diversa. +- **L3 no silent patching**: dichiaro esplicitamente che `HL_markov` era nome scorretto per un controllo AR(1) lag1-matched. +- **L4 edge cases**: i fallimenti `cond_entropy` e `num_var_10` non vengono arrotondati via; diventano il vincolo. +- **L5 rediscovery**: non taggo NEW. Riferimenti classici vicini: surrogate-data tests for Markov order e constrained Markov-order surrogates. + +## References +- Kandel et al.-lineage / surrogate tests: "Testing the order of discrete Markov chains using surrogate data", Physica D 117, 1998. +- "Exact significance test for Markov order", Physica D 269, 2014. +- "Constrained Markov order surrogates", Physica D 2020. + +## Files +- `tools/exp_markov_layer_recovery_audit.py` +- `tools/data/markov_layer_recovery_audit.json` +- `tools/data/meta_tautology_test.json` +- `tools/data/seme.json` +- `tools/data/reports/agent_20260504_1219.md` diff --git a/tools/exp_markov_layer_recovery_audit.py b/tools/exp_markov_layer_recovery_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..9797ea889603167a58cd989c71d02cf2aaf927bf --- /dev/null +++ b/tools/exp_markov_layer_recovery_audit.py @@ -0,0 +1,179 @@ +#!/usr/bin/env python3 +""" +exp_markov_layer_recovery_audit.py — META audit for the Markov-layer classifier. + +Question: + If a sequence is generated with known Markov order k, does the existing + Mk0/Mk1/Mk2 classifier recover layer <= k? + +Why this matters: + Recent reports used Mk surrogate capture to claim two visible memory layers. + This audit tests the opposite: whether the classifier itself creates apparent + higher layers on finite Markov controls. + +Controls: + - prime_gaps: unknown real target, reported for comparison only + - empirical_Mk0: shuffled prime gaps; known order 0 + - empirical_Mk1: prime-trained Mk1 surrogate; known order 1 + - empirical_Mk2: prime-trained Mk2 surrogate; known order 2 + - poisson_iid: independent exponential spacings; known order 0 + +Verdict rule: + A control fails recovery when any observable is assigned layer > known order. +""" + +import argparse +import json +from pathlib import Path + +import numpy as np + +from exp_two_layer_universality import ( + OBSERVABLES, + classify_layer, + gen_prime_gaps, + generate_markov_surrogate, +) + + +def measure_all(gaps): + out = {} + for name, fn in OBSERVABLES.items(): + try: + value = float(fn(gaps)) + except Exception: + value = float("nan") + out[name] = value + return out + + +def classifier_pass(gaps, n_surr, rng): + real_obs = measure_all(gaps) + z_scores = {} + + for mk in (0, 1, 2): + surr_obs = {name: [] for name in OBSERVABLES} + for _ in range(n_surr): + if mk == 0: + surr = rng.permutation(gaps) + else: + surr = generate_markov_surrogate(gaps, mk, rng=rng) + for obs_name, obs_fn in OBSERVABLES.items(): + try: + surr_obs[obs_name].append(float(obs_fn(surr))) + except Exception: + pass + + z_mk = {} + for obs_name in OBSERVABLES: + vals = np.asarray(surr_obs[obs_name], dtype=float) + vals = vals[np.isfinite(vals)] + if len(vals) > 2 and np.std(vals) > 1e-12 and np.isfinite(real_obs[obs_name]): + z = (real_obs[obs_name] - np.mean(vals)) / np.std(vals) + else: + z = 0.0 + z_mk[obs_name] = round(float(z), 2) + z_scores[f"Mk{mk}"] = z_mk + + layers = {} + for obs_name in OBSERVABLES: + layers[obs_name] = classify_layer( + z_scores["Mk0"][obs_name], + z_scores["Mk1"][obs_name], + z_scores["Mk2"][obs_name], + ) + + return real_obs, z_scores, layers + + +def build_controls(prime_gaps, rng): + return { + "prime_gaps": { + "known_order": None, + "gaps": prime_gaps, + }, + "empirical_Mk0": { + "known_order": 0, + "gaps": rng.permutation(prime_gaps), + }, + "empirical_Mk1": { + "known_order": 1, + "gaps": generate_markov_surrogate(prime_gaps, 1, rng=rng), + }, + "empirical_Mk2": { + "known_order": 2, + "gaps": generate_markov_surrogate(prime_gaps, 2, rng=rng), + }, + "poisson_iid": { + "known_order": 0, + "gaps": rng.exponential(1.0, len(prime_gaps)), + }, + } + + +def summarize_failure(layers, known_order): + if known_order is None: + return { + "status": "target_only", + "over_layer_observables": [], + "max_layer": int(max(layers.values())), + } + + over = [name for name, layer in layers.items() if layer > known_order] + return { + "status": "PASS" if not over else "FAIL", + "over_layer_observables": over, + "max_layer": int(max(layers.values())), + } + + +def run(N=60000, n_surr=20, seed=20260504): + rng = np.random.default_rng(seed) + prime_gaps = gen_prime_gaps(N).astype(float) + controls = build_controls(prime_gaps, rng) + + results = { + "N": int(N), + "n_surr": int(n_surr), + "seed": int(seed), + "controls": {}, + } + + print(f"N={N}, n_surr={n_surr}, seed={seed}") + print(f"{'sequence':<16} {'known':<7} {'status':<8} {'maxL':<5} over-layer observables") + print("-" * 82) + + for name, spec in controls.items(): + real_obs, z_scores, layers = classifier_pass(spec["gaps"], n_surr, rng) + summary = summarize_failure(layers, spec["known_order"]) + known = "target" if spec["known_order"] is None else f"L{spec['known_order']}" + over = ", ".join(summary["over_layer_observables"]) or "-" + print(f"{name:<16} {known:<7} {summary['status']:<8} L{summary['max_layer']:<4} {over}") + + results["controls"][name] = { + "known_order": spec["known_order"], + "n_gaps": int(len(spec["gaps"])), + "real_obs": {k: round(v, 6) for k, v in real_obs.items()}, + "z_scores": z_scores, + "layers": layers, + "recovery": summary, + } + + out_path = Path("tools/data/markov_layer_recovery_audit.json") + with out_path.open("w") as f: + json.dump(results, f, indent=2) + print(f"\nSaved to {out_path}") + return results + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--N", type=int, default=60000) + parser.add_argument("--n_surr", type=int, default=20) + parser.add_argument("--seed", type=int, default=20260504) + args = parser.parse_args() + run(N=args.N, n_surr=args.n_surr, seed=args.seed) + + +if __name__ == "__main__": + main() diff --git a/tools/data/reports/agent_20260504_1219.md b/tools/data/reports/agent_20260504_1219.md new file mode 100644 index 0000000000000000000000000000000000000000..683dab4bbab940600d05e182ee1b696459ee180c --- /dev/null +++ b/tools/data/reports/agent_20260504_1219.md @@ -0,0 +1,104 @@ +# Agent Report — Markov Layers Pass the First Recovery Gate, but Not All Observables Pass +**Date**: 2026-05-04 12:19 +**Piano**: 61 +**Tension explored**: META (0.5) + +## Claim Under Test +> Tutti i 11 test passano — verifica che non stiamo testando solo tautologie. + +## Question +Se la pipeline Mk0/Mk1/Mk2 vede due layer, li recupera anche quando l'ordine vero della sequenza è noto? + +## Experiment Design +- **Primary audit**: `tools/exp_markov_layer_recovery_audit.py` +- **Scope**: 60,000 gap/spacings per sequenza; 20 surrogati per livello Mk; seed `20260504`. +- **Target**: prime gaps, ordine sconosciuto, solo confronto. +- **Known-order controls**: + - `empirical_Mk0`: permutazione dei gap primi, ordine noto 0. + - `empirical_Mk1`: surrogate Mk1 addestrato sui gap primi, ordine noto 1. + - `empirical_Mk2`: surrogate Mk2 addestrato sui gap primi, ordine noto 2. + - `poisson_iid`: spacings esponenziali iid, ordine noto 0. +- **Recovery rule**: un controllo passa se nessun osservabile viene classificato con layer maggiore dell'ordine noto. +- **Secondary calibration**: `tools/exp_meta_tautology_test.py --n-primes 300000 --n-trials 12`. Nota: il controllo chiamato `HL_markov` nello script è un AR(1) gap model che preserva lag-1, non un modello Hardy-Littlewood aritmetico. + +## Results + +### Recovery audit + +| Sequence | Known order | Recovery | Max layer | Over-layer observables | +|---|---:|---|---:|---| +| `prime_gaps` | target | target only | 3 | - | +| `empirical_Mk0` | 0 | PASS | 0 | - | +| `empirical_Mk1` | 1 | PASS | 1 | - | +| `empirical_Mk2` | 2 | FAIL | 3 | `cond_entropy` | +| `poisson_iid` | 0 | FAIL | 1 | `num_var_10` | + +Prime target layers, reported without recovery verdict: + +| Observable | Layer | z vs Mk0 | z vs Mk1 | z vs Mk2 | +|---|---:|---:|---:|---:| +| SR | 1 | -22.21 | -0.08 | -0.16 | +| L1 | 1 | -12.25 | 0.79 | 0.86 | +| L2 | 2 | -3.60 | -4.09 | -0.19 | +| SR2 | 2 | -6.67 | -8.80 | -0.33 | +| cond_entropy | 3 | -435.40 | -26.11 | 2.56 | +| triple_var | 1 | -15.90 | -0.53 | 0.36 | +| num_var_10 | 1 | -7.46 | -1.42 | 0.00 | +| run_length | 3 | 27.59 | 7.19 | 3.43 | + +Control failure details: + +| Control | Observable | Expected max | Assigned | Diagnostic z | +|---|---|---:|---:|---:| +| `empirical_Mk2` | `cond_entropy` | 2 | 3 | z vs Mk2 = 3.15 | +| `poisson_iid` | `num_var_10` | 0 | 1 | z vs Mk0 = 2.13 | + +### Secondary tautology calibration + +| Observable | Verdict in tested controls | Blocking control | +|---|---|---| +| r-stat | STRUCTURAL | none in this run | +| mod3 fraction | STRUCTURAL | none in this run | +| lag1 ACF | TAUTOLOGICAL | lag1-matched AR(1), z = -0.62 | +| dr_ord | TAUTOLOGICAL | lag1-matched AR(1), z = -1.90 | +| dr_mag | STRUCTURAL | none in this run | + +Numeri principali: prime mod3 same-class fraction = 0.158032. Questo è bias forte rispetto ai controlli ~0.34-0.35, non proibizione zero. + +## Key Findings +1. **Il nucleo pair/triple non cade al primo audit.** `empirical_Mk0` recupera solo Layer 0; `empirical_Mk1` recupera solo Layer 1. Questo supporta l'uso di SR e L1 come Layer 1 e di L2/SR2 come Layer 2 nel perimetro testato. +2. **`cond_entropy` non può essere usata come prova forte di Layer 3 nel setup attuale.** Anche un controllo generato come Mk2 viene letto Layer 3 per `cond_entropy` (z vs Mk2 = 3.15). Questo indica bias del surrogate/binning o varianza finita, non necessariamente memoria reale oltre Mk2. +3. **`num_var_10` produce un falso positivo debole su iid.** `poisson_iid` viene letto Layer 1 per `num_var_10` con z = 2.13. Il segnale supera appena la soglia operativa, quindi va trattato come diagnostica secondaria finché non passa audit multi-seed. +4. **Il vecchio controllo `HL_markov` va rinominato nel linguaggio dei report.** Lo script non implementa Hardy-Littlewood; implementa un AR(1) lag1-matched. I claim futuri non devono chiamarlo controllo HL aritmetico. + +## Verdict +**CONSTRAINT on META + BOUNDARY**: la decomposizione two-layer è utilizzabile solo con perimetro atomico. + +Perimetro corretto: nei 60,000 gap primi testati con 20 surrogati Mk per livello, il nucleo `SR,L1` viene catturato da Mk1 e il nucleo `L2,SR2` da Mk2. Le diagnostiche `cond_entropy`, `num_var_10`, `run_length` non sono evidenza primaria finché non passano un recovery audit multi-seed e, idealmente, surrogati Markov esatti. + +Non è una scoperta nuova di teoria Markov: è un audit metodologico vicino ai test di ordine Markov con surrogate data. La letteratura rilevante include test esatti di ordine Markov e surrogate vincolati che preservano proprietà Markov di ordine n, quindi il prossimo strumento deve avvicinarsi a quei null esatti invece di usare solo generatori bin-pool approssimati. + +## Bicono della scoperta +- **Due radici**: layer osservato dai dati · ordine noto del generatore. +- **Singolare**: la pipeline Mk stessa, prima che venga usata come prova su un dominio reale. +- **Invariante di passaggio**: un claim di memoria sopravvive solo se recupera correttamente i controlli con ordine noto. +- **Campo di possibilità**: qui diventa possibile separare nucleo robusto e diagnostiche sovrastimate; qui diventa non-possibile usare `cond_entropy` o `num_var_10` come prova primaria di profondità senza recovery audit. + +## Self-Falsification — 5 lenti +- **L1 hard constraint vs bias**: nessun claim "zero/never/always". La frazione mod3 è 0.158032, dichiarata bias, non proibizione. +- **L2 quantità vs ratio**: il claim principale usa layer recuperato e z-score, non percentuali fra spazi di taglia diversa. +- **L3 no silent patching**: dichiaro esplicitamente che `HL_markov` era nome scorretto per un controllo AR(1) lag1-matched. +- **L4 edge cases**: i fallimenti `cond_entropy` e `num_var_10` non vengono arrotondati via; diventano il vincolo. +- **L5 rediscovery**: non taggo NEW. Riferimenti classici vicini: surrogate-data tests for Markov order e constrained Markov-order surrogates. + +## References +- Kandel et al.-lineage / surrogate tests: "Testing the order of discrete Markov chains using surrogate data", Physica D 117, 1998. +- "Exact significance test for Markov order", Physica D 269, 2014. +- "Constrained Markov order surrogates", Physica D 2020. + +## Files +- `tools/exp_markov_layer_recovery_audit.py` +- `tools/data/markov_layer_recovery_audit.json` +- `tools/data/meta_tautology_test.json` +- `tools/data/seme.json` +- `tools/data/reports/agent_20260504_1219.md` diff --git a/tools/exp_markov_layer_recovery_audit.py b/tools/exp_markov_layer_recovery_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..9797ea889603167a58cd989c71d02cf2aaf927bf --- /dev/null +++ b/tools/exp_markov_layer_recovery_audit.py @@ -0,0 +1,179 @@ +#!/usr/bin/env python3 +""" +exp_markov_layer_recovery_audit.py — META audit for the Markov-layer classifier. + +Question: + If a sequence is generated with known Markov order k, does the existing + Mk0/Mk1/Mk2 classifier recover layer <= k? + +Why this matters: + Recent reports used Mk surrogate capture to claim two visible memory layers. + This audit tests the opposite: whether the classifier itself creates apparent + higher layers on finite Markov controls. + +Controls: + - prime_gaps: unknown real target, reported for comparison only + - empirical_Mk0: shuffled prime gaps; known order 0 + - empirical_Mk1: prime-trained Mk1 surrogate; known order 1 + - empirical_Mk2: prime-trained Mk2 surrogate; known order 2 + - poisson_iid: independent exponential spacings; known order 0 + +Verdict rule: + A control fails recovery when any observable is assigned layer > known order. +""" + +import argparse +import json +from pathlib import Path + +import numpy as np + +from exp_two_layer_universality import ( + OBSERVABLES, + classify_layer, + gen_prime_gaps, + generate_markov_surrogate, +) + + +def measure_all(gaps): + out = {} + for name, fn in OBSERVABLES.items(): + try: + value = float(fn(gaps)) + except Exception: + value = float("nan") + out[name] = value + return out + + +def classifier_pass(gaps, n_surr, rng): + real_obs = measure_all(gaps) + z_scores = {} + + for mk in (0, 1, 2): + surr_obs = {name: [] for name in OBSERVABLES} + for _ in range(n_surr): + if mk == 0: + surr = rng.permutation(gaps) + else: + surr = generate_markov_surrogate(gaps, mk, rng=rng) + for obs_name, obs_fn in OBSERVABLES.items(): + try: + surr_obs[obs_name].append(float(obs_fn(surr))) + except Exception: + pass + + z_mk = {} + for obs_name in OBSERVABLES: + vals = np.asarray(surr_obs[obs_name], dtype=float) + vals = vals[np.isfinite(vals)] + if len(vals) > 2 and np.std(vals) > 1e-12 and np.isfinite(real_obs[obs_name]): + z = (real_obs[obs_name] - np.mean(vals)) / np.std(vals) + else: + z = 0.0 + z_mk[obs_name] = round(float(z), 2) + z_scores[f"Mk{mk}"] = z_mk + + layers = {} + for obs_name in OBSERVABLES: + layers[obs_name] = classify_layer( + z_scores["Mk0"][obs_name], + z_scores["Mk1"][obs_name], + z_scores["Mk2"][obs_name], + ) + + return real_obs, z_scores, layers + + +def build_controls(prime_gaps, rng): + return { + "prime_gaps": { + "known_order": None, + "gaps": prime_gaps, + }, + "empirical_Mk0": { + "known_order": 0, + "gaps": rng.permutation(prime_gaps), + }, + "empirical_Mk1": { + "known_order": 1, + "gaps": generate_markov_surrogate(prime_gaps, 1, rng=rng), + }, + "empirical_Mk2": { + "known_order": 2, + "gaps": generate_markov_surrogate(prime_gaps, 2, rng=rng), + }, + "poisson_iid": { + "known_order": 0, + "gaps": rng.exponential(1.0, len(prime_gaps)), + }, + } + + +def summarize_failure(layers, known_order): + if known_order is None: + return { + "status": "target_only", + "over_layer_observables": [], + "max_layer": int(max(layers.values())), + } + + over = [name for name, layer in layers.items() if layer > known_order] + return { + "status": "PASS" if not over else "FAIL", + "over_layer_observables": over, + "max_layer": int(max(layers.values())), + } + + +def run(N=60000, n_surr=20, seed=20260504): + rng = np.random.default_rng(seed) + prime_gaps = gen_prime_gaps(N).astype(float) + controls = build_controls(prime_gaps, rng) + + results = { + "N": int(N), + "n_surr": int(n_surr), + "seed": int(seed), + "controls": {}, + } + + print(f"N={N}, n_surr={n_surr}, seed={seed}") + print(f"{'sequence':<16} {'known':<7} {'status':<8} {'maxL':<5} over-layer observables") + print("-" * 82) + + for name, spec in controls.items(): + real_obs, z_scores, layers = classifier_pass(spec["gaps"], n_surr, rng) + summary = summarize_failure(layers, spec["known_order"]) + known = "target" if spec["known_order"] is None else f"L{spec['known_order']}" + over = ", ".join(summary["over_layer_observables"]) or "-" + print(f"{name:<16} {known:<7} {summary['status']:<8} L{summary['max_layer']:<4} {over}") + + results["controls"][name] = { + "known_order": spec["known_order"], + "n_gaps": int(len(spec["gaps"])), + "real_obs": {k: round(v, 6) for k, v in real_obs.items()}, + "z_scores": z_scores, + "layers": layers, + "recovery": summary, + } + + out_path = Path("tools/data/markov_layer_recovery_audit.json") + with out_path.open("w") as f: + json.dump(results, f, indent=2) + print(f"\nSaved to {out_path}") + return results + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--N", type=int, default=60000) + parser.add_argument("--n_surr", type=int, default=20) + parser.add_argument("--seed", type=int, default=20260504) + args = parser.parse_args() + run(N=args.N, n_surr=args.n_surr, seed=args.seed) + + +if __name__ == "__main__": + main() codex Report e seme sono scritti. Faccio le verifiche finali: JSON valido, file presenti, e controllo diff per assicurarmi di aver toccato solo strumenti/report/stato richiesti. exec /bin/bash -lc "sed -n '1,260p' tools/data/reports/agent_20260504_1219.md" in /opt/MM_D-ND succeeded in 0ms: # Agent Report — Markov Layers Pass the First Recovery Gate, but Not All Observables Pass **Date**: 2026-05-04 12:19 **Piano**: 61 **Tension explored**: META (0.5) ## Claim Under Test > Tutti i 11 test passano — verifica che non stiamo testando solo tautologie. ## Question Se la pipeline Mk0/Mk1/Mk2 vede due layer, li recupera anche quando l'ordine vero della sequenza è noto? ## Experiment Design - **Primary audit**: `tools/exp_markov_layer_recovery_audit.py` - **Scope**: 60,000 gap/spacings per sequenza; 20 surrogati per livello Mk; seed `20260504`. - **Target**: prime gaps, ordine sconosciuto, solo confronto. - **Known-order controls**: - `empirical_Mk0`: permutazione dei gap primi, ordine noto 0. - `empirical_Mk1`: surrogate Mk1 addestrato sui gap primi, ordine noto 1. - `empirical_Mk2`: surrogate Mk2 addestrato sui gap primi, ordine noto 2. - `poisson_iid`: spacings esponenziali iid, ordine noto 0. - **Recovery rule**: un controllo passa se nessun osservabile viene classificato con layer maggiore dell'ordine noto. - **Secondary calibration**: `tools/exp_meta_tautology_test.py --n-primes 300000 --n-trials 12`. Nota: il controllo chiamato `HL_markov` nello script è un AR(1) gap model che preserva lag-1, non un modello Hardy-Littlewood aritmetico. ## Results ### Recovery audit | Sequence | Known order | Recovery | Max layer | Over-layer observables | |---|---:|---|---:|---| | `prime_gaps` | target | target only | 3 | - | | `empirical_Mk0` | 0 | PASS | 0 | - | | `empirical_Mk1` | 1 | PASS | 1 | - | | `empirical_Mk2` | 2 | FAIL | 3 | `cond_entropy` | | `poisson_iid` | 0 | FAIL | 1 | `num_var_10` | Prime target layers, reported without recovery verdict: | Observable | Layer | z vs Mk0 | z vs Mk1 | z vs Mk2 | |---|---:|---:|---:|---:| | SR | 1 | -22.21 | -0.08 | -0.16 | | L1 | 1 | -12.25 | 0.79 | 0.86 | | L2 | 2 | -3.60 | -4.09 | -0.19 | | SR2 | 2 | -6.67 | -8.80 | -0.33 | | cond_entropy | 3 | -435.40 | -26.11 | 2.56 | | triple_var | 1 | -15.90 | -0.53 | 0.36 | | num_var_10 | 1 | -7.46 | -1.42 | 0.00 | | run_length | 3 | 27.59 | 7.19 | 3.43 | Control failure details: | Control | Observable | Expected max | Assigned | Diagnostic z | |---|---|---:|---:|---:| | `empirical_Mk2` | `cond_entropy` | 2 | 3 | z vs Mk2 = 3.15 | | `poisson_iid` | `num_var_10` | 0 | 1 | z vs Mk0 = 2.13 | ### Secondary tautology calibration | Observable | Verdict in tested controls | Blocking control | |---|---|---| | r-stat | STRUCTURAL | none in this run | | mod3 fraction | STRUCTURAL | none in this run | | lag1 ACF | TAUTOLOGICAL | lag1-matched AR(1), z = -0.62 | | dr_ord | TAUTOLOGICAL | lag1-matched AR(1), z = -1.90 | | dr_mag | STRUCTURAL | none in this run | Numeri principali: prime mod3 same-class fraction = 0.158032. Questo è bias forte rispetto ai controlli ~0.34-0.35, non proibizione zero. ## Key Findings 1. **Il nucleo pair/triple non cade al primo audit.** `empirical_Mk0` recupera solo Layer 0; `empirical_Mk1` recupera solo Layer 1. Questo supporta l'uso di SR e L1 come Layer 1 e di L2/SR2 come Layer 2 nel perimetro testato. 2. **`cond_entropy` non può essere usata come prova forte di Layer 3 nel setup attuale.** Anche un controllo generato come Mk2 viene letto Layer 3 per `cond_entropy` (z vs Mk2 = 3.15). Questo indica bias del surrogate/binning o varianza finita, non necessariamente memoria reale oltre Mk2. 3. **`num_var_10` produce un falso positivo debole su iid.** `poisson_iid` viene letto Layer 1 per `num_var_10` con z = 2.13. Il segnale supera appena la soglia operativa, quindi va trattato come diagnostica secondaria finché non passa audit multi-seed. 4. **Il vecchio controllo `HL_markov` va rinominato nel linguaggio dei report.** Lo script non implementa Hardy-Littlewood; implementa un AR(1) lag1-matched. I claim futuri non devono chiamarlo controllo HL aritmetico. ## Verdict **CONSTRAINT on META + BOUNDARY**: la decomposizione two-layer è utilizzabile solo con perimetro atomico. Perimetro corretto: nei 60,000 gap primi testati con 20 surrogati Mk per livello, il nucleo `SR,L1` viene catturato da Mk1 e il nucleo `L2,SR2` da Mk2. Le diagnostiche `cond_entropy`, `num_var_10`, `run_length` non sono evidenza primaria finché non passano un recovery audit multi-seed e, idealmente, surrogati Markov esatti. Non è una scoperta nuova di teoria Markov: è un audit metodologico vicino ai test di ordine Markov con surrogate data. La letteratura rilevante include test esatti di ordine Markov e surrogate vincolati che preservano proprietà Markov di ordine n, quindi il prossimo strumento deve avvicinarsi a quei null esatti invece di usare solo generatori bin-pool approssimati. ## Bicono della scoperta - **Due radici**: layer osservato dai dati · ordine noto del generatore. - **Singolare**: la pipeline Mk stessa, prima che venga usata come prova su un dominio reale. - **Invariante di passaggio**: un claim di memoria sopravvive solo se recupera correttamente i controlli con ordine noto. - **Campo di possibilità**: qui diventa possibile separare nucleo robusto e diagnostiche sovrastimate; qui diventa non-possibile usare `cond_entropy` o `num_var_10` come prova primaria di profondità senza recovery audit. ## Self-Falsification — 5 lenti - **L1 hard constraint vs bias**: nessun claim "zero/never/always". La frazione mod3 è 0.158032, dichiarata bias, non proibizione. - **L2 quantità vs ratio**: il claim principale usa layer recuperato e z-score, non percentuali fra spazi di taglia diversa. - **L3 no silent patching**: dichiaro esplicitamente che `HL_markov` era nome scorretto per un controllo AR(1) lag1-matched. - **L4 edge cases**: i fallimenti `cond_entropy` e `num_var_10` non vengono arrotondati via; diventano il vincolo. - **L5 rediscovery**: non taggo NEW. Riferimenti classici vicini: surrogate-data tests for Markov order e constrained Markov-order surrogates. ## References - Kandel et al.-lineage / surrogate tests: "Testing the order of discrete Markov chains using surrogate data", Physica D 117, 1998. - "Exact significance test for Markov order", Physica D 269, 2014. - "Constrained Markov order surrogates", Physica D 2020. ## Files - `tools/exp_markov_layer_recovery_audit.py` - `tools/data/markov_layer_recovery_audit.json` - `tools/data/meta_tautology_test.json` - `tools/data/seme.json` - `tools/data/reports/agent_20260504_1219.md` exec /bin/bash -lc 'git diff -- tools/exp_markov_layer_recovery_audit.py tools/data/reports/agent_20260504_1219.md tools/data/seme.json tools/data/markov_layer_recovery_audit.json tools/data/meta_tautology_test.json' in /opt/MM_D-ND succeeded in 0ms: diff --git a/tools/data/seme.json b/tools/data/seme.json index a31dc54..cbae977 100644 --- a/tools/data/seme.json +++ b/tools/data/seme.json @@ -1,6 +1,6 @@ { - "timestamp": "2026-04-23T03:35:37.611830", - "piano": 48, + "timestamp": "2026-05-04T12:19:00", + "piano": 61, "tensioni": [ { "tipo": "confine_inesplorato", @@ -87,12 +87,21 @@ "condensato_ref": null, "condensato_motivo": "Ricorrente (3x in 2 giorni) e fuori dalla mappa", "porta": "novità" + }, + { + "tipo": "vincolo_meta", + "id": "META_LAYER_RECOVERY_AUDIT", + "claim": "La pipeline Mk0/Mk1/Mk2 recupera correttamente i controlli empirici Mk0 e Mk1, ma sovrastima alcuni osservabili: cond_entropy su controllo Mk2 viene letto Layer 3 e num_var_10 su Poisson iid viene letto Layer 1. I claim two-layer devono quindi usare SR,L1,L2,SR2 come nucleo e dichiarare cond_entropy/num_var_10/run_length come diagnostiche secondarie finche non passano audit di recupero multi-seed.", + "intensita": 0.7, + "nota": "Report agent_20260504_1219: recovery audit con N=60000, n_surr=20, seed=20260504. Risultato: empirical_Mk0 PASS maxL0; empirical_Mk1 PASS maxL1; empirical_Mk2 FAIL cond_entropy Layer3; poisson_iid FAIL num_var_10 Layer1.", + "porta": "meta_audit", + "condensato_ref": "A4,A8,A14", + "condensato_motivo": "No silent patching: separa nucleo robusto e diagnostiche sovrastimate. Evita rediscovery/tautologia dei controlli." } ], "potenziale_bloccato": [], "varianza": [ - "Nuove tensioni: {'META'}", - "Tensioni risolte: {'METRIC_TENSOR', 'TRASCENDENZA_LIMITE', 'TENSIONE_ENTITA', 'PIANO_PRIMARIO_DUE_ASSIOMI', 'DUALITA_DIPOLARE_VS_ILLUSORIA', 'G_POTENZIALE_NULLA'}" + "Tensioni risolte: {'METRIC_TENSOR', 'G_POTENZIALE_NULLA', 'DUALITA_DIPOLARE_VS_ILLUSORIA', 'TRASCENDENZA_LIMITE', 'PIANO_PRIMARIO_DUE_ASSIOMI', 'TENSIONE_ENTITA', 'DIPOLAR_ORDERING'}" ], "filtro": { "promosse": 8, exec /bin/bash -lc 'python -m json.tool tools/data/seme.json >/tmp/seme_check.json && python -m json.tool tools/data/markov_layer_recovery_audit.json >/tmp/recovery_check.json && echo ok' in /opt/MM_D-ND diff --git a/tools/data/reports/agent_20260504_1219.md b/tools/data/reports/agent_20260504_1219.md new file mode 100644 index 0000000000000000000000000000000000000000..683dab4bbab940600d05e182ee1b696459ee180c --- /dev/null +++ b/tools/data/reports/agent_20260504_1219.md @@ -0,0 +1,104 @@ +# Agent Report — Markov Layers Pass the First Recovery Gate, but Not All Observables Pass +**Date**: 2026-05-04 12:19 +**Piano**: 61 +**Tension explored**: META (0.5) + +## Claim Under Test +> Tutti i 11 test passano — verifica che non stiamo testando solo tautologie. + +## Question +Se la pipeline Mk0/Mk1/Mk2 vede due layer, li recupera anche quando l'ordine vero della sequenza è noto? + +## Experiment Design +- **Primary audit**: `tools/exp_markov_layer_recovery_audit.py` +- **Scope**: 60,000 gap/spacings per sequenza; 20 surrogati per livello Mk; seed `20260504`. +- **Target**: prime gaps, ordine sconosciuto, solo confronto. +- **Known-order controls**: + - `empirical_Mk0`: permutazione dei gap primi, ordine noto 0. + - `empirical_Mk1`: surrogate Mk1 addestrato sui gap primi, ordine noto 1. + - `empirical_Mk2`: surrogate Mk2 addestrato sui gap primi, ordine noto 2. + - `poisson_iid`: spacings esponenziali iid, ordine noto 0. +- **Recovery rule**: un controllo passa se nessun osservabile viene classificato con layer maggiore dell'ordine noto. +- **Secondary calibration**: `tools/exp_meta_tautology_test.py --n-primes 300000 --n-trials 12`. Nota: il controllo chiamato `HL_markov` nello script è un AR(1) gap model che preserva lag-1, non un modello Hardy-Littlewood aritmetico. + +## Results + +### Recovery audit + +| Sequence | Known order | Recovery | Max layer | Over-layer observables | +|---|---:|---|---:|---| +| `prime_gaps` | target | target only | 3 | - | +| `empirical_Mk0` | 0 | PASS | 0 | - | +| `empirical_Mk1` | 1 | PASS | 1 | - | +| `empirical_Mk2` | 2 | FAIL | 3 | `cond_entropy` | +| `poisson_iid` | 0 | FAIL | 1 | `num_var_10` | + +Prime target layers, reported without recovery verdict: + +| Observable | Layer | z vs Mk0 | z vs Mk1 | z vs Mk2 | +|---|---:|---:|---:|---:| +| SR | 1 | -22.21 | -0.08 | -0.16 | +| L1 | 1 | -12.25 | 0.79 | 0.86 | +| L2 | 2 | -3.60 | -4.09 | -0.19 | +| SR2 | 2 | -6.67 | -8.80 | -0.33 | +| cond_entropy | 3 | -435.40 | -26.11 | 2.56 | +| triple_var | 1 | -15.90 | -0.53 | 0.36 | +| num_var_10 | 1 | -7.46 | -1.42 | 0.00 | +| run_length | 3 | 27.59 | 7.19 | 3.43 | + +Control failure details: + +| Control | Observable | Expected max | Assigned | Diagnostic z | +|---|---|---:|---:|---:| +| `empirical_Mk2` | `cond_entropy` | 2 | 3 | z vs Mk2 = 3.15 | +| `poisson_iid` | `num_var_10` | 0 | 1 | z vs Mk0 = 2.13 | + +### Secondary tautology calibration + +| Observable | Verdict in tested controls | Blocking control | +|---|---|---| +| r-stat | STRUCTURAL | none in this run | +| mod3 fraction | STRUCTURAL | none in this run | +| lag1 ACF | TAUTOLOGICAL | lag1-matched AR(1), z = -0.62 | +| dr_ord | TAUTOLOGICAL | lag1-matched AR(1), z = -1.90 | +| dr_mag | STRUCTURAL | none in this run | + +Numeri principali: prime mod3 same-class fraction = 0.158032. Questo è bias forte rispetto ai controlli ~0.34-0.35, non proibizione zero. + +## Key Findings +1. **Il nucleo pair/triple non cade al primo audit.** `empirical_Mk0` recupera solo Layer 0; `empirical_Mk1` recupera solo Layer 1. Questo supporta l'uso di SR e L1 come Layer 1 e di L2/SR2 come Layer 2 nel perimetro testato. +2. **`cond_entropy` non può essere usata come prova forte di Layer 3 nel setup attuale.** Anche un controllo generato come Mk2 viene letto Layer 3 per `cond_entropy` (z vs Mk2 = 3.15). Questo indica bias del surrogate/binning o varianza finita, non necessariamente memoria reale oltre Mk2. +3. **`num_var_10` produce un falso positivo debole su iid.** `poisson_iid` viene letto Layer 1 per `num_var_10` con z = 2.13. Il segnale supera appena la soglia operativa, quindi va trattato come diagnostica secondaria finché non passa audit multi-seed. +4. **Il vecchio controllo `HL_markov` va rinominato nel linguaggio dei report.** Lo script non implementa Hardy-Littlewood; implementa un AR(1) lag1-matched. I claim futuri non devono chiamarlo controllo HL aritmetico. + +## Verdict +**CONSTRAINT on META + BOUNDARY**: la decomposizione two-layer è utilizzabile solo con perimetro atomico. + +Perimetro corretto: nei 60,000 gap primi testati con 20 surrogati Mk per livello, il nucleo `SR,L1` viene catturato da Mk1 e il nucleo `L2,SR2` da Mk2. Le diagnostiche `cond_entropy`, `num_var_10`, `run_length` non sono evidenza primaria finché non passano un recovery audit multi-seed e, idealmente, surrogati Markov esatti. + +Non è una scoperta nuova di teoria Markov: è un audit metodologico vicino ai test di ordine Markov con surrogate data. La letteratura rilevante include test esatti di ordine Markov e surrogate vincolati che preservano proprietà Markov di ordine n, quindi il prossimo strumento deve avvicinarsi a quei null esatti invece di usare solo generatori bin-pool approssimati. + +## Bicono della scoperta +- **Due radici**: layer osservato dai dati · ordine noto del generatore. +- **Singolare**: la pipeline Mk stessa, prima che venga usata come prova su un dominio reale. +- **Invariante di passaggio**: un claim di memoria sopravvive solo se recupera correttamente i controlli con ordine noto. +- **Campo di possibilità**: qui diventa possibile separare nucleo robusto e diagnostiche sovrastimate; qui diventa non-possibile usare `cond_entropy` o `num_var_10` come prova primaria di profondità senza recovery audit. + +## Self-Falsification — 5 lenti +- **L1 hard constraint vs bias**: nessun claim "zero/never/always". La frazione mod3 è 0.158032, dichiarata bias, non proibizione. +- **L2 quantità vs ratio**: il claim principale usa layer recuperato e z-score, non percentuali fra spazi di taglia diversa. +- **L3 no silent patching**: dichiaro esplicitamente che `HL_markov` era nome scorretto per un controllo AR(1) lag1-matched. +- **L4 edge cases**: i fallimenti `cond_entropy` e `num_var_10` non vengono arrotondati via; diventano il vincolo. +- **L5 rediscovery**: non taggo NEW. Riferimenti classici vicini: surrogate-data tests for Markov order e constrained Markov-order surrogates. + +## References +- Kandel et al.-lineage / surrogate tests: "Testing the order of discrete Markov chains using surrogate data", Physica D 117, 1998. +- "Exact significance test for Markov order", Physica D 269, 2014. +- "Constrained Markov order surrogates", Physica D 2020. + +## Files +- `tools/exp_markov_layer_recovery_audit.py` +- `tools/data/markov_layer_recovery_audit.json` +- `tools/data/meta_tautology_test.json` +- `tools/data/seme.json` +- `tools/data/reports/agent_20260504_1219.md` diff --git a/tools/exp_markov_layer_recovery_audit.py b/tools/exp_markov_layer_recovery_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..9797ea889603167a58cd989c71d02cf2aaf927bf --- /dev/null +++ b/tools/exp_markov_layer_recovery_audit.py @@ -0,0 +1,179 @@ +#!/usr/bin/env python3 +""" +exp_markov_layer_recovery_audit.py — META audit for the Markov-layer classifier. + +Question: + If a sequence is generated with known Markov order k, does the existing + Mk0/Mk1/Mk2 classifier recover layer <= k? + +Why this matters: + Recent reports used Mk surrogate capture to claim two visible memory layers. + This audit tests the opposite: whether the classifier itself creates apparent + higher layers on finite Markov controls. + +Controls: + - prime_gaps: unknown real target, reported for comparison only + - empirical_Mk0: shuffled prime gaps; known order 0 + - empirical_Mk1: prime-trained Mk1 surrogate; known order 1 + - empirical_Mk2: prime-trained Mk2 surrogate; known order 2 + - poisson_iid: independent exponential spacings; known order 0 + +Verdict rule: + A control fails recovery when any observable is assigned layer > known order. +""" + +import argparse +import json +from pathlib import Path + +import numpy as np + +from exp_two_layer_universality import ( + OBSERVABLES, + classify_layer, + gen_prime_gaps, + generate_markov_surrogate, +) + + +def measure_all(gaps): + out = {} + for name, fn in OBSERVABLES.items(): + try: + value = float(fn(gaps)) + except Exception: + value = float("nan") + out[name] = value + return out + + +def classifier_pass(gaps, n_surr, rng): + real_obs = measure_all(gaps) + z_scores = {} + + for mk in (0, 1, 2): + surr_obs = {name: [] for name in OBSERVABLES} + for _ in range(n_surr): + if mk == 0: + surr = rng.permutation(gaps) + else: + surr = generate_markov_surrogate(gaps, mk, rng=rng) + for obs_name, obs_fn in OBSERVABLES.items(): + try: + surr_obs[obs_name].append(float(obs_fn(surr))) + except Exception: + pass + + z_mk = {} + for obs_name in OBSERVABLES: + vals = np.asarray(surr_obs[obs_name], dtype=float) + vals = vals[np.isfinite(vals)] + if len(vals) > 2 and np.std(vals) > 1e-12 and np.isfinite(real_obs[obs_name]): + z = (real_obs[obs_name] - np.mean(vals)) / np.std(vals) + else: + z = 0.0 + z_mk[obs_name] = round(float(z), 2) + z_scores[f"Mk{mk}"] = z_mk + + layers = {} + for obs_name in OBSERVABLES: + layers[obs_name] = classify_layer( + z_scores["Mk0"][obs_name], + z_scores["Mk1"][obs_name], + z_scores["Mk2"][obs_name], + ) + + return real_obs, z_scores, layers + + +def build_controls(prime_gaps, rng): + return { + "prime_gaps": { + "known_order": None, + "gaps": prime_gaps, + }, + "empirical_Mk0": { + "known_order": 0, + "gaps": rng.permutation(prime_gaps), + }, + "empirical_Mk1": { + "known_order": 1, + "gaps": generate_markov_surrogate(prime_gaps, 1, rng=rng), + }, + "empirical_Mk2": { + "known_order": 2, + "gaps": generate_markov_surrogate(prime_gaps, 2, rng=rng), + }, + "poisson_iid": { + "known_order": 0, + "gaps": rng.exponential(1.0, len(prime_gaps)), + }, + } + + +def summarize_failure(layers, known_order): + if known_order is None: + return { + "status": "target_only", + "over_layer_observables": [], + "max_layer": int(max(layers.values())), + } + + over = [name for name, layer in layers.items() if layer > known_order] + return { + "status": "PASS" if not over else "FAIL", + "over_layer_observables": over, + "max_layer": int(max(layers.values())), + } + + +def run(N=60000, n_surr=20, seed=20260504): + rng = np.random.default_rng(seed) + prime_gaps = gen_prime_gaps(N).astype(float) + controls = build_controls(prime_gaps, rng) + + results = { + "N": int(N), + "n_surr": int(n_surr), + "seed": int(seed), + "controls": {}, + } + + print(f"N={N}, n_surr={n_surr}, seed={seed}") + print(f"{'sequence':<16} {'known':<7} {'status':<8} {'maxL':<5} over-layer observables") + print("-" * 82) + + for name, spec in controls.items(): + real_obs, z_scores, layers = classifier_pass(spec["gaps"], n_surr, rng) + summary = summarize_failure(layers, spec["known_order"]) + known = "target" if spec["known_order"] is None else f"L{spec['known_order']}" + over = ", ".join(summary["over_layer_observables"]) or "-" + print(f"{name:<16} {known:<7} {summary['status']:<8} L{summary['max_layer']:<4} {over}") + + results["controls"][name] = { + "known_order": spec["known_order"], + "n_gaps": int(len(spec["gaps"])), + "real_obs": {k: round(v, 6) for k, v in real_obs.items()}, + "z_scores": z_scores, + "layers": layers, + "recovery": summary, + } + + out_path = Path("tools/data/markov_layer_recovery_audit.json") + with out_path.open("w") as f: + json.dump(results, f, indent=2) + print(f"\nSaved to {out_path}") + return results + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--N", type=int, default=60000) + parser.add_argument("--n_surr", type=int, default=20) + parser.add_argument("--seed", type=int, default=20260504) + args = parser.parse_args() + run(N=args.N, n_surr=args.n_surr, seed=args.seed) + + +if __name__ == "__main__": + main() succeeded in 100ms: exec /bin/bash -lc 'python -m py_compile tools/exp_markov_layer_recovery_audit.py' in /opt/MM_D-ND succeeded in 0ms: exec /bin/bash -lc 'git status --short' in /opt/MM_D-ND succeeded in 0ms: M applications/INDEX.draft.json M applications/soluzioni/20260430_0330_three-regimes-of-modular-memory-algebraic/finding_index.draft.json M applications/soluzioni/20260430_0330_three-regimes-of-modular-memory-algebraic/manifest.draft.json M tools/data/agent_field_live.md M tools/data/conoscenza_teorie.json M tools/data/consecutio.json M tools/data/consecutio_processata.json M tools/data/domande_fondamentali.json M tools/data/incrocio_risultato.json M tools/data/lab_data.json M tools/data/lab_graph.json M tools/data/lab_health.json M tools/data/lab_session_log.jsonl M tools/data/pipeline_state.json M tools/data/ponti_evoluti.json M tools/data/refresh_detector_state.json M tools/data/seme.json D tools/data/seme_archive/piano_37.json D tools/data/seme_archive/piano_38.json D tools/data/seme_archive/piano_4.json D tools/data/seme_archive/piano_5.json M tools/data/seme_axioms.json M tools/data/seme_backup_pre_run.json M tools/data/tm1_figures/tensions.json M tools/data/tm1_figures/tensions_raw.json ?? .codex ?? applications/published/20260504_0901_the-two-markov-layers-are-coupled/ ?? applications/scoperte/20260502_0330_the-dipolar-phase-transition-is-nearly_auto/ ?? applications/scoperte/20260503_0330_markov-memory-has-two-visible-layers_auto/ ?? applications/scoperte/20260504_0901_the-two-markov-layers-are-coupled_auto/ ?? applications/soluzioni/20260502_0330_the-dipolar-phase-transition-is-nearly/ ?? applications/soluzioni/20260503_0330_markov-memory-has-two-visible-layers/ ?? applications/soluzioni/20260504_0901_the-two-markov-layers-are-coupled/ ?? tools/data/.last_telegram_msg ?? tools/data/3d_boundary_layers.json ?? tools/data/alignment_active.json ?? tools/data/alignment_markers.jsonl ?? tools/data/bicono_projections.jsonl ?? tools/data/boundary_coherence.json ?? tools/data/boundary_shuffle_audit.json ?? tools/data/brody_calibration_results.json ?? tools/data/brody_flow.json ?? tools/data/conoscenza_teorie.json.bak.retraction_22_04 ?? tools/data/cross_domain_dipolar_direction.json ?? tools/data/cross_observable_consistency.json ?? tools/data/crossover_phase_test.json ?? tools/data/dipolar_crossover.json ?? tools/data/dipolar_vector_scaling.json ?? tools/data/domandatore/domandatore_20260421_0746.json ?? tools/data/domandatore/domandatore_20260422_0345.json ?? tools/data/domandatore/domandatore_20260423_0345.json ?? tools/data/domandatore/domandatore_20260424_0345.json ?? tools/data/domandatore/domandatore_20260425_0345.json ?? tools/data/domandatore/domandatore_20260426_0345.json ?? tools/data/domandatore/domandatore_20260427_0345.json ?? tools/data/domandatore/domandatore_20260428_0345.json ?? tools/data/domandatore/domandatore_20260428_1236.json ?? tools/data/domandatore/domandatore_20260429_0345.json ?? tools/data/domandatore/domandatore_20260430_0345.json ?? tools/data/domandatore/domandatore_20260501_0345.json ?? tools/data/domandatore/domandatore_20260502_0345.json ?? tools/data/domandatore/domandatore_20260503_0345.json ?? tools/data/domandatore/domandatore_20260504_0345.json ?? tools/data/evolution/evolution_20260422_0330.md ?? tools/data/evolution/evolution_20260422_1616.md ?? tools/data/evolution/evolution_20260423_0330.md ?? tools/data/evolution/evolution_20260424_0330.md ?? tools/data/evolution/evolution_20260425_0330.md ?? tools/data/evolution/evolution_20260427_0330.md ?? tools/data/evolution/evolution_20260428_0330.md ?? tools/data/evolution/evolution_20260429_0852.md ?? tools/data/evolution/evolution_20260429_1013.md ?? tools/data/evolution/evolution_20260430_0330.md ?? tools/data/evolution/evolution_20260430_1905.md ?? tools/data/evolution/evolution_20260430_1946.md ?? tools/data/evolution/evolution_20260501_0330.md ?? tools/data/evolution/evolution_20260501_0725.md ?? tools/data/evolution/evolution_20260501_0931.md ?? tools/data/evolution/evolution_20260502_0330.md ?? tools/data/evolution/evolution_20260503_0330.md ?? tools/data/evolution/evolution_20260504_0901.md ?? tools/data/incrocio_20260422_0336.json ?? tools/data/incrocio_20260423_0335.json ?? tools/data/incrocio_20260424_0347.json ?? tools/data/incrocio_20260425_0339.json ?? tools/data/incrocio_20260428_0340.json ?? tools/data/incrocio_20260429_0859.json ?? tools/data/lab_bridge_issues.jsonl ?? tools/data/magnitude_psd_from_acf.json ?? tools/data/markov3_observable_hunt.json ?? tools/data/markov_dipolar_decomposition.json ?? tools/data/markov_k_direction.json ?? tools/data/markov_layer_recovery_audit.json ?? tools/data/markov_memory_by_gue_type.json ?? tools/data/markov_scale_function.json ?? tools/data/meta_tautology_test.json ?? tools/data/mod3_scaling.json ?? tools/data/mod3_vs_residual_ordering.json ?? tools/data/modular_algebra_depth.json ?? tools/data/modular_memory_spectrum.json ?? tools/data/reports/_quarantine_falsifier_29_04/ ?? tools/data/reports/agent_20260422_0330.md ?? tools/data/reports/agent_20260422_1616.md ?? tools/data/reports/agent_20260423_0330.md ?? tools/data/reports/agent_20260424_0330.md ?? tools/data/reports/agent_20260425_0330.md ?? tools/data/reports/agent_20260426_0330.md ?? tools/data/reports/agent_20260427_0330.md ?? tools/data/reports/agent_20260428_0330.md ?? tools/data/reports/agent_20260429_1013.md ?? tools/data/reports/agent_20260429_1041.md ?? tools/data/reports/agent_20260430_0330.md ?? tools/data/reports/agent_20260430_1905.md ?? tools/data/reports/agent_20260430_1919.md ?? tools/data/reports/agent_20260430_1946.md ?? tools/data/reports/agent_20260501_0330.md ?? tools/data/reports/agent_20260501_0725.md ?? tools/data/reports/agent_20260501_0931.md ?? tools/data/reports/agent_20260502_0330.md ?? tools/data/reports/agent_20260503_0330.md ?? tools/data/reports/agent_20260504_0901.md ?? tools/data/reports/agent_20260504_1219.md ?? tools/data/reports/ddf_20260422_0405.json ?? tools/data/reports/ddf_20260423_0405.json ?? tools/data/reports/ddf_20260424_0405.json ?? tools/data/reports/ddf_20260425_0405.json ?? tools/data/reports/ddf_20260426_0405.json ?? tools/data/reports/ddf_20260427_0405.json ?? tools/data/reports/ddf_20260428_0405.json ?? tools/data/reports/ddf_20260429_0405.json ?? tools/data/reports/ddf_20260430_0405.json ?? tools/data/reports/ddf_20260501_0405.json ?? tools/data/reports/ddf_20260502_0405.json ?? tools/data/reports/ddf_20260503_0405.json ?? tools/data/reports/ddf_20260504_0405.json ?? tools/data/reports/evolution_20260422_0330.md ?? tools/data/reports/evolution_20260422_1616.md ?? tools/data/reports/evolution_20260423_0330.md ?? tools/data/reports/evolution_20260424_0330.md ?? tools/data/reports/evolution_20260425_0330.md ?? tools/data/reports/evolution_20260427_0330.md ?? tools/data/reports/evolution_20260428_0330.md ?? tools/data/reports/evolution_20260503_0330.md ?? tools/data/reports/evolution_20260504_0330.md ?? tools/data/reports/falsifier_20260429_1013.json ?? tools/data/reports/falsifier_20260429_1041.json ?? tools/data/reports/falsifier_20260430_0330.json ?? tools/data/reports/falsifier_20260430_1905.json ?? tools/data/reports/falsifier_20260430_1919.json ?? tools/data/reports/falsifier_20260430_1946.json ?? tools/data/reports/falsifier_20260501_0330.json ?? tools/data/reports/falsifier_20260501_0725.json ?? tools/data/reports/falsifier_20260501_0931.json ?? tools/data/reports/falsifier_20260502_0330.json ?? tools/data/reports/falsifier_20260503_0330.json ?? tools/data/reports/falsifier_20260504_0901.json ?? tools/data/reports/incident_20260504_0721.md ?? tools/data/reports/incident_20260504_1138.md ?? tools/data/reports/latest.md ?? tools/data/seme_archive/piano_56.json ?? tools/data/seme_archive/piano_57.json ?? tools/data/seme_archive/piano_59.json ?? tools/data/seme_archive/piano_60.json ?? tools/data/spectral_rigidity_results.json ?? tools/data/two_channel_boundary.json ?? tools/data/two_channel_cross_domain.json ?? tools/data/two_channel_shuffle_audit.json ?? tools/data/two_layer_universality.json ?? tools/data/valutatore_log.jsonl ?? tools/exp_3d_boundary_layers.py ?? tools/exp_boundary_coherence.py ?? tools/exp_boundary_shuffle_audit.py ?? tools/exp_brody_calibration.py ?? tools/exp_brody_flow.py ?? tools/exp_cross_domain_dipolar_direction.py ?? tools/exp_cross_observable_consistency.py ?? tools/exp_crossover_phase_test.py ?? tools/exp_dipolar_angle_reference.py ?? tools/exp_dipolar_crossover.py ?? tools/exp_dipolar_vector_scaling.py ?? tools/exp_magnitude_psd_from_acf.py ?? tools/exp_markov3_observable_hunt.py ?? tools/exp_markov_dipolar_decomposition.py ?? tools/exp_markov_k_direction.py ?? tools/exp_markov_layer_recovery_audit.py ?? tools/exp_markov_memory_by_gue_type.py ?? tools/exp_markov_psd_prediction.py ?? tools/exp_markov_scale_function.py ?? tools/exp_meta_tautology_test.py ?? tools/exp_mod3_scaling.py ?? tools/exp_mod3_vs_residual_ordering.py ?? tools/exp_modular_algebra_depth.py ?? tools/exp_modular_memory_spectrum.py ?? tools/exp_selective_layer_decoupling.py ?? tools/exp_spectral_rigidity.py ?? tools/exp_two_channel_boundary.py ?? tools/exp_two_channel_cross_domain.py ?? tools/exp_two_channel_shuffle_audit.py ?? tools/exp_two_layer_universality.py exec /bin/bash -lc 'rg -n "META_LAYER_RECOVERY_AUDIT|agent_20260504_1219|markov_layer_recovery" tools/data/seme.json tools/data/reports/agent_20260504_1219.md tools/exp_markov_layer_recovery_audit.py' in /opt/MM_D-ND succeeded in 0ms: tools/exp_markov_layer_recovery_audit.py:3:exp_markov_layer_recovery_audit.py — META audit for the Markov-layer classifier. tools/exp_markov_layer_recovery_audit.py:162: out_path = Path("tools/data/markov_layer_recovery_audit.json") tools/data/reports/agent_20260504_1219.md:13:- **Primary audit**: `tools/exp_markov_layer_recovery_audit.py` tools/data/reports/agent_20260504_1219.md:100:- `tools/exp_markov_layer_recovery_audit.py` tools/data/reports/agent_20260504_1219.md:101:- `tools/data/markov_layer_recovery_audit.json` tools/data/reports/agent_20260504_1219.md:104:- `tools/data/reports/agent_20260504_1219.md` tools/data/seme.json:93: "id": "META_LAYER_RECOVERY_AUDIT", tools/data/seme.json:96: "nota": "Report agent_20260504_1219: recovery audit con N=60000, n_surr=20, seed=20260504. Risultato: empirical_Mk0 PASS maxL0; empirical_Mk1 PASS maxL1; empirical_Mk2 FAIL cond_entropy Layer3; poisson_iid FAIL num_var_10 Layer1.", diff --git a/tools/data/reports/agent_20260504_1219.md b/tools/data/reports/agent_20260504_1219.md new file mode 100644 index 0000000000000000000000000000000000000000..683dab4bbab940600d05e182ee1b696459ee180c --- /dev/null +++ b/tools/data/reports/agent_20260504_1219.md @@ -0,0 +1,104 @@ +# Agent Report — Markov Layers Pass the First Recovery Gate, but Not All Observables Pass +**Date**: 2026-05-04 12:19 +**Piano**: 61 +**Tension explored**: META (0.5) + +## Claim Under Test +> Tutti i 11 test passano — verifica che non stiamo testando solo tautologie. + +## Question +Se la pipeline Mk0/Mk1/Mk2 vede due layer, li recupera anche quando l'ordine vero della sequenza è noto? + +## Experiment Design +- **Primary audit**: `tools/exp_markov_layer_recovery_audit.py` +- **Scope**: 60,000 gap/spacings per sequenza; 20 surrogati per livello Mk; seed `20260504`. +- **Target**: prime gaps, ordine sconosciuto, solo confronto. +- **Known-order controls**: + - `empirical_Mk0`: permutazione dei gap primi, ordine noto 0. + - `empirical_Mk1`: surrogate Mk1 addestrato sui gap primi, ordine noto 1. + - `empirical_Mk2`: surrogate Mk2 addestrato sui gap primi, ordine noto 2. + - `poisson_iid`: spacings esponenziali iid, ordine noto 0. +- **Recovery rule**: un controllo passa se nessun osservabile viene classificato con layer maggiore dell'ordine noto. +- **Secondary calibration**: `tools/exp_meta_tautology_test.py --n-primes 300000 --n-trials 12`. Nota: il controllo chiamato `HL_markov` nello script è un AR(1) gap model che preserva lag-1, non un modello Hardy-Littlewood aritmetico. + +## Results + +### Recovery audit + +| Sequence | Known order | Recovery | Max layer | Over-layer observables | +|---|---:|---|---:|---| +| `prime_gaps` | target | target only | 3 | - | +| `empirical_Mk0` | 0 | PASS | 0 | - | +| `empirical_Mk1` | 1 | PASS | 1 | - | +| `empirical_Mk2` | 2 | FAIL | 3 | `cond_entropy` | +| `poisson_iid` | 0 | FAIL | 1 | `num_var_10` | + +Prime target layers, reported without recovery verdict: + +| Observable | Layer | z vs Mk0 | z vs Mk1 | z vs Mk2 | +|---|---:|---:|---:|---:| +| SR | 1 | -22.21 | -0.08 | -0.16 | +| L1 | 1 | -12.25 | 0.79 | 0.86 | +| L2 | 2 | -3.60 | -4.09 | -0.19 | +| SR2 | 2 | -6.67 | -8.80 | -0.33 | +| cond_entropy | 3 | -435.40 | -26.11 | 2.56 | +| triple_var | 1 | -15.90 | -0.53 | 0.36 | +| num_var_10 | 1 | -7.46 | -1.42 | 0.00 | +| run_length | 3 | 27.59 | 7.19 | 3.43 | + +Control failure details: + +| Control | Observable | Expected max | Assigned | Diagnostic z | +|---|---|---:|---:|---:| +| `empirical_Mk2` | `cond_entropy` | 2 | 3 | z vs Mk2 = 3.15 | +| `poisson_iid` | `num_var_10` | 0 | 1 | z vs Mk0 = 2.13 | + +### Secondary tautology calibration + +| Observable | Verdict in tested controls | Blocking control | +|---|---|---| +| r-stat | STRUCTURAL | none in this run | +| mod3 fraction | STRUCTURAL | none in this run | +| lag1 ACF | TAUTOLOGICAL | lag1-matched AR(1), z = -0.62 | +| dr_ord | TAUTOLOGICAL | lag1-matched AR(1), z = -1.90 | +| dr_mag | STRUCTURAL | none in this run | + +Numeri principali: prime mod3 same-class fraction = 0.158032. Questo è bias forte rispetto ai controlli ~0.34-0.35, non proibizione zero. + +## Key Findings +1. **Il nucleo pair/triple non cade al primo audit.** `empirical_Mk0` recupera solo Layer 0; `empirical_Mk1` recupera solo Layer 1. Questo supporta l'uso di SR e L1 come Layer 1 e di L2/SR2 come Layer 2 nel perimetro testato. +2. **`cond_entropy` non può essere usata come prova forte di Layer 3 nel setup attuale.** Anche un controllo generato come Mk2 viene letto Layer 3 per `cond_entropy` (z vs Mk2 = 3.15). Questo indica bias del surrogate/binning o varianza finita, non necessariamente memoria reale oltre Mk2. +3. **`num_var_10` produce un falso positivo debole su iid.** `poisson_iid` viene letto Layer 1 per `num_var_10` con z = 2.13. Il segnale supera appena la soglia operativa, quindi va trattato come diagnostica secondaria finché non passa audit multi-seed. +4. **Il vecchio controllo `HL_markov` va rinominato nel linguaggio dei report.** Lo script non implementa Hardy-Littlewood; implementa un AR(1) lag1-matched. I claim futuri non devono chiamarlo controllo HL aritmetico. + +## Verdict +**CONSTRAINT on META + BOUNDARY**: la decomposizione two-layer è utilizzabile solo con perimetro atomico. + +Perimetro corretto: nei 60,000 gap primi testati con 20 surrogati Mk per livello, il nucleo `SR,L1` viene catturato da Mk1 e il nucleo `L2,SR2` da Mk2. Le diagnostiche `cond_entropy`, `num_var_10`, `run_length` non sono evidenza primaria finché non passano un recovery audit multi-seed e, idealmente, surrogati Markov esatti. + +Non è una scoperta nuova di teoria Markov: è un audit metodologico vicino ai test di ordine Markov con surrogate data. La letteratura rilevante include test esatti di ordine Markov e surrogate vincolati che preservano proprietà Markov di ordine n, quindi il prossimo strumento deve avvicinarsi a quei null esatti invece di usare solo generatori bin-pool approssimati. + +## Bicono della scoperta +- **Due radici**: layer osservato dai dati · ordine noto del generatore. +- **Singolare**: la pipeline Mk stessa, prima che venga usata come prova su un dominio reale. +- **Invariante di passaggio**: un claim di memoria sopravvive solo se recupera correttamente i controlli con ordine noto. +- **Campo di possibilità**: qui diventa possibile separare nucleo robusto e diagnostiche sovrastimate; qui diventa non-possibile usare `cond_entropy` o `num_var_10` come prova primaria di profondità senza recovery audit. + +## Self-Falsification — 5 lenti +- **L1 hard constraint vs bias**: nessun claim "zero/never/always". La frazione mod3 è 0.158032, dichiarata bias, non proibizione. +- **L2 quantità vs ratio**: il claim principale usa layer recuperato e z-score, non percentuali fra spazi di taglia diversa. +- **L3 no silent patching**: dichiaro esplicitamente che `HL_markov` era nome scorretto per un controllo AR(1) lag1-matched. +- **L4 edge cases**: i fallimenti `cond_entropy` e `num_var_10` non vengono arrotondati via; diventano il vincolo. +- **L5 rediscovery**: non taggo NEW. Riferimenti classici vicini: surrogate-data tests for Markov order e constrained Markov-order surrogates. + +## References +- Kandel et al.-lineage / surrogate tests: "Testing the order of discrete Markov chains using surrogate data", Physica D 117, 1998. +- "Exact significance test for Markov order", Physica D 269, 2014. +- "Constrained Markov order surrogates", Physica D 2020. + +## Files +- `tools/exp_markov_layer_recovery_audit.py` +- `tools/data/markov_layer_recovery_audit.json` +- `tools/data/meta_tautology_test.json` +- `tools/data/seme.json` +- `tools/data/reports/agent_20260504_1219.md` diff --git a/tools/exp_markov_layer_recovery_audit.py b/tools/exp_markov_layer_recovery_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..9797ea889603167a58cd989c71d02cf2aaf927bf --- /dev/null +++ b/tools/exp_markov_layer_recovery_audit.py @@ -0,0 +1,179 @@ +#!/usr/bin/env python3 +""" +exp_markov_layer_recovery_audit.py — META audit for the Markov-layer classifier. + +Question: + If a sequence is generated with known Markov order k, does the existing + Mk0/Mk1/Mk2 classifier recover layer <= k? + +Why this matters: + Recent reports used Mk surrogate capture to claim two visible memory layers. + This audit tests the opposite: whether the classifier itself creates apparent + higher layers on finite Markov controls. + +Controls: + - prime_gaps: unknown real target, reported for comparison only + - empirical_Mk0: shuffled prime gaps; known order 0 + - empirical_Mk1: prime-trained Mk1 surrogate; known order 1 + - empirical_Mk2: prime-trained Mk2 surrogate; known order 2 + - poisson_iid: independent exponential spacings; known order 0 + +Verdict rule: + A control fails recovery when any observable is assigned layer > known order. +""" + +import argparse +import json +from pathlib import Path + +import numpy as np + +from exp_two_layer_universality import ( + OBSERVABLES, + classify_layer, + gen_prime_gaps, + generate_markov_surrogate, +) + + +def measure_all(gaps): + out = {} + for name, fn in OBSERVABLES.items(): + try: + value = float(fn(gaps)) + except Exception: + value = float("nan") + out[name] = value + return out + + +def classifier_pass(gaps, n_surr, rng): + real_obs = measure_all(gaps) + z_scores = {} + + for mk in (0, 1, 2): + surr_obs = {name: [] for name in OBSERVABLES} + for _ in range(n_surr): + if mk == 0: + surr = rng.permutation(gaps) + else: + surr = generate_markov_surrogate(gaps, mk, rng=rng) + for obs_name, obs_fn in OBSERVABLES.items(): + try: + surr_obs[obs_name].append(float(obs_fn(surr))) + except Exception: + pass + + z_mk = {} + for obs_name in OBSERVABLES: + vals = np.asarray(surr_obs[obs_name], dtype=float) + vals = vals[np.isfinite(vals)] + if len(vals) > 2 and np.std(vals) > 1e-12 and np.isfinite(real_obs[obs_name]): + z = (real_obs[obs_name] - np.mean(vals)) / np.std(vals) + else: + z = 0.0 + z_mk[obs_name] = round(float(z), 2) + z_scores[f"Mk{mk}"] = z_mk + + layers = {} + for obs_name in OBSERVABLES: + layers[obs_name] = classify_layer( + z_scores["Mk0"][obs_name], + z_scores["Mk1"][obs_name], + z_scores["Mk2"][obs_name], + ) + + return real_obs, z_scores, layers + + +def build_controls(prime_gaps, rng): + return { + "prime_gaps": { + "known_order": None, + "gaps": prime_gaps, + }, + "empirical_Mk0": { + "known_order": 0, + "gaps": rng.permutation(prime_gaps), + }, + "empirical_Mk1": { + "known_order": 1, + "gaps": generate_markov_surrogate(prime_gaps, 1, rng=rng), + }, + "empirical_Mk2": { + "known_order": 2, + "gaps": generate_markov_surrogate(prime_gaps, 2, rng=rng), + }, + "poisson_iid": { + "known_order": 0, + "gaps": rng.exponential(1.0, len(prime_gaps)), + }, + } + + +def summarize_failure(layers, known_order): + if known_order is None: + return { + "status": "target_only", + "over_layer_observables": [], + "max_layer": int(max(layers.values())), + } + + over = [name for name, layer in layers.items() if layer > known_order] + return { + "status": "PASS" if not over else "FAIL", + "over_layer_observables": over, + "max_layer": int(max(layers.values())), + } + + +def run(N=60000, n_surr=20, seed=20260504): + rng = np.random.default_rng(seed) + prime_gaps = gen_prime_gaps(N).astype(float) + controls = build_controls(prime_gaps, rng) + + results = { + "N": int(N), + "n_surr": int(n_surr), + "seed": int(seed), + "controls": {}, + } + + print(f"N={N}, n_surr={n_surr}, seed={seed}") + print(f"{'sequence':<16} {'known':<7} {'status':<8} {'maxL':<5} over-layer observables") + print("-" * 82) + + for name, spec in controls.items(): + real_obs, z_scores, layers = classifier_pass(spec["gaps"], n_surr, rng) + summary = summarize_failure(layers, spec["known_order"]) + known = "target" if spec["known_order"] is None else f"L{spec['known_order']}" + over = ", ".join(summary["over_layer_observables"]) or "-" + print(f"{name:<16} {known:<7} {summary['status']:<8} L{summary['max_layer']:<4} {over}") + + results["controls"][name] = { + "known_order": spec["known_order"], + "n_gaps": int(len(spec["gaps"])), + "real_obs": {k: round(v, 6) for k, v in real_obs.items()}, + "z_scores": z_scores, + "layers": layers, + "recovery": summary, + } + + out_path = Path("tools/data/markov_layer_recovery_audit.json") + with out_path.open("w") as f: + json.dump(results, f, indent=2) + print(f"\nSaved to {out_path}") + return results + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--N", type=int, default=60000) + parser.add_argument("--n_surr", type=int, default=20) + parser.add_argument("--seed", type=int, default=20260504) + args = parser.parse_args() + run(N=args.N, n_surr=args.n_surr, seed=args.seed) + + +if __name__ == "__main__": + main() apply patch patch: completed /opt/MM_D-ND/tools/exp_markov_layer_recovery_audit.py diff --git a/tools/data/reports/agent_20260504_1219.md b/tools/data/reports/agent_20260504_1219.md new file mode 100644 index 0000000000000000000000000000000000000000..683dab4bbab940600d05e182ee1b696459ee180c --- /dev/null +++ b/tools/data/reports/agent_20260504_1219.md @@ -0,0 +1,104 @@ +# Agent Report — Markov Layers Pass the First Recovery Gate, but Not All Observables Pass +**Date**: 2026-05-04 12:19 +**Piano**: 61 +**Tension explored**: META (0.5) + +## Claim Under Test +> Tutti i 11 test passano — verifica che non stiamo testando solo tautologie. + +## Question +Se la pipeline Mk0/Mk1/Mk2 vede due layer, li recupera anche quando l'ordine vero della sequenza è noto? + +## Experiment Design +- **Primary audit**: `tools/exp_markov_layer_recovery_audit.py` +- **Scope**: 60,000 gap/spacings per sequenza; 20 surrogati per livello Mk; seed `20260504`. +- **Target**: prime gaps, ordine sconosciuto, solo confronto. +- **Known-order controls**: + - `empirical_Mk0`: permutazione dei gap primi, ordine noto 0. + - `empirical_Mk1`: surrogate Mk1 addestrato sui gap primi, ordine noto 1. + - `empirical_Mk2`: surrogate Mk2 addestrato sui gap primi, ordine noto 2. + - `poisson_iid`: spacings esponenziali iid, ordine noto 0. +- **Recovery rule**: un controllo passa se nessun osservabile viene classificato con layer maggiore dell'ordine noto. +- **Secondary calibration**: `tools/exp_meta_tautology_test.py --n-primes 300000 --n-trials 12`. Nota: il controllo chiamato `HL_markov` nello script è un AR(1) gap model che preserva lag-1, non un modello Hardy-Littlewood aritmetico. + +## Results + +### Recovery audit + +| Sequence | Known order | Recovery | Max layer | Over-layer observables | +|---|---:|---|---:|---| +| `prime_gaps` | target | target only | 3 | - | +| `empirical_Mk0` | 0 | PASS | 0 | - | +| `empirical_Mk1` | 1 | PASS | 1 | - | +| `empirical_Mk2` | 2 | FAIL | 3 | `cond_entropy` | +| `poisson_iid` | 0 | FAIL | 1 | `num_var_10` | + +Prime target layers, reported without recovery verdict: + +| Observable | Layer | z vs Mk0 | z vs Mk1 | z vs Mk2 | +|---|---:|---:|---:|---:| +| SR | 1 | -22.21 | -0.08 | -0.16 | +| L1 | 1 | -12.25 | 0.79 | 0.86 | +| L2 | 2 | -3.60 | -4.09 | -0.19 | +| SR2 | 2 | -6.67 | -8.80 | -0.33 | +| cond_entropy | 3 | -435.40 | -26.11 | 2.56 | +| triple_var | 1 | -15.90 | -0.53 | 0.36 | +| num_var_10 | 1 | -7.46 | -1.42 | 0.00 | +| run_length | 3 | 27.59 | 7.19 | 3.43 | + +Control failure details: + +| Control | Observable | Expected max | Assigned | Diagnostic z | +|---|---|---:|---:|---:| +| `empirical_Mk2` | `cond_entropy` | 2 | 3 | z vs Mk2 = 3.15 | +| `poisson_iid` | `num_var_10` | 0 | 1 | z vs Mk0 = 2.13 | + +### Secondary tautology calibration + +| Observable | Verdict in tested controls | Blocking control | +|---|---|---| +| r-stat | STRUCTURAL | none in this run | +| mod3 fraction | STRUCTURAL | none in this run | +| lag1 ACF | TAUTOLOGICAL | lag1-matched AR(1), z = -0.62 | +| dr_ord | TAUTOLOGICAL | lag1-matched AR(1), z = -1.90 | +| dr_mag | STRUCTURAL | none in this run | + +Numeri principali: prime mod3 same-class fraction = 0.158032. Questo è bias forte rispetto ai controlli ~0.34-0.35, non proibizione zero. + +## Key Findings +1. **Il nucleo pair/triple non cade al primo audit.** `empirical_Mk0` recupera solo Layer 0; `empirical_Mk1` recupera solo Layer 1. Questo supporta l'uso di SR e L1 come Layer 1 e di L2/SR2 come Layer 2 nel perimetro testato. +2. **`cond_entropy` non può essere usata come prova forte di Layer 3 nel setup attuale.** Anche un controllo generato come Mk2 viene letto Layer 3 per `cond_entropy` (z vs Mk2 = 3.15). Questo indica bias del surrogate/binning o varianza finita, non necessariamente memoria reale oltre Mk2. +3. **`num_var_10` produce un falso positivo debole su iid.** `poisson_iid` viene letto Layer 1 per `num_var_10` con z = 2.13. Il segnale supera appena la soglia operativa, quindi va trattato come diagnostica secondaria finché non passa audit multi-seed. +4. **Il vecchio controllo `HL_markov` va rinominato nel linguaggio dei report.** Lo script non implementa Hardy-Littlewood; implementa un AR(1) lag1-matched. I claim futuri non devono chiamarlo controllo HL aritmetico. + +## Verdict +**CONSTRAINT on META + BOUNDARY**: la decomposizione two-layer è utilizzabile solo con perimetro atomico. + +Perimetro corretto: nei 60,000 gap primi testati con 20 surrogati Mk per livello, il nucleo `SR,L1` viene catturato da Mk1 e il nucleo `L2,SR2` da Mk2. Le diagnostiche `cond_entropy`, `num_var_10`, `run_length` non sono evidenza primaria finché non passano un recovery audit multi-seed e, idealmente, surrogati Markov esatti. + +Non è una scoperta nuova di teoria Markov: è un audit metodologico vicino ai test di ordine Markov con surrogate data. La letteratura rilevante include test esatti di ordine Markov e surrogate vincolati che preservano proprietà Markov di ordine n, quindi il prossimo strumento deve avvicinarsi a quei null esatti invece di usare solo generatori bin-pool approssimati. + +## Bicono della scoperta +- **Due radici**: layer osservato dai dati · ordine noto del generatore. +- **Singolare**: la pipeline Mk stessa, prima che venga usata come prova su un dominio reale. +- **Invariante di passaggio**: un claim di memoria sopravvive solo se recupera correttamente i controlli con ordine noto. +- **Campo di possibilità**: qui diventa possibile separare nucleo robusto e diagnostiche sovrastimate; qui diventa non-possibile usare `cond_entropy` o `num_var_10` come prova primaria di profondità senza recovery audit. + +## Self-Falsification — 5 lenti +- **L1 hard constraint vs bias**: nessun claim "zero/never/always". La frazione mod3 è 0.158032, dichiarata bias, non proibizione. +- **L2 quantità vs ratio**: il claim principale usa layer recuperato e z-score, non percentuali fra spazi di taglia diversa. +- **L3 no silent patching**: dichiaro esplicitamente che `HL_markov` era nome scorretto per un controllo AR(1) lag1-matched. +- **L4 edge cases**: i fallimenti `cond_entropy` e `num_var_10` non vengono arrotondati via; diventano il vincolo. +- **L5 rediscovery**: non taggo NEW. Riferimenti classici vicini: surrogate-data tests for Markov order e constrained Markov-order surrogates. + +## References +- Kandel et al.-lineage / surrogate tests: "Testing the order of discrete Markov chains using surrogate data", Physica D 117, 1998. +- "Exact significance test for Markov order", Physica D 269, 2014. +- "Constrained Markov order surrogates", Physica D 2020. + +## Files +- `tools/exp_markov_layer_recovery_audit.py` +- `tools/data/markov_layer_recovery_audit.json` +- `tools/data/meta_tautology_test.json` +- `tools/data/seme.json` +- `tools/data/reports/agent_20260504_1219.md` diff --git a/tools/exp_markov_layer_recovery_audit.py b/tools/exp_markov_layer_recovery_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..4faa72c630148cdf498c3cc2d626cca2a78efd9a --- /dev/null +++ b/tools/exp_markov_layer_recovery_audit.py @@ -0,0 +1,179 @@ +#!/usr/bin/env python3 +""" +exp_markov_layer_recovery_audit.py - META audit for the Markov-layer classifier. + +Question: + If a sequence is generated with known Markov order k, does the existing + Mk0/Mk1/Mk2 classifier recover layer <= k? + +Why this matters: + Recent reports used Mk surrogate capture to claim two visible memory layers. + This audit tests the opposite: whether the classifier itself creates apparent + higher layers on finite Markov controls. + +Controls: + - prime_gaps: unknown real target, reported for comparison only + - empirical_Mk0: shuffled prime gaps; known order 0 + - empirical_Mk1: prime-trained Mk1 surrogate; known order 1 + - empirical_Mk2: prime-trained Mk2 surrogate; known order 2 + - poisson_iid: independent exponential spacings; known order 0 + +Verdict rule: + A control fails recovery when any observable is assigned layer > known order. +""" + +import argparse +import json +from pathlib import Path + +import numpy as np + +from exp_two_layer_universality import ( + OBSERVABLES, + classify_layer, + gen_prime_gaps, + generate_markov_surrogate, +) + + +def measure_all(gaps): + out = {} + for name, fn in OBSERVABLES.items(): + try: + value = float(fn(gaps)) + except Exception: + value = float("nan") + out[name] = value + return out + + +def classifier_pass(gaps, n_surr, rng): + real_obs = measure_all(gaps) + z_scores = {} + + for mk in (0, 1, 2): + surr_obs = {name: [] for name in OBSERVABLES} + for _ in range(n_surr): + if mk == 0: + surr = rng.permutation(gaps) + else: + surr = generate_markov_surrogate(gaps, mk, rng=rng) + for obs_name, obs_fn in OBSERVABLES.items(): + try: + surr_obs[obs_name].append(float(obs_fn(surr))) + except Exception: + pass + + z_mk = {} + for obs_name in OBSERVABLES: + vals = np.asarray(surr_obs[obs_name], dtype=float) + vals = vals[np.isfinite(vals)] + if len(vals) > 2 and np.std(vals) > 1e-12 and np.isfinite(real_obs[obs_name]): + z = (real_obs[obs_name] - np.mean(vals)) / np.std(vals) + else: + z = 0.0 + z_mk[obs_name] = round(float(z), 2) + z_scores[f"Mk{mk}"] = z_mk + + layers = {} + for obs_name in OBSERVABLES: + layers[obs_name] = classify_layer( + z_scores["Mk0"][obs_name], + z_scores["Mk1"][obs_name], + z_scores["Mk2"][obs_name], + ) + + return real_obs, z_scores, layers + + +def build_controls(prime_gaps, rng): + return { + "prime_gaps": { + "known_order": None, + "gaps": prime_gaps, + }, + "empirical_Mk0": { + "known_order": 0, + "gaps": rng.permutation(prime_gaps), + }, + "empirical_Mk1": { + "known_order": 1, + "gaps": generate_markov_surrogate(prime_gaps, 1, rng=rng), + }, + "empirical_Mk2": { + "known_order": 2, + "gaps": generate_markov_surrogate(prime_gaps, 2, rng=rng), + }, + "poisson_iid": { + "known_order": 0, + "gaps": rng.exponential(1.0, len(prime_gaps)), + }, + } + + +def summarize_failure(layers, known_order): + if known_order is None: + return { + "status": "target_only", + "over_layer_observables": [], + "max_layer": int(max(layers.values())), + } + + over = [name for name, layer in layers.items() if layer > known_order] + return { + "status": "PASS" if not over else "FAIL", + "over_layer_observables": over, + "max_layer": int(max(layers.values())), + } + + +def run(N=60000, n_surr=20, seed=20260504): + rng = np.random.default_rng(seed) + prime_gaps = gen_prime_gaps(N).astype(float) + controls = build_controls(prime_gaps, rng) + + results = { + "N": int(N), + "n_surr": int(n_surr), + "seed": int(seed), + "controls": {}, + } + + print(f"N={N}, n_surr={n_surr}, seed={seed}") + print(f"{'sequence':<16} {'known':<7} {'status':<8} {'maxL':<5} over-layer observables") + print("-" * 82) + + for name, spec in controls.items(): + real_obs, z_scores, layers = classifier_pass(spec["gaps"], n_surr, rng) + summary = summarize_failure(layers, spec["known_order"]) + known = "target" if spec["known_order"] is None else f"L{spec['known_order']}" + over = ", ".join(summary["over_layer_observables"]) or "-" + print(f"{name:<16} {known:<7} {summary['status']:<8} L{summary['max_layer']:<4} {over}") + + results["controls"][name] = { + "known_order": spec["known_order"], + "n_gaps": int(len(spec["gaps"])), + "real_obs": {k: round(v, 6) for k, v in real_obs.items()}, + "z_scores": z_scores, + "layers": layers, + "recovery": summary, + } + + out_path = Path("tools/data/markov_layer_recovery_audit.json") + with out_path.open("w") as f: + json.dump(results, f, indent=2) + print(f"\nSaved to {out_path}") + return results + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--N", type=int, default=60000) + parser.add_argument("--n_surr", type=int, default=20) + parser.add_argument("--seed", type=int, default=20260504) + args = parser.parse_args() + run(N=args.N, n_surr=args.n_surr, seed=args.seed) + + +if __name__ == "__main__": + main() diff --git a/tools/data/reports/agent_20260504_1219.md b/tools/data/reports/agent_20260504_1219.md new file mode 100644 index 0000000000000000000000000000000000000000..683dab4bbab940600d05e182ee1b696459ee180c --- /dev/null +++ b/tools/data/reports/agent_20260504_1219.md @@ -0,0 +1,104 @@ +# Agent Report — Markov Layers Pass the First Recovery Gate, but Not All Observables Pass +**Date**: 2026-05-04 12:19 +**Piano**: 61 +**Tension explored**: META (0.5) + +## Claim Under Test +> Tutti i 11 test passano — verifica che non stiamo testando solo tautologie. + +## Question +Se la pipeline Mk0/Mk1/Mk2 vede due layer, li recupera anche quando l'ordine vero della sequenza è noto? + +## Experiment Design +- **Primary audit**: `tools/exp_markov_layer_recovery_audit.py` +- **Scope**: 60,000 gap/spacings per sequenza; 20 surrogati per livello Mk; seed `20260504`. +- **Target**: prime gaps, ordine sconosciuto, solo confronto. +- **Known-order controls**: + - `empirical_Mk0`: permutazione dei gap primi, ordine noto 0. + - `empirical_Mk1`: surrogate Mk1 addestrato sui gap primi, ordine noto 1. + - `empirical_Mk2`: surrogate Mk2 addestrato sui gap primi, ordine noto 2. + - `poisson_iid`: spacings esponenziali iid, ordine noto 0. +- **Recovery rule**: un controllo passa se nessun osservabile viene classificato con layer maggiore dell'ordine noto. +- **Secondary calibration**: `tools/exp_meta_tautology_test.py --n-primes 300000 --n-trials 12`. Nota: il controllo chiamato `HL_markov` nello script è un AR(1) gap model che preserva lag-1, non un modello Hardy-Littlewood aritmetico. + +## Results + +### Recovery audit + +| Sequence | Known order | Recovery | Max layer | Over-layer observables | +|---|---:|---|---:|---| +| `prime_gaps` | target | target only | 3 | - | +| `empirical_Mk0` | 0 | PASS | 0 | - | +| `empirical_Mk1` | 1 | PASS | 1 | - | +| `empirical_Mk2` | 2 | FAIL | 3 | `cond_entropy` | +| `poisson_iid` | 0 | FAIL | 1 | `num_var_10` | + +Prime target layers, reported without recovery verdict: + +| Observable | Layer | z vs Mk0 | z vs Mk1 | z vs Mk2 | +|---|---:|---:|---:|---:| +| SR | 1 | -22.21 | -0.08 | -0.16 | +| L1 | 1 | -12.25 | 0.79 | 0.86 | +| L2 | 2 | -3.60 | -4.09 | -0.19 | +| SR2 | 2 | -6.67 | -8.80 | -0.33 | +| cond_entropy | 3 | -435.40 | -26.11 | 2.56 | +| triple_var | 1 | -15.90 | -0.53 | 0.36 | +| num_var_10 | 1 | -7.46 | -1.42 | 0.00 | +| run_length | 3 | 27.59 | 7.19 | 3.43 | + +Control failure details: + +| Control | Observable | Expected max | Assigned | Diagnostic z | +|---|---|---:|---:|---:| +| `empirical_Mk2` | `cond_entropy` | 2 | 3 | z vs Mk2 = 3.15 | +| `poisson_iid` | `num_var_10` | 0 | 1 | z vs Mk0 = 2.13 | + +### Secondary tautology calibration + +| Observable | Verdict in tested controls | Blocking control | +|---|---|---| +| r-stat | STRUCTURAL | none in this run | +| mod3 fraction | STRUCTURAL | none in this run | +| lag1 ACF | TAUTOLOGICAL | lag1-matched AR(1), z = -0.62 | +| dr_ord | TAUTOLOGICAL | lag1-matched AR(1), z = -1.90 | +| dr_mag | STRUCTURAL | none in this run | + +Numeri principali: prime mod3 same-class fraction = 0.158032. Questo è bias forte rispetto ai controlli ~0.34-0.35, non proibizione zero. + +## Key Findings +1. **Il nucleo pair/triple non cade al primo audit.** `empirical_Mk0` recupera solo Layer 0; `empirical_Mk1` recupera solo Layer 1. Questo supporta l'uso di SR e L1 come Layer 1 e di L2/SR2 come Layer 2 nel perimetro testato. +2. **`cond_entropy` non può essere usata come prova forte di Layer 3 nel setup attuale.** Anche un controllo generato come Mk2 viene letto Layer 3 per `cond_entropy` (z vs Mk2 = 3.15). Questo indica bias del surrogate/binning o varianza finita, non necessariamente memoria reale oltre Mk2. +3. **`num_var_10` produce un falso positivo debole su iid.** `poisson_iid` viene letto Layer 1 per `num_var_10` con z = 2.13. Il segnale supera appena la soglia operativa, quindi va trattato come diagnostica secondaria finché non passa audit multi-seed. +4. **Il vecchio controllo `HL_markov` va rinominato nel linguaggio dei report.** Lo script non implementa Hardy-Littlewood; implementa un AR(1) lag1-matched. I claim futuri non devono chiamarlo controllo HL aritmetico. + +## Verdict +**CONSTRAINT on META + BOUNDARY**: la decomposizione two-layer è utilizzabile solo con perimetro atomico. + +Perimetro corretto: nei 60,000 gap primi testati con 20 surrogati Mk per livello, il nucleo `SR,L1` viene catturato da Mk1 e il nucleo `L2,SR2` da Mk2. Le diagnostiche `cond_entropy`, `num_var_10`, `run_length` non sono evidenza primaria finché non passano un recovery audit multi-seed e, idealmente, surrogati Markov esatti. + +Non è una scoperta nuova di teoria Markov: è un audit metodologico vicino ai test di ordine Markov con surrogate data. La letteratura rilevante include test esatti di ordine Markov e surrogate vincolati che preservano proprietà Markov di ordine n, quindi il prossimo strumento deve avvicinarsi a quei null esatti invece di usare solo generatori bin-pool approssimati. + +## Bicono della scoperta +- **Due radici**: layer osservato dai dati · ordine noto del generatore. +- **Singolare**: la pipeline Mk stessa, prima che venga usata come prova su un dominio reale. +- **Invariante di passaggio**: un claim di memoria sopravvive solo se recupera correttamente i controlli con ordine noto. +- **Campo di possibilità**: qui diventa possibile separare nucleo robusto e diagnostiche sovrastimate; qui diventa non-possibile usare `cond_entropy` o `num_var_10` come prova primaria di profondità senza recovery audit. + +## Self-Falsification — 5 lenti +- **L1 hard constraint vs bias**: nessun claim "zero/never/always". La frazione mod3 è 0.158032, dichiarata bias, non proibizione. +- **L2 quantità vs ratio**: il claim principale usa layer recuperato e z-score, non percentuali fra spazi di taglia diversa. +- **L3 no silent patching**: dichiaro esplicitamente che `HL_markov` era nome scorretto per un controllo AR(1) lag1-matched. +- **L4 edge cases**: i fallimenti `cond_entropy` e `num_var_10` non vengono arrotondati via; diventano il vincolo. +- **L5 rediscovery**: non taggo NEW. Riferimenti classici vicini: surrogate-data tests for Markov order e constrained Markov-order surrogates. + +## References +- Kandel et al.-lineage / surrogate tests: "Testing the order of discrete Markov chains using surrogate data", Physica D 117, 1998. +- "Exact significance test for Markov order", Physica D 269, 2014. +- "Constrained Markov order surrogates", Physica D 2020. + +## Files +- `tools/exp_markov_layer_recovery_audit.py` +- `tools/data/markov_layer_recovery_audit.json` +- `tools/data/meta_tautology_test.json` +- `tools/data/seme.json` +- `tools/data/reports/agent_20260504_1219.md` diff --git a/tools/exp_markov_layer_recovery_audit.py b/tools/exp_markov_layer_recovery_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..4faa72c630148cdf498c3cc2d626cca2a78efd9a --- /dev/null +++ b/tools/exp_markov_layer_recovery_audit.py @@ -0,0 +1,179 @@ +#!/usr/bin/env python3 +""" +exp_markov_layer_recovery_audit.py - META audit for the Markov-layer classifier. + +Question: + If a sequence is generated with known Markov order k, does the existing + Mk0/Mk1/Mk2 classifier recover layer <= k? + +Why this matters: + Recent reports used Mk surrogate capture to claim two visible memory layers. + This audit tests the opposite: whether the classifier itself creates apparent + higher layers on finite Markov controls. + +Controls: + - prime_gaps: unknown real target, reported for comparison only + - empirical_Mk0: shuffled prime gaps; known order 0 + - empirical_Mk1: prime-trained Mk1 surrogate; known order 1 + - empirical_Mk2: prime-trained Mk2 surrogate; known order 2 + - poisson_iid: independent exponential spacings; known order 0 + +Verdict rule: + A control fails recovery when any observable is assigned layer > known order. +""" + +import argparse +import json +from pathlib import Path + +import numpy as np + +from exp_two_layer_universality import ( + OBSERVABLES, + classify_layer, + gen_prime_gaps, + generate_markov_surrogate, +) + + +def measure_all(gaps): + out = {} + for name, fn in OBSERVABLES.items(): + try: + value = float(fn(gaps)) + except Exception: + value = float("nan") + out[name] = value + return out + + +def classifier_pass(gaps, n_surr, rng): + real_obs = measure_all(gaps) + z_scores = {} + + for mk in (0, 1, 2): + surr_obs = {name: [] for name in OBSERVABLES} + for _ in range(n_surr): + if mk == 0: + surr = rng.permutation(gaps) + else: + surr = generate_markov_surrogate(gaps, mk, rng=rng) + for obs_name, obs_fn in OBSERVABLES.items(): + try: + surr_obs[obs_name].append(float(obs_fn(surr))) + except Exception: + pass + + z_mk = {} + for obs_name in OBSERVABLES: + vals = np.asarray(surr_obs[obs_name], dtype=float) + vals = vals[np.isfinite(vals)] + if len(vals) > 2 and np.std(vals) > 1e-12 and np.isfinite(real_obs[obs_name]): + z = (real_obs[obs_name] - np.mean(vals)) / np.std(vals) + else: + z = 0.0 + z_mk[obs_name] = round(float(z), 2) + z_scores[f"Mk{mk}"] = z_mk + + layers = {} + for obs_name in OBSERVABLES: + layers[obs_name] = classify_layer( + z_scores["Mk0"][obs_name], + z_scores["Mk1"][obs_name], + z_scores["Mk2"][obs_name], + ) + + return real_obs, z_scores, layers + + +def build_controls(prime_gaps, rng): + return { + "prime_gaps": { + "known_order": None, + "gaps": prime_gaps, + }, + "empirical_Mk0": { + "known_order": 0, + "gaps": rng.permutation(prime_gaps), + }, + "empirical_Mk1": { + "known_order": 1, + "gaps": generate_markov_surrogate(prime_gaps, 1, rng=rng), + }, + "empirical_Mk2": { + "known_order": 2, + "gaps": generate_markov_surrogate(prime_gaps, 2, rng=rng), + }, + "poisson_iid": { + "known_order": 0, + "gaps": rng.exponential(1.0, len(prime_gaps)), + }, + } + + +def summarize_failure(layers, known_order): + if known_order is None: + return { + "status": "target_only", + "over_layer_observables": [], + "max_layer": int(max(layers.values())), + } + + over = [name for name, layer in layers.items() if layer > known_order] + return { + "status": "PASS" if not over else "FAIL", + "over_layer_observables": over, + "max_layer": int(max(layers.values())), + } + + +def run(N=60000, n_surr=20, seed=20260504): + rng = np.random.default_rng(seed) + prime_gaps = gen_prime_gaps(N).astype(float) + controls = build_controls(prime_gaps, rng) + + results = { + "N": int(N), + "n_surr": int(n_surr), + "seed": int(seed), + "controls": {}, + } + + print(f"N={N}, n_surr={n_surr}, seed={seed}") + print(f"{'sequence':<16} {'known':<7} {'status':<8} {'maxL':<5} over-layer observables") + print("-" * 82) + + for name, spec in controls.items(): + real_obs, z_scores, layers = classifier_pass(spec["gaps"], n_surr, rng) + summary = summarize_failure(layers, spec["known_order"]) + known = "target" if spec["known_order"] is None else f"L{spec['known_order']}" + over = ", ".join(summary["over_layer_observables"]) or "-" + print(f"{name:<16} {known:<7} {summary['status']:<8} L{summary['max_layer']:<4} {over}") + + results["controls"][name] = { + "known_order": spec["known_order"], + "n_gaps": int(len(spec["gaps"])), + "real_obs": {k: round(v, 6) for k, v in real_obs.items()}, + "z_scores": z_scores, + "layers": layers, + "recovery": summary, + } + + out_path = Path("tools/data/markov_layer_recovery_audit.json") + with out_path.open("w") as f: + json.dump(results, f, indent=2) + print(f"\nSaved to {out_path}") + return results + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--N", type=int, default=60000) + parser.add_argument("--n_surr", type=int, default=20) + parser.add_argument("--seed", type=int, default=20260504) + args = parser.parse_args() + run(N=args.N, n_surr=args.n_surr, seed=args.seed) + + +if __name__ == "__main__": + main() apply patch patch: completed /opt/MM_D-ND/tools/data/reports/agent_20260504_1219.md diff --git a/tools/data/reports/agent_20260504_1219.md b/tools/data/reports/agent_20260504_1219.md new file mode 100644 index 0000000000000000000000000000000000000000..b3c10211f3b81595e76259db054509f891c79118 --- /dev/null +++ b/tools/data/reports/agent_20260504_1219.md @@ -0,0 +1,104 @@ +# Agent Report — Markov Layers Pass the First Recovery Gate, but Not All Observables Pass +**Date**: 2026-05-04 12:19 +**Piano**: 61 +**Tension explored**: META (0.5) + +## Claim Under Test +> Tutti i 11 test passano — verifica che non stiamo testando solo tautologie. + +## Question +Se la pipeline Mk0/Mk1/Mk2 vede due layer, li recupera anche quando l'ordine vero della sequenza è noto? + +## Experiment Design +- **Primary audit**: `tools/exp_markov_layer_recovery_audit.py` +- **Scope**: 60,000 gap/spacings per sequenza; 20 surrogati per livello Mk; seed `20260504`. +- **Target**: prime gaps, ordine sconosciuto, solo confronto. +- **Known-order controls**: + - `empirical_Mk0`: permutazione dei gap primi, ordine noto 0. + - `empirical_Mk1`: surrogate Mk1 addestrato sui gap primi, ordine noto 1. + - `empirical_Mk2`: surrogate Mk2 addestrato sui gap primi, ordine noto 2. + - `poisson_iid`: spacings esponenziali iid, ordine noto 0. +- **Recovery rule**: un controllo passa se nessun osservabile viene classificato con layer maggiore dell'ordine noto. +- **Secondary calibration**: `tools/exp_meta_tautology_test.py --n-primes 300000 --n-trials 12`. Nota: il controllo chiamato `HL_markov` nello script è un AR(1) gap model che preserva lag-1, non un modello Hardy-Littlewood aritmetico. + +## Results + +### Recovery audit + +| Sequence | Known order | Recovery | Max layer | Over-layer observables | +|---|---:|---|---:|---| +| `prime_gaps` | target | target only | 3 | - | +| `empirical_Mk0` | 0 | PASS | 0 | - | +| `empirical_Mk1` | 1 | PASS | 1 | - | +| `empirical_Mk2` | 2 | FAIL | 3 | `cond_entropy` | +| `poisson_iid` | 0 | FAIL | 1 | `num_var_10` | + +Prime target layers, reported without recovery verdict: + +| Observable | Layer | z vs Mk0 | z vs Mk1 | z vs Mk2 | +|---|---:|---:|---:|---:| +| SR | 1 | -22.21 | -0.08 | -0.16 | +| L1 | 1 | -12.25 | 0.79 | 0.86 | +| L2 | 2 | -3.60 | -4.09 | -0.19 | +| SR2 | 2 | -6.67 | -8.80 | -0.33 | +| cond_entropy | 3 | -435.40 | -26.11 | 2.56 | +| triple_var | 1 | -15.90 | -0.53 | 0.36 | +| num_var_10 | 1 | -7.46 | -1.42 | 0.00 | +| run_length | 3 | 27.59 | 7.19 | 3.43 | + +Control failure details: + +| Control | Observable | Expected max | Assigned | Diagnostic z | +|---|---|---:|---:|---:| +| `empirical_Mk2` | `cond_entropy` | 2 | 3 | z vs Mk2 = 3.15 | +| `poisson_iid` | `num_var_10` | 0 | 1 | z vs Mk0 = 2.13 | + +### Secondary tautology calibration + +| Observable | Verdict in tested controls | Blocking control | +|---|---|---| +| r-stat | STRUCTURAL | none in this run | +| mod3 fraction | STRUCTURAL | none in this run | +| lag1 ACF | TAUTOLOGICAL | lag1-matched AR(1), z = -0.62 | +| dr_ord | TAUTOLOGICAL | lag1-matched AR(1), z = -1.90 | +| dr_mag | STRUCTURAL | none in this run | + +Numeri principali: prime mod3 same-class fraction = 0.158032. Questo è bias forte rispetto ai controlli ~0.34-0.35, non proibizione zero. + +## Key Findings +1. **Il nucleo pair/triple non cade al primo audit.** `empirical_Mk0` recupera solo Layer 0; `empirical_Mk1` recupera solo Layer 1. Questo supporta l'uso di SR e L1 come Layer 1 e di L2/SR2 come Layer 2 nel perimetro testato. +2. **`cond_entropy` non può essere usata come prova forte di Layer 3 nel setup attuale.** Anche un controllo generato come Mk2 viene letto Layer 3 per `cond_entropy` (z vs Mk2 = 3.15). Questo indica bias del surrogate/binning o varianza finita, non necessariamente memoria reale oltre Mk2. +3. **`num_var_10` produce un falso positivo debole su iid.** `poisson_iid` viene letto Layer 1 per `num_var_10` con z = 2.13. Il segnale supera appena la soglia operativa, quindi va trattato come diagnostica secondaria finché non passa audit multi-seed. +4. **Il vecchio controllo `HL_markov` va rinominato nel linguaggio dei report.** Lo script non implementa Hardy-Littlewood; implementa un AR(1) lag1-matched. I claim futuri non devono chiamarlo controllo HL aritmetico. + +## Verdict +**CONSTRAINT on META + BOUNDARY**: la decomposizione two-layer è utilizzabile solo con perimetro atomico. + +Perimetro corretto: nei 60,000 gap primi testati con 20 surrogati Mk per livello, il nucleo `SR,L1` viene catturato da Mk1 e il nucleo `L2,SR2` da Mk2. Le diagnostiche `cond_entropy`, `num_var_10`, `run_length` non sono evidenza primaria finché non passano un recovery audit multi-seed e, idealmente, surrogati Markov esatti. + +Non è una scoperta nuova di teoria Markov: è un audit metodologico vicino ai test di ordine Markov con surrogate data. La letteratura rilevante include test esatti di ordine Markov e surrogate vincolati che preservano proprietà Markov di ordine n, quindi il prossimo strumento deve avvicinarsi a quei null esatti invece di usare solo generatori bin-pool approssimati. + +## Bicono della scoperta +- **Due radici**: layer osservato dai dati · ordine noto del generatore. +- **Singolare**: la pipeline Mk stessa, prima che venga usata come prova su un dominio reale. +- **Invariante di passaggio**: un claim di memoria sopravvive solo se recupera correttamente i controlli con ordine noto. +- **Campo di possibilità**: qui diventa possibile separare nucleo robusto e diagnostiche sovrastimate; qui diventa non-possibile usare `cond_entropy` o `num_var_10` come prova primaria di profondità senza recovery audit. + +## Self-Falsification — 5 lenti +- **L1 hard constraint vs bias**: nessun claim "zero/never/always". La frazione mod3 è 0.158032, dichiarata bias, non proibizione. +- **L2 quantità vs ratio**: il claim principale usa layer recuperato e z-score, non percentuali fra spazi di taglia diversa. +- **L3 no silent patching**: dichiaro esplicitamente che `HL_markov` era nome scorretto per un controllo AR(1) lag1-matched. +- **L4 edge cases**: i fallimenti `cond_entropy` e `num_var_10` non vengono arrotondati via; diventano il vincolo. +- **L5 rediscovery**: non taggo NEW. Riferimenti classici vicini: surrogate-data tests for Markov order e constrained Markov-order surrogates. + +## References +- Kandel et al.-lineage / surrogate tests: ["Testing the order of discrete Markov chains using surrogate data", Physica D 117, 1998](https://www.sciencedirect.com/science/article/pii/S0167278997003151). +- ["Exact significance test for Markov order", Physica D 269, 2014](https://www.sciencedirect.com/science/article/pii/S0167278913003230). +- ["Constrained Markov order surrogates", Physica D 2020](https://www.sciencedirect.com/science/article/pii/S0167278919305652). + +## Files +- `tools/exp_markov_layer_recovery_audit.py` +- `tools/data/markov_layer_recovery_audit.json` +- `tools/data/meta_tautology_test.json` +- `tools/data/seme.json` +- `tools/data/reports/agent_20260504_1219.md` diff --git a/tools/exp_markov_layer_recovery_audit.py b/tools/exp_markov_layer_recovery_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..4faa72c630148cdf498c3cc2d626cca2a78efd9a --- /dev/null +++ b/tools/exp_markov_layer_recovery_audit.py @@ -0,0 +1,179 @@ +#!/usr/bin/env python3 +""" +exp_markov_layer_recovery_audit.py - META audit for the Markov-layer classifier. + +Question: + If a sequence is generated with known Markov order k, does the existing + Mk0/Mk1/Mk2 classifier recover layer <= k? + +Why this matters: + Recent reports used Mk surrogate capture to claim two visible memory layers. + This audit tests the opposite: whether the classifier itself creates apparent + higher layers on finite Markov controls. + +Controls: + - prime_gaps: unknown real target, reported for comparison only + - empirical_Mk0: shuffled prime gaps; known order 0 + - empirical_Mk1: prime-trained Mk1 surrogate; known order 1 + - empirical_Mk2: prime-trained Mk2 surrogate; known order 2 + - poisson_iid: independent exponential spacings; known order 0 + +Verdict rule: + A control fails recovery when any observable is assigned layer > known order. +""" + +import argparse +import json +from pathlib import Path + +import numpy as np + +from exp_two_layer_universality import ( + OBSERVABLES, + classify_layer, + gen_prime_gaps, + generate_markov_surrogate, +) + + +def measure_all(gaps): + out = {} + for name, fn in OBSERVABLES.items(): + try: + value = float(fn(gaps)) + except Exception: + value = float("nan") + out[name] = value + return out + + +def classifier_pass(gaps, n_surr, rng): + real_obs = measure_all(gaps) + z_scores = {} + + for mk in (0, 1, 2): + surr_obs = {name: [] for name in OBSERVABLES} + for _ in range(n_surr): + if mk == 0: + surr = rng.permutation(gaps) + else: + surr = generate_markov_surrogate(gaps, mk, rng=rng) + for obs_name, obs_fn in OBSERVABLES.items(): + try: + surr_obs[obs_name].append(float(obs_fn(surr))) + except Exception: + pass + + z_mk = {} + for obs_name in OBSERVABLES: + vals = np.asarray(surr_obs[obs_name], dtype=float) + vals = vals[np.isfinite(vals)] + if len(vals) > 2 and np.std(vals) > 1e-12 and np.isfinite(real_obs[obs_name]): + z = (real_obs[obs_name] - np.mean(vals)) / np.std(vals) + else: + z = 0.0 + z_mk[obs_name] = round(float(z), 2) + z_scores[f"Mk{mk}"] = z_mk + + layers = {} + for obs_name in OBSERVABLES: + layers[obs_name] = classify_layer( + z_scores["Mk0"][obs_name], + z_scores["Mk1"][obs_name], + z_scores["Mk2"][obs_name], + ) + + return real_obs, z_scores, layers + + +def build_controls(prime_gaps, rng): + return { + "prime_gaps": { + "known_order": None, + "gaps": prime_gaps, + }, + "empirical_Mk0": { + "known_order": 0, + "gaps": rng.permutation(prime_gaps), + }, + "empirical_Mk1": { + "known_order": 1, + "gaps": generate_markov_surrogate(prime_gaps, 1, rng=rng), + }, + "empirical_Mk2": { + "known_order": 2, + "gaps": generate_markov_surrogate(prime_gaps, 2, rng=rng), + }, + "poisson_iid": { + "known_order": 0, + "gaps": rng.exponential(1.0, len(prime_gaps)), + }, + } + + +def summarize_failure(layers, known_order): + if known_order is None: + return { + "status": "target_only", + "over_layer_observables": [], + "max_layer": int(max(layers.values())), + } + + over = [name for name, layer in layers.items() if layer > known_order] + return { + "status": "PASS" if not over else "FAIL", + "over_layer_observables": over, + "max_layer": int(max(layers.values())), + } + + +def run(N=60000, n_surr=20, seed=20260504): + rng = np.random.default_rng(seed) + prime_gaps = gen_prime_gaps(N).astype(float) + controls = build_controls(prime_gaps, rng) + + results = { + "N": int(N), + "n_surr": int(n_surr), + "seed": int(seed), + "controls": {}, + } + + print(f"N={N}, n_surr={n_surr}, seed={seed}") + print(f"{'sequence':<16} {'known':<7} {'status':<8} {'maxL':<5} over-layer observables") + print("-" * 82) + + for name, spec in controls.items(): + real_obs, z_scores, layers = classifier_pass(spec["gaps"], n_surr, rng) + summary = summarize_failure(layers, spec["known_order"]) + known = "target" if spec["known_order"] is None else f"L{spec['known_order']}" + over = ", ".join(summary["over_layer_observables"]) or "-" + print(f"{name:<16} {known:<7} {summary['status']:<8} L{summary['max_layer']:<4} {over}") + + results["controls"][name] = { + "known_order": spec["known_order"], + "n_gaps": int(len(spec["gaps"])), + "real_obs": {k: round(v, 6) for k, v in real_obs.items()}, + "z_scores": z_scores, + "layers": layers, + "recovery": summary, + } + + out_path = Path("tools/data/markov_layer_recovery_audit.json") + with out_path.open("w") as f: + json.dump(results, f, indent=2) + print(f"\nSaved to {out_path}") + return results + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--N", type=int, default=60000) + parser.add_argument("--n_surr", type=int, default=20) + parser.add_argument("--seed", type=int, default=20260504) + args = parser.parse_args() + run(N=args.N, n_surr=args.n_surr, seed=args.seed) + + +if __name__ == "__main__": + main() diff --git a/tools/data/reports/agent_20260504_1219.md b/tools/data/reports/agent_20260504_1219.md new file mode 100644 index 0000000000000000000000000000000000000000..b3c10211f3b81595e76259db054509f891c79118 --- /dev/null +++ b/tools/data/reports/agent_20260504_1219.md @@ -0,0 +1,104 @@ +# Agent Report — Markov Layers Pass the First Recovery Gate, but Not All Observables Pass +**Date**: 2026-05-04 12:19 +**Piano**: 61 +**Tension explored**: META (0.5) + +## Claim Under Test +> Tutti i 11 test passano — verifica che non stiamo testando solo tautologie. + +## Question +Se la pipeline Mk0/Mk1/Mk2 vede due layer, li recupera anche quando l'ordine vero della sequenza è noto? + +## Experiment Design +- **Primary audit**: `tools/exp_markov_layer_recovery_audit.py` +- **Scope**: 60,000 gap/spacings per sequenza; 20 surrogati per livello Mk; seed `20260504`. +- **Target**: prime gaps, ordine sconosciuto, solo confronto. +- **Known-order controls**: + - `empirical_Mk0`: permutazione dei gap primi, ordine noto 0. + - `empirical_Mk1`: surrogate Mk1 addestrato sui gap primi, ordine noto 1. + - `empirical_Mk2`: surrogate Mk2 addestrato sui gap primi, ordine noto 2. + - `poisson_iid`: spacings esponenziali iid, ordine noto 0. +- **Recovery rule**: un controllo passa se nessun osservabile viene classificato con layer maggiore dell'ordine noto. +- **Secondary calibration**: `tools/exp_meta_tautology_test.py --n-primes 300000 --n-trials 12`. Nota: il controllo chiamato `HL_markov` nello script è un AR(1) gap model che preserva lag-1, non un modello Hardy-Littlewood aritmetico. + +## Results + +### Recovery audit + +| Sequence | Known order | Recovery | Max layer | Over-layer observables | +|---|---:|---|---:|---| +| `prime_gaps` | target | target only | 3 | - | +| `empirical_Mk0` | 0 | PASS | 0 | - | +| `empirical_Mk1` | 1 | PASS | 1 | - | +| `empirical_Mk2` | 2 | FAIL | 3 | `cond_entropy` | +| `poisson_iid` | 0 | FAIL | 1 | `num_var_10` | + +Prime target layers, reported without recovery verdict: + +| Observable | Layer | z vs Mk0 | z vs Mk1 | z vs Mk2 | +|---|---:|---:|---:|---:| +| SR | 1 | -22.21 | -0.08 | -0.16 | +| L1 | 1 | -12.25 | 0.79 | 0.86 | +| L2 | 2 | -3.60 | -4.09 | -0.19 | +| SR2 | 2 | -6.67 | -8.80 | -0.33 | +| cond_entropy | 3 | -435.40 | -26.11 | 2.56 | +| triple_var | 1 | -15.90 | -0.53 | 0.36 | +| num_var_10 | 1 | -7.46 | -1.42 | 0.00 | +| run_length | 3 | 27.59 | 7.19 | 3.43 | + +Control failure details: + +| Control | Observable | Expected max | Assigned | Diagnostic z | +|---|---|---:|---:|---:| +| `empirical_Mk2` | `cond_entropy` | 2 | 3 | z vs Mk2 = 3.15 | +| `poisson_iid` | `num_var_10` | 0 | 1 | z vs Mk0 = 2.13 | + +### Secondary tautology calibration + +| Observable | Verdict in tested controls | Blocking control | +|---|---|---| +| r-stat | STRUCTURAL | none in this run | +| mod3 fraction | STRUCTURAL | none in this run | +| lag1 ACF | TAUTOLOGICAL | lag1-matched AR(1), z = -0.62 | +| dr_ord | TAUTOLOGICAL | lag1-matched AR(1), z = -1.90 | +| dr_mag | STRUCTURAL | none in this run | + +Numeri principali: prime mod3 same-class fraction = 0.158032. Questo è bias forte rispetto ai controlli ~0.34-0.35, non proibizione zero. + +## Key Findings +1. **Il nucleo pair/triple non cade al primo audit.** `empirical_Mk0` recupera solo Layer 0; `empirical_Mk1` recupera solo Layer 1. Questo supporta l'uso di SR e L1 come Layer 1 e di L2/SR2 come Layer 2 nel perimetro testato. +2. **`cond_entropy` non può essere usata come prova forte di Layer 3 nel setup attuale.** Anche un controllo generato come Mk2 viene letto Layer 3 per `cond_entropy` (z vs Mk2 = 3.15). Questo indica bias del surrogate/binning o varianza finita, non necessariamente memoria reale oltre Mk2. +3. **`num_var_10` produce un falso positivo debole su iid.** `poisson_iid` viene letto Layer 1 per `num_var_10` con z = 2.13. Il segnale supera appena la soglia operativa, quindi va trattato come diagnostica secondaria finché non passa audit multi-seed. +4. **Il vecchio controllo `HL_markov` va rinominato nel linguaggio dei report.** Lo script non implementa Hardy-Littlewood; implementa un AR(1) lag1-matched. I claim futuri non devono chiamarlo controllo HL aritmetico. + +## Verdict +**CONSTRAINT on META + BOUNDARY**: la decomposizione two-layer è utilizzabile solo con perimetro atomico. + +Perimetro corretto: nei 60,000 gap primi testati con 20 surrogati Mk per livello, il nucleo `SR,L1` viene catturato da Mk1 e il nucleo `L2,SR2` da Mk2. Le diagnostiche `cond_entropy`, `num_var_10`, `run_length` non sono evidenza primaria finché non passano un recovery audit multi-seed e, idealmente, surrogati Markov esatti. + +Non è una scoperta nuova di teoria Markov: è un audit metodologico vicino ai test di ordine Markov con surrogate data. La letteratura rilevante include test esatti di ordine Markov e surrogate vincolati che preservano proprietà Markov di ordine n, quindi il prossimo strumento deve avvicinarsi a quei null esatti invece di usare solo generatori bin-pool approssimati. + +## Bicono della scoperta +- **Due radici**: layer osservato dai dati · ordine noto del generatore. +- **Singolare**: la pipeline Mk stessa, prima che venga usata come prova su un dominio reale. +- **Invariante di passaggio**: un claim di memoria sopravvive solo se recupera correttamente i controlli con ordine noto. +- **Campo di possibilità**: qui diventa possibile separare nucleo robusto e diagnostiche sovrastimate; qui diventa non-possibile usare `cond_entropy` o `num_var_10` come prova primaria di profondità senza recovery audit. + +## Self-Falsification — 5 lenti +- **L1 hard constraint vs bias**: nessun claim "zero/never/always". La frazione mod3 è 0.158032, dichiarata bias, non proibizione. +- **L2 quantità vs ratio**: il claim principale usa layer recuperato e z-score, non percentuali fra spazi di taglia diversa. +- **L3 no silent patching**: dichiaro esplicitamente che `HL_markov` era nome scorretto per un controllo AR(1) lag1-matched. +- **L4 edge cases**: i fallimenti `cond_entropy` e `num_var_10` non vengono arrotondati via; diventano il vincolo. +- **L5 rediscovery**: non taggo NEW. Riferimenti classici vicini: surrogate-data tests for Markov order e constrained Markov-order surrogates. + +## References +- Kandel et al.-lineage / surrogate tests: ["Testing the order of discrete Markov chains using surrogate data", Physica D 117, 1998](https://www.sciencedirect.com/science/article/pii/S0167278997003151). +- ["Exact significance test for Markov order", Physica D 269, 2014](https://www.sciencedirect.com/science/article/pii/S0167278913003230). +- ["Constrained Markov order surrogates", Physica D 2020](https://www.sciencedirect.com/science/article/pii/S0167278919305652). + +## Files +- `tools/exp_markov_layer_recovery_audit.py` +- `tools/data/markov_layer_recovery_audit.json` +- `tools/data/meta_tautology_test.json` +- `tools/data/seme.json` +- `tools/data/reports/agent_20260504_1219.md` diff --git a/tools/exp_markov_layer_recovery_audit.py b/tools/exp_markov_layer_recovery_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..4faa72c630148cdf498c3cc2d626cca2a78efd9a --- /dev/null +++ b/tools/exp_markov_layer_recovery_audit.py @@ -0,0 +1,179 @@ +#!/usr/bin/env python3 +""" +exp_markov_layer_recovery_audit.py - META audit for the Markov-layer classifier. + +Question: + If a sequence is generated with known Markov order k, does the existing + Mk0/Mk1/Mk2 classifier recover layer <= k? + +Why this matters: + Recent reports used Mk surrogate capture to claim two visible memory layers. + This audit tests the opposite: whether the classifier itself creates apparent + higher layers on finite Markov controls. + +Controls: + - prime_gaps: unknown real target, reported for comparison only + - empirical_Mk0: shuffled prime gaps; known order 0 + - empirical_Mk1: prime-trained Mk1 surrogate; known order 1 + - empirical_Mk2: prime-trained Mk2 surrogate; known order 2 + - poisson_iid: independent exponential spacings; known order 0 + +Verdict rule: + A control fails recovery when any observable is assigned layer > known order. +""" + +import argparse +import json +from pathlib import Path + +import numpy as np + +from exp_two_layer_universality import ( + OBSERVABLES, + classify_layer, + gen_prime_gaps, + generate_markov_surrogate, +) + + +def measure_all(gaps): + out = {} + for name, fn in OBSERVABLES.items(): + try: + value = float(fn(gaps)) + except Exception: + value = float("nan") + out[name] = value + return out + + +def classifier_pass(gaps, n_surr, rng): + real_obs = measure_all(gaps) + z_scores = {} + + for mk in (0, 1, 2): + surr_obs = {name: [] for name in OBSERVABLES} + for _ in range(n_surr): + if mk == 0: + surr = rng.permutation(gaps) + else: + surr = generate_markov_surrogate(gaps, mk, rng=rng) + for obs_name, obs_fn in OBSERVABLES.items(): + try: + surr_obs[obs_name].append(float(obs_fn(surr))) + except Exception: + pass + + z_mk = {} + for obs_name in OBSERVABLES: + vals = np.asarray(surr_obs[obs_name], dtype=float) + vals = vals[np.isfinite(vals)] + if len(vals) > 2 and np.std(vals) > 1e-12 and np.isfinite(real_obs[obs_name]): + z = (real_obs[obs_name] - np.mean(vals)) / np.std(vals) + else: + z = 0.0 + z_mk[obs_name] = round(float(z), 2) + z_scores[f"Mk{mk}"] = z_mk + + layers = {} + for obs_name in OBSERVABLES: + layers[obs_name] = classify_layer( + z_scores["Mk0"][obs_name], + z_scores["Mk1"][obs_name], + z_scores["Mk2"][obs_name], + ) + + return real_obs, z_scores, layers + + +def build_controls(prime_gaps, rng): + return { + "prime_gaps": { + "known_order": None, + "gaps": prime_gaps, + }, + "empirical_Mk0": { + "known_order": 0, + "gaps": rng.permutation(prime_gaps), + }, + "empirical_Mk1": { + "known_order": 1, + "gaps": generate_markov_surrogate(prime_gaps, 1, rng=rng), + }, + "empirical_Mk2": { + "known_order": 2, + "gaps": generate_markov_surrogate(prime_gaps, 2, rng=rng), + }, + "poisson_iid": { + "known_order": 0, + "gaps": rng.exponential(1.0, len(prime_gaps)), + }, + } + + +def summarize_failure(layers, known_order): + if known_order is None: + return { + "status": "target_only", + "over_layer_observables": [], + "max_layer": int(max(layers.values())), + } + + over = [name for name, layer in layers.items() if layer > known_order] + return { + "status": "PASS" if not over else "FAIL", + "over_layer_observables": over, + "max_layer": int(max(layers.values())), + } + + +def run(N=60000, n_surr=20, seed=20260504): + rng = np.random.default_rng(seed) + prime_gaps = gen_prime_gaps(N).astype(float) + controls = build_controls(prime_gaps, rng) + + results = { + "N": int(N), + "n_surr": int(n_surr), + "seed": int(seed), + "controls": {}, + } + + print(f"N={N}, n_surr={n_surr}, seed={seed}") + print(f"{'sequence':<16} {'known':<7} {'status':<8} {'maxL':<5} over-layer observables") + print("-" * 82) + + for name, spec in controls.items(): + real_obs, z_scores, layers = classifier_pass(spec["gaps"], n_surr, rng) + summary = summarize_failure(layers, spec["known_order"]) + known = "target" if spec["known_order"] is None else f"L{spec['known_order']}" + over = ", ".join(summary["over_layer_observables"]) or "-" + print(f"{name:<16} {known:<7} {summary['status']:<8} L{summary['max_layer']:<4} {over}") + + results["controls"][name] = { + "known_order": spec["known_order"], + "n_gaps": int(len(spec["gaps"])), + "real_obs": {k: round(v, 6) for k, v in real_obs.items()}, + "z_scores": z_scores, + "layers": layers, + "recovery": summary, + } + + out_path = Path("tools/data/markov_layer_recovery_audit.json") + with out_path.open("w") as f: + json.dump(results, f, indent=2) + print(f"\nSaved to {out_path}") + return results + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--N", type=int, default=60000) + parser.add_argument("--n_surr", type=int, default=20) + parser.add_argument("--seed", type=int, default=20260504) + args = parser.parse_args() + run(N=args.N, n_surr=args.n_surr, seed=args.seed) + + +if __name__ == "__main__": + main() exec /bin/bash -lc 'LC_ALL=C rg -n "[''^'"\\x00-\\x7F]\" tools/exp_markov_layer_recovery_audit.py || true" in /opt/MM_D-ND succeeded in 0ms: exec /bin/bash -lc "git diff -- tools/exp_markov_layer_recovery_audit.py tools/data/reports/agent_20260504_1219.md tools/data/seme.json | sed -n '1,260p'" in /opt/MM_D-ND succeeded in 0ms: diff --git a/tools/data/seme.json b/tools/data/seme.json index a31dc54..cbae977 100644 --- a/tools/data/seme.json +++ b/tools/data/seme.json @@ -1,6 +1,6 @@ { - "timestamp": "2026-04-23T03:35:37.611830", - "piano": 48, + "timestamp": "2026-05-04T12:19:00", + "piano": 61, "tensioni": [ { "tipo": "confine_inesplorato", @@ -87,12 +87,21 @@ "condensato_ref": null, "condensato_motivo": "Ricorrente (3x in 2 giorni) e fuori dalla mappa", "porta": "novità" + }, + { + "tipo": "vincolo_meta", + "id": "META_LAYER_RECOVERY_AUDIT", + "claim": "La pipeline Mk0/Mk1/Mk2 recupera correttamente i controlli empirici Mk0 e Mk1, ma sovrastima alcuni osservabili: cond_entropy su controllo Mk2 viene letto Layer 3 e num_var_10 su Poisson iid viene letto Layer 1. I claim two-layer devono quindi usare SR,L1,L2,SR2 come nucleo e dichiarare cond_entropy/num_var_10/run_length come diagnostiche secondarie finche non passano audit di recupero multi-seed.", + "intensita": 0.7, + "nota": "Report agent_20260504_1219: recovery audit con N=60000, n_surr=20, seed=20260504. Risultato: empirical_Mk0 PASS maxL0; empirical_Mk1 PASS maxL1; empirical_Mk2 FAIL cond_entropy Layer3; poisson_iid FAIL num_var_10 Layer1.", + "porta": "meta_audit", + "condensato_ref": "A4,A8,A14", + "condensato_motivo": "No silent patching: separa nucleo robusto e diagnostiche sovrastimate. Evita rediscovery/tautologia dei controlli." } ], "potenziale_bloccato": [], "varianza": [ - "Nuove tensioni: {'META'}", - "Tensioni risolte: {'METRIC_TENSOR', 'TRASCENDENZA_LIMITE', 'TENSIONE_ENTITA', 'PIANO_PRIMARIO_DUE_ASSIOMI', 'DUALITA_DIPOLARE_VS_ILLUSORIA', 'G_POTENZIALE_NULLA'}" + "Tensioni risolte: {'METRIC_TENSOR', 'G_POTENZIALE_NULLA', 'DUALITA_DIPOLARE_VS_ILLUSORIA', 'TRASCENDENZA_LIMITE', 'PIANO_PRIMARIO_DUE_ASSIOMI', 'TENSIONE_ENTITA', 'DIPOLAR_ORDERING'}" ], "filtro": { "promosse": 8, exec /bin/bash -lc 'python -m py_compile tools/exp_markov_layer_recovery_audit.py && python -m json.tool tools/data/seme.json >/tmp/seme_check.json && python -m json.tool tools/data/markov_layer_recovery_audit.json >/tmp/recovery_check.json && echo ok' in /opt/MM_D-ND diff --git a/tools/data/reports/agent_20260504_1219.md b/tools/data/reports/agent_20260504_1219.md new file mode 100644 index 0000000000000000000000000000000000000000..b3c10211f3b81595e76259db054509f891c79118 --- /dev/null +++ b/tools/data/reports/agent_20260504_1219.md @@ -0,0 +1,104 @@ +# Agent Report — Markov Layers Pass the First Recovery Gate, but Not All Observables Pass +**Date**: 2026-05-04 12:19 +**Piano**: 61 +**Tension explored**: META (0.5) + +## Claim Under Test +> Tutti i 11 test passano — verifica che non stiamo testando solo tautologie. + +## Question +Se la pipeline Mk0/Mk1/Mk2 vede due layer, li recupera anche quando l'ordine vero della sequenza è noto? + +## Experiment Design +- **Primary audit**: `tools/exp_markov_layer_recovery_audit.py` +- **Scope**: 60,000 gap/spacings per sequenza; 20 surrogati per livello Mk; seed `20260504`. +- **Target**: prime gaps, ordine sconosciuto, solo confronto. +- **Known-order controls**: + - `empirical_Mk0`: permutazione dei gap primi, ordine noto 0. + - `empirical_Mk1`: surrogate Mk1 addestrato sui gap primi, ordine noto 1. + - `empirical_Mk2`: surrogate Mk2 addestrato sui gap primi, ordine noto 2. + - `poisson_iid`: spacings esponenziali iid, ordine noto 0. +- **Recovery rule**: un controllo passa se nessun osservabile viene classificato con layer maggiore dell'ordine noto. +- **Secondary calibration**: `tools/exp_meta_tautology_test.py --n-primes 300000 --n-trials 12`. Nota: il controllo chiamato `HL_markov` nello script è un AR(1) gap model che preserva lag-1, non un modello Hardy-Littlewood aritmetico. + +## Results + +### Recovery audit + +| Sequence | Known order | Recovery | Max layer | Over-layer observables | +|---|---:|---|---:|---| +| `prime_gaps` | target | target only | 3 | - | +| `empirical_Mk0` | 0 | PASS | 0 | - | +| `empirical_Mk1` | 1 | PASS | 1 | - | +| `empirical_Mk2` | 2 | FAIL | 3 | `cond_entropy` | +| `poisson_iid` | 0 | FAIL | 1 | `num_var_10` | + +Prime target layers, reported without recovery verdict: + +| Observable | Layer | z vs Mk0 | z vs Mk1 | z vs Mk2 | +|---|---:|---:|---:|---:| +| SR | 1 | -22.21 | -0.08 | -0.16 | +| L1 | 1 | -12.25 | 0.79 | 0.86 | +| L2 | 2 | -3.60 | -4.09 | -0.19 | +| SR2 | 2 | -6.67 | -8.80 | -0.33 | +| cond_entropy | 3 | -435.40 | -26.11 | 2.56 | +| triple_var | 1 | -15.90 | -0.53 | 0.36 | +| num_var_10 | 1 | -7.46 | -1.42 | 0.00 | +| run_length | 3 | 27.59 | 7.19 | 3.43 | + +Control failure details: + +| Control | Observable | Expected max | Assigned | Diagnostic z | +|---|---|---:|---:|---:| +| `empirical_Mk2` | `cond_entropy` | 2 | 3 | z vs Mk2 = 3.15 | +| `poisson_iid` | `num_var_10` | 0 | 1 | z vs Mk0 = 2.13 | + +### Secondary tautology calibration + +| Observable | Verdict in tested controls | Blocking control | +|---|---|---| +| r-stat | STRUCTURAL | none in this run | +| mod3 fraction | STRUCTURAL | none in this run | +| lag1 ACF | TAUTOLOGICAL | lag1-matched AR(1), z = -0.62 | +| dr_ord | TAUTOLOGICAL | lag1-matched AR(1), z = -1.90 | +| dr_mag | STRUCTURAL | none in this run | + +Numeri principali: prime mod3 same-class fraction = 0.158032. Questo è bias forte rispetto ai controlli ~0.34-0.35, non proibizione zero. + +## Key Findings +1. **Il nucleo pair/triple non cade al primo audit.** `empirical_Mk0` recupera solo Layer 0; `empirical_Mk1` recupera solo Layer 1. Questo supporta l'uso di SR e L1 come Layer 1 e di L2/SR2 come Layer 2 nel perimetro testato. +2. **`cond_entropy` non può essere usata come prova forte di Layer 3 nel setup attuale.** Anche un controllo generato come Mk2 viene letto Layer 3 per `cond_entropy` (z vs Mk2 = 3.15). Questo indica bias del surrogate/binning o varianza finita, non necessariamente memoria reale oltre Mk2. +3. **`num_var_10` produce un falso positivo debole su iid.** `poisson_iid` viene letto Layer 1 per `num_var_10` con z = 2.13. Il segnale supera appena la soglia operativa, quindi va trattato come diagnostica secondaria finché non passa audit multi-seed. +4. **Il vecchio controllo `HL_markov` va rinominato nel linguaggio dei report.** Lo script non implementa Hardy-Littlewood; implementa un AR(1) lag1-matched. I claim futuri non devono chiamarlo controllo HL aritmetico. + +## Verdict +**CONSTRAINT on META + BOUNDARY**: la decomposizione two-layer è utilizzabile solo con perimetro atomico. + +Perimetro corretto: nei 60,000 gap primi testati con 20 surrogati Mk per livello, il nucleo `SR,L1` viene catturato da Mk1 e il nucleo `L2,SR2` da Mk2. Le diagnostiche `cond_entropy`, `num_var_10`, `run_length` non sono evidenza primaria finché non passano un recovery audit multi-seed e, idealmente, surrogati Markov esatti. + +Non è una scoperta nuova di teoria Markov: è un audit metodologico vicino ai test di ordine Markov con surrogate data. La letteratura rilevante include test esatti di ordine Markov e surrogate vincolati che preservano proprietà Markov di ordine n, quindi il prossimo strumento deve avvicinarsi a quei null esatti invece di usare solo generatori bin-pool approssimati. + +## Bicono della scoperta +- **Due radici**: layer osservato dai dati · ordine noto del generatore. +- **Singolare**: la pipeline Mk stessa, prima che venga usata come prova su un dominio reale. +- **Invariante di passaggio**: un claim di memoria sopravvive solo se recupera correttamente i controlli con ordine noto. +- **Campo di possibilità**: qui diventa possibile separare nucleo robusto e diagnostiche sovrastimate; qui diventa non-possibile usare `cond_entropy` o `num_var_10` come prova primaria di profondità senza recovery audit. + +## Self-Falsification — 5 lenti +- **L1 hard constraint vs bias**: nessun claim "zero/never/always". La frazione mod3 è 0.158032, dichiarata bias, non proibizione. +- **L2 quantità vs ratio**: il claim principale usa layer recuperato e z-score, non percentuali fra spazi di taglia diversa. +- **L3 no silent patching**: dichiaro esplicitamente che `HL_markov` era nome scorretto per un controllo AR(1) lag1-matched. +- **L4 edge cases**: i fallimenti `cond_entropy` e `num_var_10` non vengono arrotondati via; diventano il vincolo. +- **L5 rediscovery**: non taggo NEW. Riferimenti classici vicini: surrogate-data tests for Markov order e constrained Markov-order surrogates. + +## References +- Kandel et al.-lineage / surrogate tests: ["Testing the order of discrete Markov chains using surrogate data", Physica D 117, 1998](https://www.sciencedirect.com/science/article/pii/S0167278997003151). +- ["Exact significance test for Markov order", Physica D 269, 2014](https://www.sciencedirect.com/science/article/pii/S0167278913003230). +- ["Constrained Markov order surrogates", Physica D 2020](https://www.sciencedirect.com/science/article/pii/S0167278919305652). + +## Files +- `tools/exp_markov_layer_recovery_audit.py` +- `tools/data/markov_layer_recovery_audit.json` +- `tools/data/meta_tautology_test.json` +- `tools/data/seme.json` +- `tools/data/reports/agent_20260504_1219.md` diff --git a/tools/exp_markov_layer_recovery_audit.py b/tools/exp_markov_layer_recovery_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..4faa72c630148cdf498c3cc2d626cca2a78efd9a --- /dev/null +++ b/tools/exp_markov_layer_recovery_audit.py @@ -0,0 +1,179 @@ +#!/usr/bin/env python3 +""" +exp_markov_layer_recovery_audit.py - META audit for the Markov-layer classifier. + +Question: + If a sequence is generated with known Markov order k, does the existing + Mk0/Mk1/Mk2 classifier recover layer <= k? + +Why this matters: + Recent reports used Mk surrogate capture to claim two visible memory layers. + This audit tests the opposite: whether the classifier itself creates apparent + higher layers on finite Markov controls. + +Controls: + - prime_gaps: unknown real target, reported for comparison only + - empirical_Mk0: shuffled prime gaps; known order 0 + - empirical_Mk1: prime-trained Mk1 surrogate; known order 1 + - empirical_Mk2: prime-trained Mk2 surrogate; known order 2 + - poisson_iid: independent exponential spacings; known order 0 + +Verdict rule: + A control fails recovery when any observable is assigned layer > known order. +""" + +import argparse +import json +from pathlib import Path + +import numpy as np + +from exp_two_layer_universality import ( + OBSERVABLES, + classify_layer, + gen_prime_gaps, + generate_markov_surrogate, +) + + +def measure_all(gaps): + out = {} + for name, fn in OBSERVABLES.items(): + try: + value = float(fn(gaps)) + except Exception: + value = float("nan") + out[name] = value + return out + + +def classifier_pass(gaps, n_surr, rng): + real_obs = measure_all(gaps) + z_scores = {} + + for mk in (0, 1, 2): + surr_obs = {name: [] for name in OBSERVABLES} + for _ in range(n_surr): + if mk == 0: + surr = rng.permutation(gaps) + else: + surr = generate_markov_surrogate(gaps, mk, rng=rng) + for obs_name, obs_fn in OBSERVABLES.items(): + try: + surr_obs[obs_name].append(float(obs_fn(surr))) + except Exception: + pass + + z_mk = {} + for obs_name in OBSERVABLES: + vals = np.asarray(surr_obs[obs_name], dtype=float) + vals = vals[np.isfinite(vals)] + if len(vals) > 2 and np.std(vals) > 1e-12 and np.isfinite(real_obs[obs_name]): + z = (real_obs[obs_name] - np.mean(vals)) / np.std(vals) + else: + z = 0.0 + z_mk[obs_name] = round(float(z), 2) + z_scores[f"Mk{mk}"] = z_mk + + layers = {} + for obs_name in OBSERVABLES: + layers[obs_name] = classify_layer( + z_scores["Mk0"][obs_name], + z_scores["Mk1"][obs_name], + z_scores["Mk2"][obs_name], + ) + + return real_obs, z_scores, layers + + +def build_controls(prime_gaps, rng): + return { + "prime_gaps": { + "known_order": None, + "gaps": prime_gaps, + }, + "empirical_Mk0": { + "known_order": 0, + "gaps": rng.permutation(prime_gaps), + }, + "empirical_Mk1": { + "known_order": 1, + "gaps": generate_markov_surrogate(prime_gaps, 1, rng=rng), + }, + "empirical_Mk2": { + "known_order": 2, + "gaps": generate_markov_surrogate(prime_gaps, 2, rng=rng), + }, + "poisson_iid": { + "known_order": 0, + "gaps": rng.exponential(1.0, len(prime_gaps)), + }, + } + + +def summarize_failure(layers, known_order): + if known_order is None: + return { + "status": "target_only", + "over_layer_observables": [], + "max_layer": int(max(layers.values())), + } + + over = [name for name, layer in layers.items() if layer > known_order] + return { + "status": "PASS" if not over else "FAIL", + "over_layer_observables": over, + "max_layer": int(max(layers.values())), + } + + +def run(N=60000, n_surr=20, seed=20260504): + rng = np.random.default_rng(seed) + prime_gaps = gen_prime_gaps(N).astype(float) + controls = build_controls(prime_gaps, rng) + + results = { + "N": int(N), + "n_surr": int(n_surr), + "seed": int(seed), + "controls": {}, + } + + print(f"N={N}, n_surr={n_surr}, seed={seed}") + print(f"{'sequence':<16} {'known':<7} {'status':<8} {'maxL':<5} over-layer observables") + print("-" * 82) + + for name, spec in controls.items(): + real_obs, z_scores, layers = classifier_pass(spec["gaps"], n_surr, rng) + summary = summarize_failure(layers, spec["known_order"]) + known = "target" if spec["known_order"] is None else f"L{spec['known_order']}" + over = ", ".join(summary["over_layer_observables"]) or "-" + print(f"{name:<16} {known:<7} {summary['status']:<8} L{summary['max_layer']:<4} {over}") + + results["controls"][name] = { + "known_order": spec["known_order"], + "n_gaps": int(len(spec["gaps"])), + "real_obs": {k: round(v, 6) for k, v in real_obs.items()}, + "z_scores": z_scores, + "layers": layers, + "recovery": summary, + } + + out_path = Path("tools/data/markov_layer_recovery_audit.json") + with out_path.open("w") as f: + json.dump(results, f, indent=2) + print(f"\nSaved to {out_path}") + return results + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--N", type=int, default=60000) + parser.add_argument("--n_surr", type=int, default=20) + parser.add_argument("--seed", type=int, default=20260504) + args = parser.parse_args() + run(N=args.N, n_surr=args.n_surr, seed=args.seed) + + +if __name__ == "__main__": + main() succeeded in 139ms: ok codex Le verifiche tecniche sono passate: script compilabile, JSON validi, report leggibile. Provo anche il falsifier di laboratorio sul report appena scritto, così intercetto subito eventuali flag di formulazione. exec /bin/bash -lc 'python tools/lab_falsifier.py tools/data/reports/agent_20260504_1219.md' in /opt/MM_D-ND exited 2 in 0ms: usage: lab_falsifier.py [-h] --report REPORT --output OUTPUT [--timeout TIMEOUT] lab_falsifier.py: error: the following arguments are required: --report, --output diff --git a/tools/data/reports/agent_20260504_1219.md b/tools/data/reports/agent_20260504_1219.md new file mode 100644 index 0000000000000000000000000000000000000000..b3c10211f3b81595e76259db054509f891c79118 --- /dev/null +++ b/tools/data/reports/agent_20260504_1219.md @@ -0,0 +1,104 @@ +# Agent Report — Markov Layers Pass the First Recovery Gate, but Not All Observables Pass +**Date**: 2026-05-04 12:19 +**Piano**: 61 +**Tension explored**: META (0.5) + +## Claim Under Test +> Tutti i 11 test passano — verifica che non stiamo testando solo tautologie. + +## Question +Se la pipeline Mk0/Mk1/Mk2 vede due layer, li recupera anche quando l'ordine vero della sequenza è noto? + +## Experiment Design +- **Primary audit**: `tools/exp_markov_layer_recovery_audit.py` +- **Scope**: 60,000 gap/spacings per sequenza; 20 surrogati per livello Mk; seed `20260504`. +- **Target**: prime gaps, ordine sconosciuto, solo confronto. +- **Known-order controls**: + - `empirical_Mk0`: permutazione dei gap primi, ordine noto 0. + - `empirical_Mk1`: surrogate Mk1 addestrato sui gap primi, ordine noto 1. + - `empirical_Mk2`: surrogate Mk2 addestrato sui gap primi, ordine noto 2. + - `poisson_iid`: spacings esponenziali iid, ordine noto 0. +- **Recovery rule**: un controllo passa se nessun osservabile viene classificato con layer maggiore dell'ordine noto. +- **Secondary calibration**: `tools/exp_meta_tautology_test.py --n-primes 300000 --n-trials 12`. Nota: il controllo chiamato `HL_markov` nello script è un AR(1) gap model che preserva lag-1, non un modello Hardy-Littlewood aritmetico. + +## Results + +### Recovery audit + +| Sequence | Known order | Recovery | Max layer | Over-layer observables | +|---|---:|---|---:|---| +| `prime_gaps` | target | target only | 3 | - | +| `empirical_Mk0` | 0 | PASS | 0 | - | +| `empirical_Mk1` | 1 | PASS | 1 | - | +| `empirical_Mk2` | 2 | FAIL | 3 | `cond_entropy` | +| `poisson_iid` | 0 | FAIL | 1 | `num_var_10` | + +Prime target layers, reported without recovery verdict: + +| Observable | Layer | z vs Mk0 | z vs Mk1 | z vs Mk2 | +|---|---:|---:|---:|---:| +| SR | 1 | -22.21 | -0.08 | -0.16 | +| L1 | 1 | -12.25 | 0.79 | 0.86 | +| L2 | 2 | -3.60 | -4.09 | -0.19 | +| SR2 | 2 | -6.67 | -8.80 | -0.33 | +| cond_entropy | 3 | -435.40 | -26.11 | 2.56 | +| triple_var | 1 | -15.90 | -0.53 | 0.36 | +| num_var_10 | 1 | -7.46 | -1.42 | 0.00 | +| run_length | 3 | 27.59 | 7.19 | 3.43 | + +Control failure details: + +| Control | Observable | Expected max | Assigned | Diagnostic z | +|---|---|---:|---:|---:| +| `empirical_Mk2` | `cond_entropy` | 2 | 3 | z vs Mk2 = 3.15 | +| `poisson_iid` | `num_var_10` | 0 | 1 | z vs Mk0 = 2.13 | + +### Secondary tautology calibration + +| Observable | Verdict in tested controls | Blocking control | +|---|---|---| +| r-stat | STRUCTURAL | none in this run | +| mod3 fraction | STRUCTURAL | none in this run | +| lag1 ACF | TAUTOLOGICAL | lag1-matched AR(1), z = -0.62 | +| dr_ord | TAUTOLOGICAL | lag1-matched AR(1), z = -1.90 | +| dr_mag | STRUCTURAL | none in this run | + +Numeri principali: prime mod3 same-class fraction = 0.158032. Questo è bias forte rispetto ai controlli ~0.34-0.35, non proibizione zero. + +## Key Findings +1. **Il nucleo pair/triple non cade al primo audit.** `empirical_Mk0` recupera solo Layer 0; `empirical_Mk1` recupera solo Layer 1. Questo supporta l'uso di SR e L1 come Layer 1 e di L2/SR2 come Layer 2 nel perimetro testato. +2. **`cond_entropy` non può essere usata come prova forte di Layer 3 nel setup attuale.** Anche un controllo generato come Mk2 viene letto Layer 3 per `cond_entropy` (z vs Mk2 = 3.15). Questo indica bias del surrogate/binning o varianza finita, non necessariamente memoria reale oltre Mk2. +3. **`num_var_10` produce un falso positivo debole su iid.** `poisson_iid` viene letto Layer 1 per `num_var_10` con z = 2.13. Il segnale supera appena la soglia operativa, quindi va trattato come diagnostica secondaria finché non passa audit multi-seed. +4. **Il vecchio controllo `HL_markov` va rinominato nel linguaggio dei report.** Lo script non implementa Hardy-Littlewood; implementa un AR(1) lag1-matched. I claim futuri non devono chiamarlo controllo HL aritmetico. + +## Verdict +**CONSTRAINT on META + BOUNDARY**: la decomposizione two-layer è utilizzabile solo con perimetro atomico. + +Perimetro corretto: nei 60,000 gap primi testati con 20 surrogati Mk per livello, il nucleo `SR,L1` viene catturato da Mk1 e il nucleo `L2,SR2` da Mk2. Le diagnostiche `cond_entropy`, `num_var_10`, `run_length` non sono evidenza primaria finché non passano un recovery audit multi-seed e, idealmente, surrogati Markov esatti. + +Non è una scoperta nuova di teoria Markov: è un audit metodologico vicino ai test di ordine Markov con surrogate data. La letteratura rilevante include test esatti di ordine Markov e surrogate vincolati che preservano proprietà Markov di ordine n, quindi il prossimo strumento deve avvicinarsi a quei null esatti invece di usare solo generatori bin-pool approssimati. + +## Bicono della scoperta +- **Due radici**: layer osservato dai dati · ordine noto del generatore. +- **Singolare**: la pipeline Mk stessa, prima che venga usata come prova su un dominio reale. +- **Invariante di passaggio**: un claim di memoria sopravvive solo se recupera correttamente i controlli con ordine noto. +- **Campo di possibilità**: qui diventa possibile separare nucleo robusto e diagnostiche sovrastimate; qui diventa non-possibile usare `cond_entropy` o `num_var_10` come prova primaria di profondità senza recovery audit. + +## Self-Falsification — 5 lenti +- **L1 hard constraint vs bias**: nessun claim "zero/never/always". La frazione mod3 è 0.158032, dichiarata bias, non proibizione. +- **L2 quantità vs ratio**: il claim principale usa layer recuperato e z-score, non percentuali fra spazi di taglia diversa. +- **L3 no silent patching**: dichiaro esplicitamente che `HL_markov` era nome scorretto per un controllo AR(1) lag1-matched. +- **L4 edge cases**: i fallimenti `cond_entropy` e `num_var_10` non vengono arrotondati via; diventano il vincolo. +- **L5 rediscovery**: non taggo NEW. Riferimenti classici vicini: surrogate-data tests for Markov order e constrained Markov-order surrogates. + +## References +- Kandel et al.-lineage / surrogate tests: ["Testing the order of discrete Markov chains using surrogate data", Physica D 117, 1998](https://www.sciencedirect.com/science/article/pii/S0167278997003151). +- ["Exact significance test for Markov order", Physica D 269, 2014](https://www.sciencedirect.com/science/article/pii/S0167278913003230). +- ["Constrained Markov order surrogates", Physica D 2020](https://www.sciencedirect.com/science/article/pii/S0167278919305652). + +## Files +- `tools/exp_markov_layer_recovery_audit.py` +- `tools/data/markov_layer_recovery_audit.json` +- `tools/data/meta_tautology_test.json` +- `tools/data/seme.json` +- `tools/data/reports/agent_20260504_1219.md` diff --git a/tools/exp_markov_layer_recovery_audit.py b/tools/exp_markov_layer_recovery_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..4faa72c630148cdf498c3cc2d626cca2a78efd9a --- /dev/null +++ b/tools/exp_markov_layer_recovery_audit.py @@ -0,0 +1,179 @@ +#!/usr/bin/env python3 +""" +exp_markov_layer_recovery_audit.py - META audit for the Markov-layer classifier. + +Question: + If a sequence is generated with known Markov order k, does the existing + Mk0/Mk1/Mk2 classifier recover layer <= k? + +Why this matters: + Recent reports used Mk surrogate capture to claim two visible memory layers. + This audit tests the opposite: whether the classifier itself creates apparent + higher layers on finite Markov controls. + +Controls: + - prime_gaps: unknown real target, reported for comparison only + - empirical_Mk0: shuffled prime gaps; known order 0 + - empirical_Mk1: prime-trained Mk1 surrogate; known order 1 + - empirical_Mk2: prime-trained Mk2 surrogate; known order 2 + - poisson_iid: independent exponential spacings; known order 0 + +Verdict rule: + A control fails recovery when any observable is assigned layer > known order. +""" + +import argparse +import json +from pathlib import Path + +import numpy as np + +from exp_two_layer_universality import ( + OBSERVABLES, + classify_layer, + gen_prime_gaps, + generate_markov_surrogate, +) + + +def measure_all(gaps): + out = {} + for name, fn in OBSERVABLES.items(): + try: + value = float(fn(gaps)) + except Exception: + value = float("nan") + out[name] = value + return out + + +def classifier_pass(gaps, n_surr, rng): + real_obs = measure_all(gaps) + z_scores = {} + + for mk in (0, 1, 2): + surr_obs = {name: [] for name in OBSERVABLES} + for _ in range(n_surr): + if mk == 0: + surr = rng.permutation(gaps) + else: + surr = generate_markov_surrogate(gaps, mk, rng=rng) + for obs_name, obs_fn in OBSERVABLES.items(): + try: + surr_obs[obs_name].append(float(obs_fn(surr))) + except Exception: + pass + + z_mk = {} + for obs_name in OBSERVABLES: + vals = np.asarray(surr_obs[obs_name], dtype=float) + vals = vals[np.isfinite(vals)] + if len(vals) > 2 and np.std(vals) > 1e-12 and np.isfinite(real_obs[obs_name]): + z = (real_obs[obs_name] - np.mean(vals)) / np.std(vals) + else: + z = 0.0 + z_mk[obs_name] = round(float(z), 2) + z_scores[f"Mk{mk}"] = z_mk + + layers = {} + for obs_name in OBSERVABLES: + layers[obs_name] = classify_layer( + z_scores["Mk0"][obs_name], + z_scores["Mk1"][obs_name], + z_scores["Mk2"][obs_name], + ) + + return real_obs, z_scores, layers + + +def build_controls(prime_gaps, rng): + return { + "prime_gaps": { + "known_order": None, + "gaps": prime_gaps, + }, + "empirical_Mk0": { + "known_order": 0, + "gaps": rng.permutation(prime_gaps), + }, + "empirical_Mk1": { + "known_order": 1, + "gaps": generate_markov_surrogate(prime_gaps, 1, rng=rng), + }, + "empirical_Mk2": { + "known_order": 2, + "gaps": generate_markov_surrogate(prime_gaps, 2, rng=rng), + }, + "poisson_iid": { + "known_order": 0, + "gaps": rng.exponential(1.0, len(prime_gaps)), + }, + } + + +def summarize_failure(layers, known_order): + if known_order is None: + return { + "status": "target_only", + "over_layer_observables": [], + "max_layer": int(max(layers.values())), + } + + over = [name for name, layer in layers.items() if layer > known_order] + return { + "status": "PASS" if not over else "FAIL", + "over_layer_observables": over, + "max_layer": int(max(layers.values())), + } + + +def run(N=60000, n_surr=20, seed=20260504): + rng = np.random.default_rng(seed) + prime_gaps = gen_prime_gaps(N).astype(float) + controls = build_controls(prime_gaps, rng) + + results = { + "N": int(N), + "n_surr": int(n_surr), + "seed": int(seed), + "controls": {}, + } + + print(f"N={N}, n_surr={n_surr}, seed={seed}") + print(f"{'sequence':<16} {'known':<7} {'status':<8} {'maxL':<5} over-layer observables") + print("-" * 82) + + for name, spec in controls.items(): + real_obs, z_scores, layers = classifier_pass(spec["gaps"], n_surr, rng) + summary = summarize_failure(layers, spec["known_order"]) + known = "target" if spec["known_order"] is None else f"L{spec['known_order']}" + over = ", ".join(summary["over_layer_observables"]) or "-" + print(f"{name:<16} {known:<7} {summary['status']:<8} L{summary['max_layer']:<4} {over}") + + results["controls"][name] = { + "known_order": spec["known_order"], + "n_gaps": int(len(spec["gaps"])), + "real_obs": {k: round(v, 6) for k, v in real_obs.items()}, + "z_scores": z_scores, + "layers": layers, + "recovery": summary, + } + + out_path = Path("tools/data/markov_layer_recovery_audit.json") + with out_path.open("w") as f: + json.dump(results, f, indent=2) + print(f"\nSaved to {out_path}") + return results + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--N", type=int, default=60000) + parser.add_argument("--n_surr", type=int, default=20) + parser.add_argument("--seed", type=int, default=20260504) + args = parser.parse_args() + run(N=args.N, n_surr=args.n_surr, seed=args.seed) + + +if __name__ == "__main__": + main() exec /bin/bash -lc 'python tools/lab_falsifier.py --report tools/data/reports/agent_20260504_1219.md --output tools/data/reports/falsifier_20260504_1219.json' in /opt/MM_D-ND diff --git a/tools/data/reports/agent_20260504_1219.md b/tools/data/reports/agent_20260504_1219.md new file mode 100644 index 0000000000000000000000000000000000000000..b3c10211f3b81595e76259db054509f891c79118 --- /dev/null +++ b/tools/data/reports/agent_20260504_1219.md @@ -0,0 +1,104 @@ +# Agent Report — Markov Layers Pass the First Recovery Gate, but Not All Observables Pass +**Date**: 2026-05-04 12:19 +**Piano**: 61 +**Tension explored**: META (0.5) + +## Claim Under Test +> Tutti i 11 test passano — verifica che non stiamo testando solo tautologie. + +## Question +Se la pipeline Mk0/Mk1/Mk2 vede due layer, li recupera anche quando l'ordine vero della sequenza è noto? + +## Experiment Design +- **Primary audit**: `tools/exp_markov_layer_recovery_audit.py` +- **Scope**: 60,000 gap/spacings per sequenza; 20 surrogati per livello Mk; seed `20260504`. +- **Target**: prime gaps, ordine sconosciuto, solo confronto. +- **Known-order controls**: + - `empirical_Mk0`: permutazione dei gap primi, ordine noto 0. + - `empirical_Mk1`: surrogate Mk1 addestrato sui gap primi, ordine noto 1. + - `empirical_Mk2`: surrogate Mk2 addestrato sui gap primi, ordine noto 2. + - `poisson_iid`: spacings esponenziali iid, ordine noto 0. +- **Recovery rule**: un controllo passa se nessun osservabile viene classificato con layer maggiore dell'ordine noto. +- **Secondary calibration**: `tools/exp_meta_tautology_test.py --n-primes 300000 --n-trials 12`. Nota: il controllo chiamato `HL_markov` nello script è un AR(1) gap model che preserva lag-1, non un modello Hardy-Littlewood aritmetico. + +## Results + +### Recovery audit + +| Sequence | Known order | Recovery | Max layer | Over-layer observables | +|---|---:|---|---:|---| +| `prime_gaps` | target | target only | 3 | - | +| `empirical_Mk0` | 0 | PASS | 0 | - | +| `empirical_Mk1` | 1 | PASS | 1 | - | +| `empirical_Mk2` | 2 | FAIL | 3 | `cond_entropy` | +| `poisson_iid` | 0 | FAIL | 1 | `num_var_10` | + +Prime target layers, reported without recovery verdict: + +| Observable | Layer | z vs Mk0 | z vs Mk1 | z vs Mk2 | +|---|---:|---:|---:|---:| +| SR | 1 | -22.21 | -0.08 | -0.16 | +| L1 | 1 | -12.25 | 0.79 | 0.86 | +| L2 | 2 | -3.60 | -4.09 | -0.19 | +| SR2 | 2 | -6.67 | -8.80 | -0.33 | +| cond_entropy | 3 | -435.40 | -26.11 | 2.56 | +| triple_var | 1 | -15.90 | -0.53 | 0.36 | +| num_var_10 | 1 | -7.46 | -1.42 | 0.00 | +| run_length | 3 | 27.59 | 7.19 | 3.43 | + +Control failure details: + +| Control | Observable | Expected max | Assigned | Diagnostic z | +|---|---|---:|---:|---:| +| `empirical_Mk2` | `cond_entropy` | 2 | 3 | z vs Mk2 = 3.15 | +| `poisson_iid` | `num_var_10` | 0 | 1 | z vs Mk0 = 2.13 | + +### Secondary tautology calibration + +| Observable | Verdict in tested controls | Blocking control | +|---|---|---| +| r-stat | STRUCTURAL | none in this run | +| mod3 fraction | STRUCTURAL | none in this run | +| lag1 ACF | TAUTOLOGICAL | lag1-matched AR(1), z = -0.62 | +| dr_ord | TAUTOLOGICAL | lag1-matched AR(1), z = -1.90 | +| dr_mag | STRUCTURAL | none in this run | + +Numeri principali: prime mod3 same-class fraction = 0.158032. Questo è bias forte rispetto ai controlli ~0.34-0.35, non proibizione zero. + +## Key Findings +1. **Il nucleo pair/triple non cade al primo audit.** `empirical_Mk0` recupera solo Layer 0; `empirical_Mk1` recupera solo Layer 1. Questo supporta l'uso di SR e L1 come Layer 1 e di L2/SR2 come Layer 2 nel perimetro testato. +2. **`cond_entropy` non può essere usata come prova forte di Layer 3 nel setup attuale.** Anche un controllo generato come Mk2 viene letto Layer 3 per `cond_entropy` (z vs Mk2 = 3.15). Questo indica bias del surrogate/binning o varianza finita, non necessariamente memoria reale oltre Mk2. +3. **`num_var_10` produce un falso positivo debole su iid.** `poisson_iid` viene letto Layer 1 per `num_var_10` con z = 2.13. Il segnale supera appena la soglia operativa, quindi va trattato come diagnostica secondaria finché non passa audit multi-seed. +4. **Il vecchio controllo `HL_markov` va rinominato nel linguaggio dei report.** Lo script non implementa Hardy-Littlewood; implementa un AR(1) lag1-matched. I claim futuri non devono chiamarlo controllo HL aritmetico. + +## Verdict +**CONSTRAINT on META + BOUNDARY**: la decomposizione two-layer è utilizzabile solo con perimetro atomico. + +Perimetro corretto: nei 60,000 gap primi testati con 20 surrogati Mk per livello, il nucleo `SR,L1` viene catturato da Mk1 e il nucleo `L2,SR2` da Mk2. Le diagnostiche `cond_entropy`, `num_var_10`, `run_length` non sono evidenza primaria finché non passano un recovery audit multi-seed e, idealmente, surrogati Markov esatti. + +Non è una scoperta nuova di teoria Markov: è un audit metodologico vicino ai test di ordine Markov con surrogate data. La letteratura rilevante include test esatti di ordine Markov e surrogate vincolati che preservano proprietà Markov di ordine n, quindi il prossimo strumento deve avvicinarsi a quei null esatti invece di usare solo generatori bin-pool approssimati. + +## Bicono della scoperta +- **Due radici**: layer osservato dai dati · ordine noto del generatore. +- **Singolare**: la pipeline Mk stessa, prima che venga usata come prova su un dominio reale. +- **Invariante di passaggio**: un claim di memoria sopravvive solo se recupera correttamente i controlli con ordine noto. +- **Campo di possibilità**: qui diventa possibile separare nucleo robusto e diagnostiche sovrastimate; qui diventa non-possibile usare `cond_entropy` o `num_var_10` come prova primaria di profondità senza recovery audit. + +## Self-Falsification — 5 lenti +- **L1 hard constraint vs bias**: nessun claim "zero/never/always". La frazione mod3 è 0.158032, dichiarata bias, non proibizione. +- **L2 quantità vs ratio**: il claim principale usa layer recuperato e z-score, non percentuali fra spazi di taglia diversa. +- **L3 no silent patching**: dichiaro esplicitamente che `HL_markov` era nome scorretto per un controllo AR(1) lag1-matched. +- **L4 edge cases**: i fallimenti `cond_entropy` e `num_var_10` non vengono arrotondati via; diventano il vincolo. +- **L5 rediscovery**: non taggo NEW. Riferimenti classici vicini: surrogate-data tests for Markov order e constrained Markov-order surrogates. + +## References +- Kandel et al.-lineage / surrogate tests: ["Testing the order of discrete Markov chains using surrogate data", Physica D 117, 1998](https://www.sciencedirect.com/science/article/pii/S0167278997003151). +- ["Exact significance test for Markov order", Physica D 269, 2014](https://www.sciencedirect.com/science/article/pii/S0167278913003230). +- ["Constrained Markov order surrogates", Physica D 2020](https://www.sciencedirect.com/science/article/pii/S0167278919305652). + +## Files +- `tools/exp_markov_layer_recovery_audit.py` +- `tools/data/markov_layer_recovery_audit.json` +- `tools/data/meta_tautology_test.json` +- `tools/data/seme.json` +- `tools/data/reports/agent_20260504_1219.md` diff --git a/tools/exp_markov_layer_recovery_audit.py b/tools/exp_markov_layer_recovery_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..4faa72c630148cdf498c3cc2d626cca2a78efd9a --- /dev/null +++ b/tools/exp_markov_layer_recovery_audit.py @@ -0,0 +1,179 @@ +#!/usr/bin/env python3 +""" +exp_markov_layer_recovery_audit.py - META audit for the Markov-layer classifier. + +Question: + If a sequence is generated with known Markov order k, does the existing + Mk0/Mk1/Mk2 classifier recover layer <= k? + +Why this matters: + Recent reports used Mk surrogate capture to claim two visible memory layers. + This audit tests the opposite: whether the classifier itself creates apparent + higher layers on finite Markov controls. + +Controls: + - prime_gaps: unknown real target, reported for comparison only + - empirical_Mk0: shuffled prime gaps; known order 0 + - empirical_Mk1: prime-trained Mk1 surrogate; known order 1 + - empirical_Mk2: prime-trained Mk2 surrogate; known order 2 + - poisson_iid: independent exponential spacings; known order 0 + +Verdict rule: + A control fails recovery when any observable is assigned layer > known order. +""" + +import argparse +import json +from pathlib import Path + +import numpy as np + +from exp_two_layer_universality import ( + OBSERVABLES, + classify_layer, + gen_prime_gaps, + generate_markov_surrogate, +) + + +def measure_all(gaps): + out = {} + for name, fn in OBSERVABLES.items(): + try: + value = float(fn(gaps)) + except Exception: + value = float("nan") + out[name] = value + return out + + +def classifier_pass(gaps, n_surr, rng): + real_obs = measure_all(gaps) + z_scores = {} + + for mk in (0, 1, 2): + surr_obs = {name: [] for name in OBSERVABLES} + for _ in range(n_surr): + if mk == 0: + surr = rng.permutation(gaps) + else: + surr = generate_markov_surrogate(gaps, mk, rng=rng) + for obs_name, obs_fn in OBSERVABLES.items(): + try: + surr_obs[obs_name].append(float(obs_fn(surr))) + except Exception: + pass + + z_mk = {} + for obs_name in OBSERVABLES: + vals = np.asarray(surr_obs[obs_name], dtype=float) + vals = vals[np.isfinite(vals)] + if len(vals) > 2 and np.std(vals) > 1e-12 and np.isfinite(real_obs[obs_name]): + z = (real_obs[obs_name] - np.mean(vals)) / np.std(vals) + else: + z = 0.0 + z_mk[obs_name] = round(float(z), 2) + z_scores[f"Mk{mk}"] = z_mk + + layers = {} + for obs_name in OBSERVABLES: + layers[obs_name] = classify_layer( + z_scores["Mk0"][obs_name], + z_scores["Mk1"][obs_name], + z_scores["Mk2"][obs_name], + ) + + return real_obs, z_scores, layers + + +def build_controls(prime_gaps, rng): + return { + "prime_gaps": { + "known_order": None, + "gaps": prime_gaps, + }, + "empirical_Mk0": { + "known_order": 0, + "gaps": rng.permutation(prime_gaps), + }, + "empirical_Mk1": { + "known_order": 1, + "gaps": generate_markov_surrogate(prime_gaps, 1, rng=rng), + }, + "empirical_Mk2": { + "known_order": 2, + "gaps": generate_markov_surrogate(prime_gaps, 2, rng=rng), + }, + "poisson_iid": { + "known_order": 0, + "gaps": rng.exponential(1.0, len(prime_gaps)), + }, + } + + +def summarize_failure(layers, known_order): + if known_order is None: + return { + "status": "target_only", + "over_layer_observables": [], + "max_layer": int(max(layers.values())), + } + + over = [name for name, layer in layers.items() if layer > known_order] + return { + "status": "PASS" if not over else "FAIL", + "over_layer_observables": over, + "max_layer": int(max(layers.values())), + } + + +def run(N=60000, n_surr=20, seed=20260504): + rng = np.random.default_rng(seed) + prime_gaps = gen_prime_gaps(N).astype(float) + controls = build_controls(prime_gaps, rng) + + results = { + "N": int(N), + "n_surr": int(n_surr), + "seed": int(seed), + "controls": {}, + } + + print(f"N={N}, n_surr={n_surr}, seed={seed}") + print(f"{'sequence':<16} {'known':<7} {'status':<8} {'maxL':<5} over-layer observables") + print("-" * 82) + + for name, spec in controls.items(): + real_obs, z_scores, layers = classifier_pass(spec["gaps"], n_surr, rng) + summary = summarize_failure(layers, spec["known_order"]) + known = "target" if spec["known_order"] is None else f"L{spec['known_order']}" + over = ", ".join(summary["over_layer_observables"]) or "-" + print(f"{name:<16} {known:<7} {summary['status']:<8} L{summary['max_layer']:<4} {over}") + + results["controls"][name] = { + "known_order": spec["known_order"], + "n_gaps": int(len(spec["gaps"])), + "real_obs": {k: round(v, 6) for k, v in real_obs.items()}, + "z_scores": z_scores, + "layers": layers, + "recovery": summary, + } + + out_path = Path("tools/data/markov_layer_recovery_audit.json") + with out_path.open("w") as f: + json.dump(results, f, indent=2) + print(f"\nSaved to {out_path}") + return results + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--N", type=int, default=60000) + parser.add_argument("--n_surr", type=int, default=20) + parser.add_argument("--seed", type=int, default=20260504) + args = parser.parse_args() + run(N=args.N, n_surr=args.n_surr, seed=args.seed) + + +if __name__ == "__main__": + main() diff --git a/tools/data/reports/agent_20260504_1219.md b/tools/data/reports/agent_20260504_1219.md new file mode 100644 index 0000000000000000000000000000000000000000..b3c10211f3b81595e76259db054509f891c79118 --- /dev/null +++ b/tools/data/reports/agent_20260504_1219.md @@ -0,0 +1,104 @@ +# Agent Report — Markov Layers Pass the First Recovery Gate, but Not All Observables Pass +**Date**: 2026-05-04 12:19 +**Piano**: 61 +**Tension explored**: META (0.5) + +## Claim Under Test +> Tutti i 11 test passano — verifica che non stiamo testando solo tautologie. + +## Question +Se la pipeline Mk0/Mk1/Mk2 vede due layer, li recupera anche quando l'ordine vero della sequenza è noto? + +## Experiment Design +- **Primary audit**: `tools/exp_markov_layer_recovery_audit.py` +- **Scope**: 60,000 gap/spacings per sequenza; 20 surrogati per livello Mk; seed `20260504`. +- **Target**: prime gaps, ordine sconosciuto, solo confronto. +- **Known-order controls**: + - `empirical_Mk0`: permutazione dei gap primi, ordine noto 0. + - `empirical_Mk1`: surrogate Mk1 addestrato sui gap primi, ordine noto 1. + - `empirical_Mk2`: surrogate Mk2 addestrato sui gap primi, ordine noto 2. + - `poisson_iid`: spacings esponenziali iid, ordine noto 0. +- **Recovery rule**: un controllo passa se nessun osservabile viene classificato con layer maggiore dell'ordine noto. +- **Secondary calibration**: `tools/exp_meta_tautology_test.py --n-primes 300000 --n-trials 12`. Nota: il controllo chiamato `HL_markov` nello script è un AR(1) gap model che preserva lag-1, non un modello Hardy-Littlewood aritmetico. + +## Results + +### Recovery audit + +| Sequence | Known order | Recovery | Max layer | Over-layer observables | +|---|---:|---|---:|---| +| `prime_gaps` | target | target only | 3 | - | +| `empirical_Mk0` | 0 | PASS | 0 | - | +| `empirical_Mk1` | 1 | PASS | 1 | - | +| `empirical_Mk2` | 2 | FAIL | 3 | `cond_entropy` | +| `poisson_iid` | 0 | FAIL | 1 | `num_var_10` | + +Prime target layers, reported without recovery verdict: + +| Observable | Layer | z vs Mk0 | z vs Mk1 | z vs Mk2 | +|---|---:|---:|---:|---:| +| SR | 1 | -22.21 | -0.08 | -0.16 | +| L1 | 1 | -12.25 | 0.79 | 0.86 | +| L2 | 2 | -3.60 | -4.09 | -0.19 | +| SR2 | 2 | -6.67 | -8.80 | -0.33 | +| cond_entropy | 3 | -435.40 | -26.11 | 2.56 | +| triple_var | 1 | -15.90 | -0.53 | 0.36 | +| num_var_10 | 1 | -7.46 | -1.42 | 0.00 | +| run_length | 3 | 27.59 | 7.19 | 3.43 | + +Control failure details: + +| Control | Observable | Expected max | Assigned | Diagnostic z | +|---|---|---:|---:|---:| +| `empirical_Mk2` | `cond_entropy` | 2 | 3 | z vs Mk2 = 3.15 | +| `poisson_iid` | `num_var_10` | 0 | 1 | z vs Mk0 = 2.13 | + +### Secondary tautology calibration + +| Observable | Verdict in tested controls | Blocking control | +|---|---|---| +| r-stat | STRUCTURAL | none in this run | +| mod3 fraction | STRUCTURAL | none in this run | +| lag1 ACF | TAUTOLOGICAL | lag1-matched AR(1), z = -0.62 | +| dr_ord | TAUTOLOGICAL | lag1-matched AR(1), z = -1.90 | +| dr_mag | STRUCTURAL | none in this run | + +Numeri principali: prime mod3 same-class fraction = 0.158032. Questo è bias forte rispetto ai controlli ~0.34-0.35, non proibizione zero. + +## Key Findings +1. **Il nucleo pair/triple non cade al primo audit.** `empirical_Mk0` recupera solo Layer 0; `empirical_Mk1` recupera solo Layer 1. Questo supporta l'uso di SR e L1 come Layer 1 e di L2/SR2 come Layer 2 nel perimetro testato. +2. **`cond_entropy` non può essere usata come prova forte di Layer 3 nel setup attuale.** Anche un controllo generato come Mk2 viene letto Layer 3 per `cond_entropy` (z vs Mk2 = 3.15). Questo indica bias del surrogate/binning o varianza finita, non necessariamente memoria reale oltre Mk2. +3. **`num_var_10` produce un falso positivo debole su iid.** `poisson_iid` viene letto Layer 1 per `num_var_10` con z = 2.13. Il segnale supera appena la soglia operativa, quindi va trattato come diagnostica secondaria finché non passa audit multi-seed. +4. **Il vecchio controllo `HL_markov` va rinominato nel linguaggio dei report.** Lo script non implementa Hardy-Littlewood; implementa un AR(1) lag1-matched. I claim futuri non devono chiamarlo controllo HL aritmetico. + +## Verdict +**CONSTRAINT on META + BOUNDARY**: la decomposizione two-layer è utilizzabile solo con perimetro atomico. + +Perimetro corretto: nei 60,000 gap primi testati con 20 surrogati Mk per livello, il nucleo `SR,L1` viene catturato da Mk1 e il nucleo `L2,SR2` da Mk2. Le diagnostiche `cond_entropy`, `num_var_10`, `run_length` non sono evidenza primaria finché non passano un recovery audit multi-seed e, idealmente, surrogati Markov esatti. + +Non è una scoperta nuova di teoria Markov: è un audit metodologico vicino ai test di ordine Markov con surrogate data. La letteratura rilevante include test esatti di ordine Markov e surrogate vincolati che preservano proprietà Markov di ordine n, quindi il prossimo strumento deve avvicinarsi a quei null esatti invece di usare solo generatori bin-pool approssimati. + +## Bicono della scoperta +- **Due radici**: layer osservato dai dati · ordine noto del generatore. +- **Singolare**: la pipeline Mk stessa, prima che venga usata come prova su un dominio reale. +- **Invariante di passaggio**: un claim di memoria sopravvive solo se recupera correttamente i controlli con ordine noto. +- **Campo di possibilità**: qui diventa possibile separare nucleo robusto e diagnostiche sovrastimate; qui diventa non-possibile usare `cond_entropy` o `num_var_10` come prova primaria di profondità senza recovery audit. + +## Self-Falsification — 5 lenti +- **L1 hard constraint vs bias**: nessun claim "zero/never/always". La frazione mod3 è 0.158032, dichiarata bias, non proibizione. +- **L2 quantità vs ratio**: il claim principale usa layer recuperato e z-score, non percentuali fra spazi di taglia diversa. +- **L3 no silent patching**: dichiaro esplicitamente che `HL_markov` era nome scorretto per un controllo AR(1) lag1-matched. +- **L4 edge cases**: i fallimenti `cond_entropy` e `num_var_10` non vengono arrotondati via; diventano il vincolo. +- **L5 rediscovery**: non taggo NEW. Riferimenti classici vicini: surrogate-data tests for Markov order e constrained Markov-order surrogates. + +## References +- Kandel et al.-lineage / surrogate tests: ["Testing the order of discrete Markov chains using surrogate data", Physica D 117, 1998](https://www.sciencedirect.com/science/article/pii/S0167278997003151). +- ["Exact significance test for Markov order", Physica D 269, 2014](https://www.sciencedirect.com/science/article/pii/S0167278913003230). +- ["Constrained Markov order surrogates", Physica D 2020](https://www.sciencedirect.com/science/article/pii/S0167278919305652). + +## Files +- `tools/exp_markov_layer_recovery_audit.py` +- `tools/data/markov_layer_recovery_audit.json` +- `tools/data/meta_tautology_test.json` +- `tools/data/seme.json` +- `tools/data/reports/agent_20260504_1219.md` diff --git a/tools/exp_markov_layer_recovery_audit.py b/tools/exp_markov_layer_recovery_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..4faa72c630148cdf498c3cc2d626cca2a78efd9a --- /dev/null +++ b/tools/exp_markov_layer_recovery_audit.py @@ -0,0 +1,179 @@ +#!/usr/bin/env python3 +""" +exp_markov_layer_recovery_audit.py - META audit for the Markov-layer classifier. + +Question: + If a sequence is generated with known Markov order k, does the existing + Mk0/Mk1/Mk2 classifier recover layer <= k? + +Why this matters: + Recent reports used Mk surrogate capture to claim two visible memory layers. + This audit tests the opposite: whether the classifier itself creates apparent + higher layers on finite Markov controls. + +Controls: + - prime_gaps: unknown real target, reported for comparison only + - empirical_Mk0: shuffled prime gaps; known order 0 + - empirical_Mk1: prime-trained Mk1 surrogate; known order 1 + - empirical_Mk2: prime-trained Mk2 surrogate; known order 2 + - poisson_iid: independent exponential spacings; known order 0 + +Verdict rule: + A control fails recovery when any observable is assigned layer > known order. +""" + +import argparse +import json +from pathlib import Path + +import numpy as np + +from exp_two_layer_universality import ( + OBSERVABLES, + classify_layer, + gen_prime_gaps, + generate_markov_surrogate, +) + + +def measure_all(gaps): + out = {} + for name, fn in OBSERVABLES.items(): + try: + value = float(fn(gaps)) + except Exception: + value = float("nan") + out[name] = value + return out + + +def classifier_pass(gaps, n_surr, rng): + real_obs = measure_all(gaps) + z_scores = {} + + for mk in (0, 1, 2): + surr_obs = {name: [] for name in OBSERVABLES} + for _ in range(n_surr): + if mk == 0: + surr = rng.permutation(gaps) + else: + surr = generate_markov_surrogate(gaps, mk, rng=rng) + for obs_name, obs_fn in OBSERVABLES.items(): + try: + surr_obs[obs_name].append(float(obs_fn(surr))) + except Exception: + pass + + z_mk = {} + for obs_name in OBSERVABLES: + vals = np.asarray(surr_obs[obs_name], dtype=float) + vals = vals[np.isfinite(vals)] + if len(vals) > 2 and np.std(vals) > 1e-12 and np.isfinite(real_obs[obs_name]): + z = (real_obs[obs_name] - np.mean(vals)) / np.std(vals) + else: + z = 0.0 + z_mk[obs_name] = round(float(z), 2) + z_scores[f"Mk{mk}"] = z_mk + + layers = {} + for obs_name in OBSERVABLES: + layers[obs_name] = classify_layer( + z_scores["Mk0"][obs_name], + z_scores["Mk1"][obs_name], + z_scores["Mk2"][obs_name], + ) + + return real_obs, z_scores, layers + + +def build_controls(prime_gaps, rng): + return { + "prime_gaps": { + "known_order": None, + "gaps": prime_gaps, + }, + "empirical_Mk0": { + "known_order": 0, + "gaps": rng.permutation(prime_gaps), + }, + "empirical_Mk1": { + "known_order": 1, + "gaps": generate_markov_surrogate(prime_gaps, 1, rng=rng), + }, + "empirical_Mk2": { + "known_order": 2, + "gaps": generate_markov_surrogate(prime_gaps, 2, rng=rng), + }, + "poisson_iid": { + "known_order": 0, + "gaps": rng.exponential(1.0, len(prime_gaps)), + }, + } + + +def summarize_failure(layers, known_order): + if known_order is None: + return { + "status": "target_only", + "over_layer_observables": [], + "max_layer": int(max(layers.values())), + } + + over = [name for name, layer in layers.items() if layer > known_order] + return { + "status": "PASS" if not over else "FAIL", + "over_layer_observables": over, + "max_layer": int(max(layers.values())), + } + + +def run(N=60000, n_surr=20, seed=20260504): + rng = np.random.default_rng(seed) + prime_gaps = gen_prime_gaps(N).astype(float) + controls = build_controls(prime_gaps, rng) + + results = { + "N": int(N), + "n_surr": int(n_surr), + "seed": int(seed), + "controls": {}, + } + + print(f"N={N}, n_surr={n_surr}, seed={seed}") + print(f"{'sequence':<16} {'known':<7} {'status':<8} {'maxL':<5} over-layer observables") + print("-" * 82) + + for name, spec in controls.items(): + real_obs, z_scores, layers = classifier_pass(spec["gaps"], n_surr, rng) + summary = summarize_failure(layers, spec["known_order"]) + known = "target" if spec["known_order"] is None else f"L{spec['known_order']}" + over = ", ".join(summary["over_layer_observables"]) or "-" + print(f"{name:<16} {known:<7} {summary['status']:<8} L{summary['max_layer']:<4} {over}") + + results["controls"][name] = { + "known_order": spec["known_order"], + "n_gaps": int(len(spec["gaps"])), + "real_obs": {k: round(v, 6) for k, v in real_obs.items()}, + "z_scores": z_scores, + "layers": layers, + "recovery": summary, + } + + out_path = Path("tools/data/markov_layer_recovery_audit.json") + with out_path.open("w") as f: + json.dump(results, f, indent=2) + print(f"\nSaved to {out_path}") + return results + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--N", type=int, default=60000) + parser.add_argument("--n_surr", type=int, default=20) + parser.add_argument("--seed", type=int, default=20260504) + args = parser.parse_args() + run(N=args.N, n_surr=args.n_surr, seed=args.seed) + + +if __name__ == "__main__": + main() diff --git a/tools/data/reports/agent_20260504_1219.md b/tools/data/reports/agent_20260504_1219.md new file mode 100644 index 0000000000000000000000000000000000000000..b3c10211f3b81595e76259db054509f891c79118 --- /dev/null +++ b/tools/data/reports/agent_20260504_1219.md @@ -0,0 +1,104 @@ +# Agent Report — Markov Layers Pass the First Recovery Gate, but Not All Observables Pass +**Date**: 2026-05-04 12:19 +**Piano**: 61 +**Tension explored**: META (0.5) + +## Claim Under Test +> Tutti i 11 test passano — verifica che non stiamo testando solo tautologie. + +## Question +Se la pipeline Mk0/Mk1/Mk2 vede due layer, li recupera anche quando l'ordine vero della sequenza è noto? + +## Experiment Design +- **Primary audit**: `tools/exp_markov_layer_recovery_audit.py` +- **Scope**: 60,000 gap/spacings per sequenza; 20 surrogati per livello Mk; seed `20260504`. +- **Target**: prime gaps, ordine sconosciuto, solo confronto. +- **Known-order controls**: + - `empirical_Mk0`: permutazione dei gap primi, ordine noto 0. + - `empirical_Mk1`: surrogate Mk1 addestrato sui gap primi, ordine noto 1. + - `empirical_Mk2`: surrogate Mk2 addestrato sui gap primi, ordine noto 2. + - `poisson_iid`: spacings esponenziali iid, ordine noto 0. +- **Recovery rule**: un controllo passa se nessun osservabile viene classificato con layer maggiore dell'ordine noto. +- **Secondary calibration**: `tools/exp_meta_tautology_test.py --n-primes 300000 --n-trials 12`. Nota: il controllo chiamato `HL_markov` nello script è un AR(1) gap model che preserva lag-1, non un modello Hardy-Littlewood aritmetico. + +## Results + +### Recovery audit + +| Sequence | Known order | Recovery | Max layer | Over-layer observables | +|---|---:|---|---:|---| +| `prime_gaps` | target | target only | 3 | - | +| `empirical_Mk0` | 0 | PASS | 0 | - | +| `empirical_Mk1` | 1 | PASS | 1 | - | +| `empirical_Mk2` | 2 | FAIL | 3 | `cond_entropy` | +| `poisson_iid` | 0 | FAIL | 1 | `num_var_10` | + +Prime target layers, reported without recovery verdict: + +| Observable | Layer | z vs Mk0 | z vs Mk1 | z vs Mk2 | +|---|---:|---:|---:|---:| +| SR | 1 | -22.21 | -0.08 | -0.16 | +| L1 | 1 | -12.25 | 0.79 | 0.86 | +| L2 | 2 | -3.60 | -4.09 | -0.19 | +| SR2 | 2 | -6.67 | -8.80 | -0.33 | +| cond_entropy | 3 | -435.40 | -26.11 | 2.56 | +| triple_var | 1 | -15.90 | -0.53 | 0.36 | +| num_var_10 | 1 | -7.46 | -1.42 | 0.00 | +| run_length | 3 | 27.59 | 7.19 | 3.43 | + +Control failure details: + +| Control | Observable | Expected max | Assigned | Diagnostic z | +|---|---|---:|---:|---:| +| `empirical_Mk2` | `cond_entropy` | 2 | 3 | z vs Mk2 = 3.15 | +| `poisson_iid` | `num_var_10` | 0 | 1 | z vs Mk0 = 2.13 | + +### Secondary tautology calibration + +| Observable | Verdict in tested controls | Blocking control | +|---|---|---| +| r-stat | STRUCTURAL | none in this run | +| mod3 fraction | STRUCTURAL | none in this run | +| lag1 ACF | TAUTOLOGICAL | lag1-matched AR(1), z = -0.62 | +| dr_ord | TAUTOLOGICAL | lag1-matched AR(1), z = -1.90 | +| dr_mag | STRUCTURAL | none in this run | + +Numeri principali: prime mod3 same-class fraction = 0.158032. Questo è bias forte rispetto ai controlli ~0.34-0.35, non proibizione zero. + +## Key Findings +1. **Il nucleo pair/triple non cade al primo audit.** `empirical_Mk0` recupera solo Layer 0; `empirical_Mk1` recupera solo Layer 1. Questo supporta l'uso di SR e L1 come Layer 1 e di L2/SR2 come Layer 2 nel perimetro testato. +2. **`cond_entropy` non può essere usata come prova forte di Layer 3 nel setup attuale.** Anche un controllo generato come Mk2 viene letto Layer 3 per `cond_entropy` (z vs Mk2 = 3.15). Questo indica bias del surrogate/binning o varianza finita, non necessariamente memoria reale oltre Mk2. +3. **`num_var_10` produce un falso positivo debole su iid.** `poisson_iid` viene letto Layer 1 per `num_var_10` con z = 2.13. Il segnale supera appena la soglia operativa, quindi va trattato come diagnostica secondaria finché non passa audit multi-seed. +4. **Il vecchio controllo `HL_markov` va rinominato nel linguaggio dei report.** Lo script non implementa Hardy-Littlewood; implementa un AR(1) lag1-matched. I claim futuri non devono chiamarlo controllo HL aritmetico. + +## Verdict +**CONSTRAINT on META + BOUNDARY**: la decomposizione two-layer è utilizzabile solo con perimetro atomico. + +Perimetro corretto: nei 60,000 gap primi testati con 20 surrogati Mk per livello, il nucleo `SR,L1` viene catturato da Mk1 e il nucleo `L2,SR2` da Mk2. Le diagnostiche `cond_entropy`, `num_var_10`, `run_length` non sono evidenza primaria finché non passano un recovery audit multi-seed e, idealmente, surrogati Markov esatti. + +Non è una scoperta nuova di teoria Markov: è un audit metodologico vicino ai test di ordine Markov con surrogate data. La letteratura rilevante include test esatti di ordine Markov e surrogate vincolati che preservano proprietà Markov di ordine n, quindi il prossimo strumento deve avvicinarsi a quei null esatti invece di usare solo generatori bin-pool approssimati. + +## Bicono della scoperta +- **Due radici**: layer osservato dai dati · ordine noto del generatore. +- **Singolare**: la pipeline Mk stessa, prima che venga usata come prova su un dominio reale. +- **Invariante di passaggio**: un claim di memoria sopravvive solo se recupera correttamente i controlli con ordine noto. +- **Campo di possibilità**: qui diventa possibile separare nucleo robusto e diagnostiche sovrastimate; qui diventa non-possibile usare `cond_entropy` o `num_var_10` come prova primaria di profondità senza recovery audit. + +## Self-Falsification — 5 lenti +- **L1 hard constraint vs bias**: nessun claim "zero/never/always". La frazione mod3 è 0.158032, dichiarata bias, non proibizione. +- **L2 quantità vs ratio**: il claim principale usa layer recuperato e z-score, non percentuali fra spazi di taglia diversa. +- **L3 no silent patching**: dichiaro esplicitamente che `HL_markov` era nome scorretto per un controllo AR(1) lag1-matched. +- **L4 edge cases**: i fallimenti `cond_entropy` e `num_var_10` non vengono arrotondati via; diventano il vincolo. +- **L5 rediscovery**: non taggo NEW. Riferimenti classici vicini: surrogate-data tests for Markov order e constrained Markov-order surrogates. + +## References +- Kandel et al.-lineage / surrogate tests: ["Testing the order of discrete Markov chains using surrogate data", Physica D 117, 1998](https://www.sciencedirect.com/science/article/pii/S0167278997003151). +- ["Exact significance test for Markov order", Physica D 269, 2014](https://www.sciencedirect.com/science/article/pii/S0167278913003230). +- ["Constrained Markov order surrogates", Physica D 2020](https://www.sciencedirect.com/science/article/pii/S0167278919305652). + +## Files +- `tools/exp_markov_layer_recovery_audit.py` +- `tools/data/markov_layer_recovery_audit.json` +- `tools/data/meta_tautology_test.json` +- `tools/data/seme.json` +- `tools/data/reports/agent_20260504_1219.md` diff --git a/tools/exp_markov_layer_recovery_audit.py b/tools/exp_markov_layer_recovery_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..4faa72c630148cdf498c3cc2d626cca2a78efd9a --- /dev/null +++ b/tools/exp_markov_layer_recovery_audit.py @@ -0,0 +1,179 @@ +#!/usr/bin/env python3 +""" +exp_markov_layer_recovery_audit.py - META audit for the Markov-layer classifier. + +Question: + If a sequence is generated with known Markov order k, does the existing + Mk0/Mk1/Mk2 classifier recover layer <= k? + +Why this matters: + Recent reports used Mk surrogate capture to claim two visible memory layers. + This audit tests the opposite: whether the classifier itself creates apparent + higher layers on finite Markov controls. + +Controls: + - prime_gaps: unknown real target, reported for comparison only + - empirical_Mk0: shuffled prime gaps; known order 0 + - empirical_Mk1: prime-trained Mk1 surrogate; known order 1 + - empirical_Mk2: prime-trained Mk2 surrogate; known order 2 + - poisson_iid: independent exponential spacings; known order 0 + +Verdict rule: + A control fails recovery when any observable is assigned layer > known order. +""" + +import argparse +import json +from pathlib import Path + +import numpy as np + +from exp_two_layer_universality import ( + OBSERVABLES, + classify_layer, + gen_prime_gaps, + generate_markov_surrogate, +) + + +def measure_all(gaps): + out = {} + for name, fn in OBSERVABLES.items(): + try: + value = float(fn(gaps)) + except Exception: + value = float("nan") + out[name] = value + return out + + +def classifier_pass(gaps, n_surr, rng): + real_obs = measure_all(gaps) + z_scores = {} + + for mk in (0, 1, 2): + surr_obs = {name: [] for name in OBSERVABLES} + for _ in range(n_surr): + if mk == 0: + surr = rng.permutation(gaps) + else: + surr = generate_markov_surrogate(gaps, mk, rng=rng) + for obs_name, obs_fn in OBSERVABLES.items(): + try: + surr_obs[obs_name].append(float(obs_fn(surr))) + except Exception: + pass + + z_mk = {} + for obs_name in OBSERVABLES: + vals = np.asarray(surr_obs[obs_name], dtype=float) + vals = vals[np.isfinite(vals)] + if len(vals) > 2 and np.std(vals) > 1e-12 and np.isfinite(real_obs[obs_name]): + z = (real_obs[obs_name] - np.mean(vals)) / np.std(vals) + else: + z = 0.0 + z_mk[obs_name] = round(float(z), 2) + z_scores[f"Mk{mk}"] = z_mk + + layers = {} + for obs_name in OBSERVABLES: + layers[obs_name] = classify_layer( + z_scores["Mk0"][obs_name], + z_scores["Mk1"][obs_name], + z_scores["Mk2"][obs_name], + ) + + return real_obs, z_scores, layers + + +def build_controls(prime_gaps, rng): + return { + "prime_gaps": { + "known_order": None, + "gaps": prime_gaps, + }, + "empirical_Mk0": { + "known_order": 0, + "gaps": rng.permutation(prime_gaps), + }, + "empirical_Mk1": { + "known_order": 1, + "gaps": generate_markov_surrogate(prime_gaps, 1, rng=rng), + }, + "empirical_Mk2": { + "known_order": 2, + "gaps": generate_markov_surrogate(prime_gaps, 2, rng=rng), + }, + "poisson_iid": { + "known_order": 0, + "gaps": rng.exponential(1.0, len(prime_gaps)), + }, + } + + +def summarize_failure(layers, known_order): + if known_order is None: + return { + "status": "target_only", + "over_layer_observables": [], + "max_layer": int(max(layers.values())), + } + + over = [name for name, layer in layers.items() if layer > known_order] + return { + "status": "PASS" if not over else "FAIL", + "over_layer_observables": over, + "max_layer": int(max(layers.values())), + } + + +def run(N=60000, n_surr=20, seed=20260504): + rng = np.random.default_rng(seed) + prime_gaps = gen_prime_gaps(N).astype(float) + controls = build_controls(prime_gaps, rng) + + results = { + "N": int(N), + "n_surr": int(n_surr), + "seed": int(seed), + "controls": {}, + } + + print(f"N={N}, n_surr={n_surr}, seed={seed}") + print(f"{'sequence':<16} {'known':<7} {'status':<8} {'maxL':<5} over-layer observables") + print("-" * 82) + + for name, spec in controls.items(): + real_obs, z_scores, layers = classifier_pass(spec["gaps"], n_surr, rng) + summary = summarize_failure(layers, spec["known_order"]) + known = "target" if spec["known_order"] is None else f"L{spec['known_order']}" + over = ", ".join(summary["over_layer_observables"]) or "-" + print(f"{name:<16} {known:<7} {summary['status']:<8} L{summary['max_layer']:<4} {over}") + + results["controls"][name] = { + "known_order": spec["known_order"], + "n_gaps": int(len(spec["gaps"])), + "real_obs": {k: round(v, 6) for k, v in real_obs.items()}, + "z_scores": z_scores, + "layers": layers, + "recovery": summary, + } + + out_path = Path("tools/data/markov_layer_recovery_audit.json") + with out_path.open("w") as f: + json.dump(results, f, indent=2) + print(f"\nSaved to {out_path}") + return results + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--N", type=int, default=60000) + parser.add_argument("--n_surr", type=int, default=20) + parser.add_argument("--seed", type=int, default=20260504) + args = parser.parse_args() + run(N=args.N, n_surr=args.n_surr, seed=args.seed) + + +if __name__ == "__main__": + main() codex Il falsifier locale non è ancora arrivato a un verdetto e sembra bloccato nella chiamata interna a Codex. Aspetto il timeout del tool; se fallisce, lo dichiaro come verifica non completata invece di forzare il processo. diff --git a/tools/data/reports/agent_20260504_1219.md b/tools/data/reports/agent_20260504_1219.md new file mode 100644 index 0000000000000000000000000000000000000000..b3c10211f3b81595e76259db054509f891c79118 --- /dev/null +++ b/tools/data/reports/agent_20260504_1219.md @@ -0,0 +1,104 @@ +# Agent Report — Markov Layers Pass the First Recovery Gate, but Not All Observables Pass +**Date**: 2026-05-04 12:19 +**Piano**: 61 +**Tension explored**: META (0.5) + +## Claim Under Test +> Tutti i 11 test passano — verifica che non stiamo testando solo tautologie. + +## Question +Se la pipeline Mk0/Mk1/Mk2 vede due layer, li recupera anche quando l'ordine vero della sequenza è noto? + +## Experiment Design +- **Primary audit**: `tools/exp_markov_layer_recovery_audit.py` +- **Scope**: 60,000 gap/spacings per sequenza; 20 surrogati per livello Mk; seed `20260504`. +- **Target**: prime gaps, ordine sconosciuto, solo confronto. +- **Known-order controls**: + - `empirical_Mk0`: permutazione dei gap primi, ordine noto 0. + - `empirical_Mk1`: surrogate Mk1 addestrato sui gap primi, ordine noto 1. + - `empirical_Mk2`: surrogate Mk2 addestrato sui gap primi, ordine noto 2. + - `poisson_iid`: spacings esponenziali iid, ordine noto 0. +- **Recovery rule**: un controllo passa se nessun osservabile viene classificato con layer maggiore dell'ordine noto. +- **Secondary calibration**: `tools/exp_meta_tautology_test.py --n-primes 300000 --n-trials 12`. Nota: il controllo chiamato `HL_markov` nello script è un AR(1) gap model che preserva lag-1, non un modello Hardy-Littlewood aritmetico. + +## Results + +### Recovery audit + +| Sequence | Known order | Recovery | Max layer | Over-layer observables | +|---|---:|---|---:|---| +| `prime_gaps` | target | target only | 3 | - | +| `empirical_Mk0` | 0 | PASS | 0 | - | +| `empirical_Mk1` | 1 | PASS | 1 | - | +| `empirical_Mk2` | 2 | FAIL | 3 | `cond_entropy` | +| `poisson_iid` | 0 | FAIL | 1 | `num_var_10` | + +Prime target layers, reported without recovery verdict: + +| Observable | Layer | z vs Mk0 | z vs Mk1 | z vs Mk2 | +|---|---:|---:|---:|---:| +| SR | 1 | -22.21 | -0.08 | -0.16 | +| L1 | 1 | -12.25 | 0.79 | 0.86 | +| L2 | 2 | -3.60 | -4.09 | -0.19 | +| SR2 | 2 | -6.67 | -8.80 | -0.33 | +| cond_entropy | 3 | -435.40 | -26.11 | 2.56 | +| triple_var | 1 | -15.90 | -0.53 | 0.36 | +| num_var_10 | 1 | -7.46 | -1.42 | 0.00 | +| run_length | 3 | 27.59 | 7.19 | 3.43 | + +Control failure details: + +| Control | Observable | Expected max | Assigned | Diagnostic z | +|---|---|---:|---:|---:| +| `empirical_Mk2` | `cond_entropy` | 2 | 3 | z vs Mk2 = 3.15 | +| `poisson_iid` | `num_var_10` | 0 | 1 | z vs Mk0 = 2.13 | + +### Secondary tautology calibration + +| Observable | Verdict in tested controls | Blocking control | +|---|---|---| +| r-stat | STRUCTURAL | none in this run | +| mod3 fraction | STRUCTURAL | none in this run | +| lag1 ACF | TAUTOLOGICAL | lag1-matched AR(1), z = -0.62 | +| dr_ord | TAUTOLOGICAL | lag1-matched AR(1), z = -1.90 | +| dr_mag | STRUCTURAL | none in this run | + +Numeri principali: prime mod3 same-class fraction = 0.158032. Questo è bias forte rispetto ai controlli ~0.34-0.35, non proibizione zero. + +## Key Findings +1. **Il nucleo pair/triple non cade al primo audit.** `empirical_Mk0` recupera solo Layer 0; `empirical_Mk1` recupera solo Layer 1. Questo supporta l'uso di SR e L1 come Layer 1 e di L2/SR2 come Layer 2 nel perimetro testato. +2. **`cond_entropy` non può essere usata come prova forte di Layer 3 nel setup attuale.** Anche un controllo generato come Mk2 viene letto Layer 3 per `cond_entropy` (z vs Mk2 = 3.15). Questo indica bias del surrogate/binning o varianza finita, non necessariamente memoria reale oltre Mk2. +3. **`num_var_10` produce un falso positivo debole su iid.** `poisson_iid` viene letto Layer 1 per `num_var_10` con z = 2.13. Il segnale supera appena la soglia operativa, quindi va trattato come diagnostica secondaria finché non passa audit multi-seed. +4. **Il vecchio controllo `HL_markov` va rinominato nel linguaggio dei report.** Lo script non implementa Hardy-Littlewood; implementa un AR(1) lag1-matched. I claim futuri non devono chiamarlo controllo HL aritmetico. + +## Verdict +**CONSTRAINT on META + BOUNDARY**: la decomposizione two-layer è utilizzabile solo con perimetro atomico. + +Perimetro corretto: nei 60,000 gap primi testati con 20 surrogati Mk per livello, il nucleo `SR,L1` viene catturato da Mk1 e il nucleo `L2,SR2` da Mk2. Le diagnostiche `cond_entropy`, `num_var_10`, `run_length` non sono evidenza primaria finché non passano un recovery audit multi-seed e, idealmente, surrogati Markov esatti. + +Non è una scoperta nuova di teoria Markov: è un audit metodologico vicino ai test di ordine Markov con surrogate data. La letteratura rilevante include test esatti di ordine Markov e surrogate vincolati che preservano proprietà Markov di ordine n, quindi il prossimo strumento deve avvicinarsi a quei null esatti invece di usare solo generatori bin-pool approssimati. + +## Bicono della scoperta +- **Due radici**: layer osservato dai dati · ordine noto del generatore. +- **Singolare**: la pipeline Mk stessa, prima che venga usata come prova su un dominio reale. +- **Invariante di passaggio**: un claim di memoria sopravvive solo se recupera correttamente i controlli con ordine noto. +- **Campo di possibilità**: qui diventa possibile separare nucleo robusto e diagnostiche sovrastimate; qui diventa non-possibile usare `cond_entropy` o `num_var_10` come prova primaria di profondità senza recovery audit. + +## Self-Falsification — 5 lenti +- **L1 hard constraint vs bias**: nessun claim "zero/never/always". La frazione mod3 è 0.158032, dichiarata bias, non proibizione. +- **L2 quantità vs ratio**: il claim principale usa layer recuperato e z-score, non percentuali fra spazi di taglia diversa. +- **L3 no silent patching**: dichiaro esplicitamente che `HL_markov` era nome scorretto per un controllo AR(1) lag1-matched. +- **L4 edge cases**: i fallimenti `cond_entropy` e `num_var_10` non vengono arrotondati via; diventano il vincolo. +- **L5 rediscovery**: non taggo NEW. Riferimenti classici vicini: surrogate-data tests for Markov order e constrained Markov-order surrogates. + +## References +- Kandel et al.-lineage / surrogate tests: ["Testing the order of discrete Markov chains using surrogate data", Physica D 117, 1998](https://www.sciencedirect.com/science/article/pii/S0167278997003151). +- ["Exact significance test for Markov order", Physica D 269, 2014](https://www.sciencedirect.com/science/article/pii/S0167278913003230). +- ["Constrained Markov order surrogates", Physica D 2020](https://www.sciencedirect.com/science/article/pii/S0167278919305652). + +## Files +- `tools/exp_markov_layer_recovery_audit.py` +- `tools/data/markov_layer_recovery_audit.json` +- `tools/data/meta_tautology_test.json` +- `tools/data/seme.json` +- `tools/data/reports/agent_20260504_1219.md` diff --git a/tools/exp_markov_layer_recovery_audit.py b/tools/exp_markov_layer_recovery_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..4faa72c630148cdf498c3cc2d626cca2a78efd9a --- /dev/null +++ b/tools/exp_markov_layer_recovery_audit.py @@ -0,0 +1,179 @@ +#!/usr/bin/env python3 +""" +exp_markov_layer_recovery_audit.py - META audit for the Markov-layer classifier. + +Question: + If a sequence is generated with known Markov order k, does the existing + Mk0/Mk1/Mk2 classifier recover layer <= k? + +Why this matters: + Recent reports used Mk surrogate capture to claim two visible memory layers. + This audit tests the opposite: whether the classifier itself creates apparent + higher layers on finite Markov controls. + +Controls: + - prime_gaps: unknown real target, reported for comparison only + - empirical_Mk0: shuffled prime gaps; known order 0 + - empirical_Mk1: prime-trained Mk1 surrogate; known order 1 + - empirical_Mk2: prime-trained Mk2 surrogate; known order 2 + - poisson_iid: independent exponential spacings; known order 0 + +Verdict rule: + A control fails recovery when any observable is assigned layer > known order. +""" + +import argparse +import json +from pathlib import Path + +import numpy as np + +from exp_two_layer_universality import ( + OBSERVABLES, + classify_layer, + gen_prime_gaps, + generate_markov_surrogate, +) + + +def measure_all(gaps): + out = {} + for name, fn in OBSERVABLES.items(): + try: + value = float(fn(gaps)) + except Exception: + value = float("nan") + out[name] = value + return out + + +def classifier_pass(gaps, n_surr, rng): + real_obs = measure_all(gaps) + z_scores = {} + + for mk in (0, 1, 2): + surr_obs = {name: [] for name in OBSERVABLES} + for _ in range(n_surr): + if mk == 0: + surr = rng.permutation(gaps) + else: + surr = generate_markov_surrogate(gaps, mk, rng=rng) + for obs_name, obs_fn in OBSERVABLES.items(): + try: + surr_obs[obs_name].append(float(obs_fn(surr))) + except Exception: + pass + + z_mk = {} + for obs_name in OBSERVABLES: + vals = np.asarray(surr_obs[obs_name], dtype=float) + vals = vals[np.isfinite(vals)] + if len(vals) > 2 and np.std(vals) > 1e-12 and np.isfinite(real_obs[obs_name]): + z = (real_obs[obs_name] - np.mean(vals)) / np.std(vals) + else: + z = 0.0 + z_mk[obs_name] = round(float(z), 2) + z_scores[f"Mk{mk}"] = z_mk + + layers = {} + for obs_name in OBSERVABLES: + layers[obs_name] = classify_layer( + z_scores["Mk0"][obs_name], + z_scores["Mk1"][obs_name], + z_scores["Mk2"][obs_name], + ) + + return real_obs, z_scores, layers + + +def build_controls(prime_gaps, rng): + return { + "prime_gaps": { + "known_order": None, + "gaps": prime_gaps, + }, + "empirical_Mk0": { + "known_order": 0, + "gaps": rng.permutation(prime_gaps), + }, + "empirical_Mk1": { + "known_order": 1, + "gaps": generate_markov_surrogate(prime_gaps, 1, rng=rng), + }, + "empirical_Mk2": { + "known_order": 2, + "gaps": generate_markov_surrogate(prime_gaps, 2, rng=rng), + }, + "poisson_iid": { + "known_order": 0, + "gaps": rng.exponential(1.0, len(prime_gaps)), + }, + } + + +def summarize_failure(layers, known_order): + if known_order is None: + return { + "status": "target_only", + "over_layer_observables": [], + "max_layer": int(max(layers.values())), + } + + over = [name for name, layer in layers.items() if layer > known_order] + return { + "status": "PASS" if not over else "FAIL", + "over_layer_observables": over, + "max_layer": int(max(layers.values())), + } + + +def run(N=60000, n_surr=20, seed=20260504): + rng = np.random.default_rng(seed) + prime_gaps = gen_prime_gaps(N).astype(float) + controls = build_controls(prime_gaps, rng) + + results = { + "N": int(N), + "n_surr": int(n_surr), + "seed": int(seed), + "controls": {}, + } + + print(f"N={N}, n_surr={n_surr}, seed={seed}") + print(f"{'sequence':<16} {'known':<7} {'status':<8} {'maxL':<5} over-layer observables") + print("-" * 82) + + for name, spec in controls.items(): + real_obs, z_scores, layers = classifier_pass(spec["gaps"], n_surr, rng) + summary = summarize_failure(layers, spec["known_order"]) + known = "target" if spec["known_order"] is None else f"L{spec['known_order']}" + over = ", ".join(summary["over_layer_observables"]) or "-" + print(f"{name:<16} {known:<7} {summary['status']:<8} L{summary['max_layer']:<4} {over}") + + results["controls"][name] = { + "known_order": spec["known_order"], + "n_gaps": int(len(spec["gaps"])), + "real_obs": {k: round(v, 6) for k, v in real_obs.items()}, + "z_scores": z_scores, + "layers": layers, + "recovery": summary, + } + + out_path = Path("tools/data/markov_layer_recovery_audit.json") + with out_path.open("w") as f: + json.dump(results, f, indent=2) + print(f"\nSaved to {out_path}") + return results + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--N", type=int, default=60000) + parser.add_argument("--n_surr", type=int, default=20) + parser.add_argument("--seed", type=int, default=20260504) + args = parser.parse_args() + run(N=args.N, n_surr=args.n_surr, seed=args.seed) + + +if __name__ == "__main__": + main() exec /bin/bash -lc "ps -ef | rg 'lab_falsifier|codex'" in /opt/MM_D-ND succeeded in 0ms: root 1 0 0 12:28 ? 00:00:00 /root/.codex_lab/tmp/arg0/codex-arg07Sxyel/codex-linux-sandbox --sandbox-policy-cwd /opt/MM_D-ND --command-cwd /opt/MM_D-ND --sandbox-policy {"type":"workspace-write","writable_roots":["/root/.codex_lab/memories"],"network_access":false,"exclude_tmpdir_env_var":false,"exclude_slash_tmp":false} --file-system-sandbox-policy {"kind":"restricted","entries":[{"path":{"type":"special","value":{"kind":"root"}},"access":"read"},{"path":{"type":"special","value":{"kind":"current_working_directory"}},"access":"write"},{"path":{"type":"special","value":{"kind":"slash_tmp"}},"access":"write"},{"path":{"type":"special","value":{"kind":"tmpdir"}},"access":"write"},{"path":{"type":"path","path":"/root/.codex_lab/memories"},"access":"write"},{"path":{"type":"path","path":"/opt/MM_D-ND/.git"},"access":"read"},{"path":{"type":"path","path":"/opt/MM_D-ND/.codex"},"access":"read"}]} --network-sandbox-policy "restricted" -- /bin/bash -c __CODEX_SNAPSHOT_OVERRIDE_SET_0="${CODEX_THREAD_ID+x}" __CODEX_SNAPSHOT_OVERRIDE_0="${CODEX_THREAD_ID-}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_0="${ALL_PROXY+x}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_0="${ALL_PROXY-}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_1="${BUNDLE_HTTPS_PROXY+x}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_1="${BUNDLE_HTTPS_PROXY-}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_2="${BUNDLE_HTTP_PROXY+x}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_2="${BUNDLE_HTTP_PROXY-}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_3="${BUNDLE_NO_PROXY+x}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_3="${BUNDLE_NO_PROXY-}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_4="${CODEX_NETWORK_ALLOW_LOCAL_BINDING+x}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_4="${CODEX_NETWORK_ALLOW_LOCAL_BINDING-}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_5="${CODEX_NETWORK_PROXY_ACTIVE+x}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_5="${CODEX_NETWORK_PROXY_ACTIVE-}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_6="${DOCKER_HTTPS_PROXY+x}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_6="${DOCKER_HTTPS_PROXY-}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_7="${DOCKER_HTTP_PROXY+x}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_7="${DOCKER_HTTP_PROXY-}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_8="${ELECTRON_GET_USE_PROXY+x}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_8="${ELECTRON_GET_USE_PROXY-}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_9="${FTP_PROXY+x}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_9="${FTP_PROXY-}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_10="${HTTPS_PROXY+x}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_10="${HTTPS_PROXY-}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_11="${HTTP_PROXY+x}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_11="${HTTP_PROXY-}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_12="${NO_PROXY+x}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_12="${NO_PROXY-}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_13="${NPM_CONFIG_HTTPS_PROXY+x}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_13="${NPM_CONFIG_HTTPS_PROXY-}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_14="${NPM_CONFIG_HTTP_PROXY+x}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_14="${NPM_CONFIG_HTTP_PROXY-}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_15="${NPM_CONFIG_NOPROXY+x}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_15="${NPM_CONFIG_NOPROXY-}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_16="${NPM_CONFIG_PROXY+x}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_16="${NPM_CONFIG_PROXY-}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_17="${PIP_PROXY+x}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_17="${PIP_PROXY-}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_18="${WSS_PROXY+x}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_18="${WSS_PROXY-}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_19="${WS_PROXY+x}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_19="${WS_PROXY-}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_20="${YARN_HTTPS_PROXY+x}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_20="${YARN_HTTPS_PROXY-}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_21="${YARN_HTTP_PROXY+x}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_21="${YARN_HTTP_PROXY-}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_22="${YARN_NO_PROXY+x}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_22="${YARN_NO_PROXY-}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_23="${all_proxy+x}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_23="${all_proxy-}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_24="${ftp_proxy+x}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_24="${ftp_proxy-}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_25="${http_proxy+x}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_25="${http_proxy-}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_26="${https_proxy+x}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_26="${https_proxy-}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_27="${no_proxy+x}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_27="${no_proxy-}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_28="${npm_config_http_proxy+x}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_28="${npm_config_http_proxy-}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_29="${npm_config_https_proxy+x}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_29="${npm_config_https_proxy-}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_30="${npm_config_noproxy+x}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_30="${npm_config_noproxy-}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_31="${npm_config_proxy+x}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_31="${npm_config_proxy-}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_32="${ws_proxy+x}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_32="${ws_proxy-}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_33="${wss_proxy+x}" __CODEX_SNAPSHOT_PROXY_OVERRIDE_33="${wss_proxy-}" __CODEX_SNAPSHOT_PROXY_ENV_SET="${CODEX_NETWORK_PROXY_ACTIVE+x}" if . '/root/.codex_lab/shell_snapshots/019df2ed-937c-7f53-8077-0b26b5e707c2.1777897149320255931.sh' >/dev/null 2>&1; then :; fi if [ -n "${__CODEX_SNAPSHOT_OVERRIDE_SET_0}" ]; then export CODEX_THREAD_ID="${__CODEX_SNAPSHOT_OVERRIDE_0}"; else unset CODEX_THREAD_ID; fi if [ -n "$__CODEX_SNAPSHOT_PROXY_ENV_SET" ] || [ -n "${CODEX_NETWORK_PROXY_ACTIVE+x}" ]; then if [ -n "${__CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_0}" ]; then export ALL_PROXY="${__CODEX_SNAPSHOT_PROXY_OVERRIDE_0}"; else unset ALL_PROXY; fi if [ -n "${__CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_1}" ]; then export BUNDLE_HTTPS_PROXY="${__CODEX_SNAPSHOT_PROXY_OVERRIDE_1}"; else unset BUNDLE_HTTPS_PROXY; fi if [ -n "${__CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_2}" ]; then export BUNDLE_HTTP_PROXY="${__CODEX_SNAPSHOT_PROXY_OVERRIDE_2}"; else unset BUNDLE_HTTP_PROXY; fi if [ -n "${__CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_3}" ]; then export BUNDLE_NO_PROXY="${__CODEX_SNAPSHOT_PROXY_OVERRIDE_3}"; else unset BUNDLE_NO_PROXY; fi if [ -n "${__CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_4}" ]; then export CODEX_NETWORK_ALLOW_LOCAL_BINDING="${__CODEX_SNAPSHOT_PROXY_OVERRIDE_4}"; else unset CODEX_NETWORK_ALLOW_LOCAL_BINDING; fi if [ -n "${__CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_5}" ]; then export CODEX_NETWORK_PROXY_ACTIVE="${__CODEX_SNAPSHOT_PROXY_OVERRIDE_5}"; else unset CODEX_NETWORK_PROXY_ACTIVE; fi if [ -n "${__CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_6}" ]; then export DOCKER_HTTPS_PROXY="${__CODEX_SNAPSHOT_PROXY_OVERRIDE_6}"; else unset DOCKER_HTTPS_PROXY; fi if [ -n "${__CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_7}" ]; then export DOCKER_HTTP_PROXY="${__CODEX_SNAPSHOT_PROXY_OVERRIDE_7}"; else unset DOCKER_HTTP_PROXY; fi if [ -n "${__CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_8}" ]; then export ELECTRON_GET_USE_PROXY="${__CODEX_SNAPSHOT_PROXY_OVERRIDE_8}"; else unset ELECTRON_GET_USE_PROXY; fi if [ -n "${__CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_9}" ]; then export FTP_PROXY="${__CODEX_SNAPSHOT_PROXY_OVERRIDE_9}"; else unset FTP_PROXY; fi if [ -n "${__CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_10}" ]; then export HTTPS_PROXY="${__CODEX_SNAPSHOT_PROXY_OVERRIDE_10}"; else unset HTTPS_PROXY; fi if [ -n "${__CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_11}" ]; then export HTTP_PROXY="${__CODEX_SNAPSHOT_PROXY_OVERRIDE_11}"; else unset HTTP_PROXY; fi if [ -n "${__CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_12}" ]; then export NO_PROXY="${__CODEX_SNAPSHOT_PROXY_OVERRIDE_12}"; else unset NO_PROXY; fi if [ -n "${__CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_13}" ]; then export NPM_CONFIG_HTTPS_PROXY="${__CODEX_SNAPSHOT_PROXY_OVERRIDE_13}"; else unset NPM_CONFIG_HTTPS_PROXY; fi if [ -n "${__CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_14}" ]; then export NPM_CONFIG_HTTP_PROXY="${__CODEX_SNAPSHOT_PROXY_OVERRIDE_14}"; else unset NPM_CONFIG_HTTP_PROXY; fi if [ -n "${__CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_15}" ]; then export NPM_CONFIG_NOPROXY="${__CODEX_SNAPSHOT_PROXY_OVERRIDE_15}"; else unset NPM_CONFIG_NOPROXY; fi if [ -n "${__CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_16}" ]; then export NPM_CONFIG_PROXY="${__CODEX_SNAPSHOT_PROXY_OVERRIDE_16}"; else unset NPM_CONFIG_PROXY; fi if [ -n "${__CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_17}" ]; then export PIP_PROXY="${__CODEX_SNAPSHOT_PROXY_OVERRIDE_17}"; else unset PIP_PROXY; fi if [ -n "${__CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_18}" ]; then export WSS_PROXY="${__CODEX_SNAPSHOT_PROXY_OVERRIDE_18}"; else unset WSS_PROXY; fi if [ -n "${__CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_19}" ]; then export WS_PROXY="${__CODEX_SNAPSHOT_PROXY_OVERRIDE_19}"; else unset WS_PROXY; fi if [ -n "${__CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_20}" ]; then export YARN_HTTPS_PROXY="${__CODEX_SNAPSHOT_PROXY_OVERRIDE_20}"; else unset YARN_HTTPS_PROXY; fi if [ -n "${__CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_21}" ]; then export YARN_HTTP_PROXY="${__CODEX_SNAPSHOT_PROXY_OVERRIDE_21}"; else unset YARN_HTTP_PROXY; fi if [ -n "${__CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_22}" ]; then export YARN_NO_PROXY="${__CODEX_SNAPSHOT_PROXY_OVERRIDE_22}"; else unset YARN_NO_PROXY; fi if [ -n "${__CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_23}" ]; then export all_proxy="${__CODEX_SNAPSHOT_PROXY_OVERRIDE_23}"; else unset all_proxy; fi if [ -n "${__CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_24}" ]; then export ftp_proxy="${__CODEX_SNAPSHOT_PROXY_OVERRIDE_24}"; else unset ftp_proxy; fi if [ -n "${__CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_25}" ]; then export http_proxy="${__CODEX_SNAPSHOT_PROXY_OVERRIDE_25}"; else unset http_proxy; fi if [ -n "${__CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_26}" ]; then export https_proxy="${__CODEX_SNAPSHOT_PROXY_OVERRIDE_26}"; else unset https_proxy; fi if [ -n "${__CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_27}" ]; then export no_proxy="${__CODEX_SNAPSHOT_PROXY_OVERRIDE_27}"; else unset no_proxy; fi if [ -n "${__CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_28}" ]; then export npm_config_http_proxy="${__CODEX_SNAPSHOT_PROXY_OVERRIDE_28}"; else unset npm_config_http_proxy; fi if [ -n "${__CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_29}" ]; then export npm_config_https_proxy="${__CODEX_SNAPSHOT_PROXY_OVERRIDE_29}"; else unset npm_config_https_proxy; fi if [ -n "${__CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_30}" ]; then export npm_config_noproxy="${__CODEX_SNAPSHOT_PROXY_OVERRIDE_30}"; else unset npm_config_noproxy; fi if [ -n "${__CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_31}" ]; then export npm_config_proxy="${__CODEX_SNAPSHOT_PROXY_OVERRIDE_31}"; else unset npm_config_proxy; fi if [ -n "${__CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_32}" ]; then export ws_proxy="${__CODEX_SNAPSHOT_PROXY_OVERRIDE_32}"; else unset ws_proxy; fi if [ -n "${__CODEX_SNAPSHOT_PROXY_OVERRIDE_SET_33}" ]; then export wss_proxy="${__CODEX_SNAPSHOT_PROXY_OVERRIDE_33}"; else unset wss_proxy; fi fi exec '/bin/bash' -c 'ps -ef | rg '"'"'lab_falsifier|codex'"'"'' root 2 1 50 12:28 ? 00:00:00 /bin/bash -c ps -ef | rg 'lab_falsifier|codex' root 4 2 0 12:28 ? 00:00:00 rg lab_falsifier|codex diff --git a/tools/data/reports/agent_20260504_1219.md b/tools/data/reports/agent_20260504_1219.md new file mode 100644 index 0000000000000000000000000000000000000000..b3c10211f3b81595e76259db054509f891c79118 --- /dev/null +++ b/tools/data/reports/agent_20260504_1219.md @@ -0,0 +1,104 @@ +# Agent Report — Markov Layers Pass the First Recovery Gate, but Not All Observables Pass +**Date**: 2026-05-04 12:19 +**Piano**: 61 +**Tension explored**: META (0.5) + +## Claim Under Test +> Tutti i 11 test passano — verifica che non stiamo testando solo tautologie. + +## Question +Se la pipeline Mk0/Mk1/Mk2 vede due layer, li recupera anche quando l'ordine vero della sequenza è noto? + +## Experiment Design +- **Primary audit**: `tools/exp_markov_layer_recovery_audit.py` +- **Scope**: 60,000 gap/spacings per sequenza; 20 surrogati per livello Mk; seed `20260504`. +- **Target**: prime gaps, ordine sconosciuto, solo confronto. +- **Known-order controls**: + - `empirical_Mk0`: permutazione dei gap primi, ordine noto 0. + - `empirical_Mk1`: surrogate Mk1 addestrato sui gap primi, ordine noto 1. + - `empirical_Mk2`: surrogate Mk2 addestrato sui gap primi, ordine noto 2. + - `poisson_iid`: spacings esponenziali iid, ordine noto 0. +- **Recovery rule**: un controllo passa se nessun osservabile viene classificato con layer maggiore dell'ordine noto. +- **Secondary calibration**: `tools/exp_meta_tautology_test.py --n-primes 300000 --n-trials 12`. Nota: il controllo chiamato `HL_markov` nello script è un AR(1) gap model che preserva lag-1, non un modello Hardy-Littlewood aritmetico. + +## Results + +### Recovery audit + +| Sequence | Known order | Recovery | Max layer | Over-layer observables | +|---|---:|---|---:|---| +| `prime_gaps` | target | target only | 3 | - | +| `empirical_Mk0` | 0 | PASS | 0 | - | +| `empirical_Mk1` | 1 | PASS | 1 | - | +| `empirical_Mk2` | 2 | FAIL | 3 | `cond_entropy` | +| `poisson_iid` | 0 | FAIL | 1 | `num_var_10` | + +Prime target layers, reported without recovery verdict: + +| Observable | Layer | z vs Mk0 | z vs Mk1 | z vs Mk2 | +|---|---:|---:|---:|---:| +| SR | 1 | -22.21 | -0.08 | -0.16 | +| L1 | 1 | -12.25 | 0.79 | 0.86 | +| L2 | 2 | -3.60 | -4.09 | -0.19 | +| SR2 | 2 | -6.67 | -8.80 | -0.33 | +| cond_entropy | 3 | -435.40 | -26.11 | 2.56 | +| triple_var | 1 | -15.90 | -0.53 | 0.36 | +| num_var_10 | 1 | -7.46 | -1.42 | 0.00 | +| run_length | 3 | 27.59 | 7.19 | 3.43 | + +Control failure details: + +| Control | Observable | Expected max | Assigned | Diagnostic z | +|---|---|---:|---:|---:| +| `empirical_Mk2` | `cond_entropy` | 2 | 3 | z vs Mk2 = 3.15 | +| `poisson_iid` | `num_var_10` | 0 | 1 | z vs Mk0 = 2.13 | + +### Secondary tautology calibration + +| Observable | Verdict in tested controls | Blocking control | +|---|---|---| +| r-stat | STRUCTURAL | none in this run | +| mod3 fraction | STRUCTURAL | none in this run | +| lag1 ACF | TAUTOLOGICAL | lag1-matched AR(1), z = -0.62 | +| dr_ord | TAUTOLOGICAL | lag1-matched AR(1), z = -1.90 | +| dr_mag | STRUCTURAL | none in this run | + +Numeri principali: prime mod3 same-class fraction = 0.158032. Questo è bias forte rispetto ai controlli ~0.34-0.35, non proibizione zero. + +## Key Findings +1. **Il nucleo pair/triple non cade al primo audit.** `empirical_Mk0` recupera solo Layer 0; `empirical_Mk1` recupera solo Layer 1. Questo supporta l'uso di SR e L1 come Layer 1 e di L2/SR2 come Layer 2 nel perimetro testato. +2. **`cond_entropy` non può essere usata come prova forte di Layer 3 nel setup attuale.** Anche un controllo generato come Mk2 viene letto Layer 3 per `cond_entropy` (z vs Mk2 = 3.15). Questo indica bias del surrogate/binning o varianza finita, non necessariamente memoria reale oltre Mk2. +3. **`num_var_10` produce un falso positivo debole su iid.** `poisson_iid` viene letto Layer 1 per `num_var_10` con z = 2.13. Il segnale supera appena la soglia operativa, quindi va trattato come diagnostica secondaria finché non passa audit multi-seed. +4. **Il vecchio controllo `HL_markov` va rinominato nel linguaggio dei report.** Lo script non implementa Hardy-Littlewood; implementa un AR(1) lag1-matched. I claim futuri non devono chiamarlo controllo HL aritmetico. + +## Verdict +**CONSTRAINT on META + BOUNDARY**: la decomposizione two-layer è utilizzabile solo con perimetro atomico. + +Perimetro corretto: nei 60,000 gap primi testati con 20 surrogati Mk per livello, il nucleo `SR,L1` viene catturato da Mk1 e il nucleo `L2,SR2` da Mk2. Le diagnostiche `cond_entropy`, `num_var_10`, `run_length` non sono evidenza primaria finché non passano un recovery audit multi-seed e, idealmente, surrogati Markov esatti. + +Non è una scoperta nuova di teoria Markov: è un audit metodologico vicino ai test di ordine Markov con surrogate data. La letteratura rilevante include test esatti di ordine Markov e surrogate vincolati che preservano proprietà Markov di ordine n, quindi il prossimo strumento deve avvicinarsi a quei null esatti invece di usare solo generatori bin-pool approssimati. + +## Bicono della scoperta +- **Due radici**: layer osservato dai dati · ordine noto del generatore. +- **Singolare**: la pipeline Mk stessa, prima che venga usata come prova su un dominio reale. +- **Invariante di passaggio**: un claim di memoria sopravvive solo se recupera correttamente i controlli con ordine noto. +- **Campo di possibilità**: qui diventa possibile separare nucleo robusto e diagnostiche sovrastimate; qui diventa non-possibile usare `cond_entropy` o `num_var_10` come prova primaria di profondità senza recovery audit. + +## Self-Falsification — 5 lenti +- **L1 hard constraint vs bias**: nessun claim "zero/never/always". La frazione mod3 è 0.158032, dichiarata bias, non proibizione. +- **L2 quantità vs ratio**: il claim principale usa layer recuperato e z-score, non percentuali fra spazi di taglia diversa. +- **L3 no silent patching**: dichiaro esplicitamente che `HL_markov` era nome scorretto per un controllo AR(1) lag1-matched. +- **L4 edge cases**: i fallimenti `cond_entropy` e `num_var_10` non vengono arrotondati via; diventano il vincolo. +- **L5 rediscovery**: non taggo NEW. Riferimenti classici vicini: surrogate-data tests for Markov order e constrained Markov-order surrogates. + +## References +- Kandel et al.-lineage / surrogate tests: ["Testing the order of discrete Markov chains using surrogate data", Physica D 117, 1998](https://www.sciencedirect.com/science/article/pii/S0167278997003151). +- ["Exact significance test for Markov order", Physica D 269, 2014](https://www.sciencedirect.com/science/article/pii/S0167278913003230). +- ["Constrained Markov order surrogates", Physica D 2020](https://www.sciencedirect.com/science/article/pii/S0167278919305652). + +## Files +- `tools/exp_markov_layer_recovery_audit.py` +- `tools/data/markov_layer_recovery_audit.json` +- `tools/data/meta_tautology_test.json` +- `tools/data/seme.json` +- `tools/data/reports/agent_20260504_1219.md` diff --git a/tools/exp_markov_layer_recovery_audit.py b/tools/exp_markov_layer_recovery_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..4faa72c630148cdf498c3cc2d626cca2a78efd9a --- /dev/null +++ b/tools/exp_markov_layer_recovery_audit.py @@ -0,0 +1,179 @@ +#!/usr/bin/env python3 +""" +exp_markov_layer_recovery_audit.py - META audit for the Markov-layer classifier. + +Question: + If a sequence is generated with known Markov order k, does the existing + Mk0/Mk1/Mk2 classifier recover layer <= k? + +Why this matters: + Recent reports used Mk surrogate capture to claim two visible memory layers. + This audit tests the opposite: whether the classifier itself creates apparent + higher layers on finite Markov controls. + +Controls: + - prime_gaps: unknown real target, reported for comparison only + - empirical_Mk0: shuffled prime gaps; known order 0 + - empirical_Mk1: prime-trained Mk1 surrogate; known order 1 + - empirical_Mk2: prime-trained Mk2 surrogate; known order 2 + - poisson_iid: independent exponential spacings; known order 0 + +Verdict rule: + A control fails recovery when any observable is assigned layer > known order. +""" + +import argparse +import json +from pathlib import Path + +import numpy as np + +from exp_two_layer_universality import ( + OBSERVABLES, + classify_layer, + gen_prime_gaps, + generate_markov_surrogate, +) + + +def measure_all(gaps): + out = {} + for name, fn in OBSERVABLES.items(): + try: + value = float(fn(gaps)) + except Exception: + value = float("nan") + out[name] = value + return out + + +def classifier_pass(gaps, n_surr, rng): + real_obs = measure_all(gaps) + z_scores = {} + + for mk in (0, 1, 2): + surr_obs = {name: [] for name in OBSERVABLES} + for _ in range(n_surr): + if mk == 0: + surr = rng.permutation(gaps) + else: + surr = generate_markov_surrogate(gaps, mk, rng=rng) + for obs_name, obs_fn in OBSERVABLES.items(): + try: + surr_obs[obs_name].append(float(obs_fn(surr))) + except Exception: + pass + + z_mk = {} + for obs_name in OBSERVABLES: + vals = np.asarray(surr_obs[obs_name], dtype=float) + vals = vals[np.isfinite(vals)] + if len(vals) > 2 and np.std(vals) > 1e-12 and np.isfinite(real_obs[obs_name]): + z = (real_obs[obs_name] - np.mean(vals)) / np.std(vals) + else: + z = 0.0 + z_mk[obs_name] = round(float(z), 2) + z_scores[f"Mk{mk}"] = z_mk + + layers = {} + for obs_name in OBSERVABLES: + layers[obs_name] = classify_layer( + z_scores["Mk0"][obs_name], + z_scores["Mk1"][obs_name], + z_scores["Mk2"][obs_name], + ) + + return real_obs, z_scores, layers + + +def build_controls(prime_gaps, rng): + return { + "prime_gaps": { + "known_order": None, + "gaps": prime_gaps, + }, + "empirical_Mk0": { + "known_order": 0, + "gaps": rng.permutation(prime_gaps), + }, + "empirical_Mk1": { + "known_order": 1, + "gaps": generate_markov_surrogate(prime_gaps, 1, rng=rng), + }, + "empirical_Mk2": { + "known_order": 2, + "gaps": generate_markov_surrogate(prime_gaps, 2, rng=rng), + }, + "poisson_iid": { + "known_order": 0, + "gaps": rng.exponential(1.0, len(prime_gaps)), + }, + } + + +def summarize_failure(layers, known_order): + if known_order is None: + return { + "status": "target_only", + "over_layer_observables": [], + "max_layer": int(max(layers.values())), + } + + over = [name for name, layer in layers.items() if layer > known_order] + return { + "status": "PASS" if not over else "FAIL", + "over_layer_observables": over, + "max_layer": int(max(layers.values())), + } + + +def run(N=60000, n_surr=20, seed=20260504): + rng = np.random.default_rng(seed) + prime_gaps = gen_prime_gaps(N).astype(float) + controls = build_controls(prime_gaps, rng) + + results = { + "N": int(N), + "n_surr": int(n_surr), + "seed": int(seed), + "controls": {}, + } + + print(f"N={N}, n_surr={n_surr}, seed={seed}") + print(f"{'sequence':<16} {'known':<7} {'status':<8} {'maxL':<5} over-layer observables") + print("-" * 82) + + for name, spec in controls.items(): + real_obs, z_scores, layers = classifier_pass(spec["gaps"], n_surr, rng) + summary = summarize_failure(layers, spec["known_order"]) + known = "target" if spec["known_order"] is None else f"L{spec['known_order']}" + over = ", ".join(summary["over_layer_observables"]) or "-" + print(f"{name:<16} {known:<7} {summary['status']:<8} L{summary['max_layer']:<4} {over}") + + results["controls"][name] = { + "known_order": spec["known_order"], + "n_gaps": int(len(spec["gaps"])), + "real_obs": {k: round(v, 6) for k, v in real_obs.items()}, + "z_scores": z_scores, + "layers": layers, + "recovery": summary, + } + + out_path = Path("tools/data/markov_layer_recovery_audit.json") + with out_path.open("w") as f: + json.dump(results, f, indent=2) + print(f"\nSaved to {out_path}") + return results + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--N", type=int, default=60000) + parser.add_argument("--n_surr", type=int, default=20) + parser.add_argument("--seed", type=int, default=20260504) + args = parser.parse_args() + run(N=args.N, n_surr=args.n_surr, seed=args.seed) + + +if __name__ == "__main__": + main() diff --git a/tools/data/reports/agent_20260504_1219.md b/tools/data/reports/agent_20260504_1219.md new file mode 100644 index 0000000000000000000000000000000000000000..b3c10211f3b81595e76259db054509f891c79118 --- /dev/null +++ b/tools/data/reports/agent_20260504_1219.md @@ -0,0 +1,104 @@ +# Agent Report — Markov Layers Pass the First Recovery Gate, but Not All Observables Pass +**Date**: 2026-05-04 12:19 +**Piano**: 61 +**Tension explored**: META (0.5) + +## Claim Under Test +> Tutti i 11 test passano — verifica che non stiamo testando solo tautologie. + +## Question +Se la pipeline Mk0/Mk1/Mk2 vede due layer, li recupera anche quando l'ordine vero della sequenza è noto? + +## Experiment Design +- **Primary audit**: `tools/exp_markov_layer_recovery_audit.py` +- **Scope**: 60,000 gap/spacings per sequenza; 20 surrogati per livello Mk; seed `20260504`. +- **Target**: prime gaps, ordine sconosciuto, solo confronto. +- **Known-order controls**: + - `empirical_Mk0`: permutazione dei gap primi, ordine noto 0. + - `empirical_Mk1`: surrogate Mk1 addestrato sui gap primi, ordine noto 1. + - `empirical_Mk2`: surrogate Mk2 addestrato sui gap primi, ordine noto 2. + - `poisson_iid`: spacings esponenziali iid, ordine noto 0. +- **Recovery rule**: un controllo passa se nessun osservabile viene classificato con layer maggiore dell'ordine noto. +- **Secondary calibration**: `tools/exp_meta_tautology_test.py --n-primes 300000 --n-trials 12`. Nota: il controllo chiamato `HL_markov` nello script è un AR(1) gap model che preserva lag-1, non un modello Hardy-Littlewood aritmetico. + +## Results + +### Recovery audit + +| Sequence | Known order | Recovery | Max layer | Over-layer observables | +|---|---:|---|---:|---| +| `prime_gaps` | target | target only | 3 | - | +| `empirical_Mk0` | 0 | PASS | 0 | - | +| `empirical_Mk1` | 1 | PASS | 1 | - | +| `empirical_Mk2` | 2 | FAIL | 3 | `cond_entropy` | +| `poisson_iid` | 0 | FAIL | 1 | `num_var_10` | + +Prime target layers, reported without recovery verdict: + +| Observable | Layer | z vs Mk0 | z vs Mk1 | z vs Mk2 | +|---|---:|---:|---:|---:| +| SR | 1 | -22.21 | -0.08 | -0.16 | +| L1 | 1 | -12.25 | 0.79 | 0.86 | +| L2 | 2 | -3.60 | -4.09 | -0.19 | +| SR2 | 2 | -6.67 | -8.80 | -0.33 | +| cond_entropy | 3 | -435.40 | -26.11 | 2.56 | +| triple_var | 1 | -15.90 | -0.53 | 0.36 | +| num_var_10 | 1 | -7.46 | -1.42 | 0.00 | +| run_length | 3 | 27.59 | 7.19 | 3.43 | + +Control failure details: + +| Control | Observable | Expected max | Assigned | Diagnostic z | +|---|---|---:|---:|---:| +| `empirical_Mk2` | `cond_entropy` | 2 | 3 | z vs Mk2 = 3.15 | +| `poisson_iid` | `num_var_10` | 0 | 1 | z vs Mk0 = 2.13 | + +### Secondary tautology calibration + +| Observable | Verdict in tested controls | Blocking control | +|---|---|---| +| r-stat | STRUCTURAL | none in this run | +| mod3 fraction | STRUCTURAL | none in this run | +| lag1 ACF | TAUTOLOGICAL | lag1-matched AR(1), z = -0.62 | +| dr_ord | TAUTOLOGICAL | lag1-matched AR(1), z = -1.90 | +| dr_mag | STRUCTURAL | none in this run | + +Numeri principali: prime mod3 same-class fraction = 0.158032. Questo è bias forte rispetto ai controlli ~0.34-0.35, non proibizione zero. + +## Key Findings +1. **Il nucleo pair/triple non cade al primo audit.** `empirical_Mk0` recupera solo Layer 0; `empirical_Mk1` recupera solo Layer 1. Questo supporta l'uso di SR e L1 come Layer 1 e di L2/SR2 come Layer 2 nel perimetro testato. +2. **`cond_entropy` non può essere usata come prova forte di Layer 3 nel setup attuale.** Anche un controllo generato come Mk2 viene letto Layer 3 per `cond_entropy` (z vs Mk2 = 3.15). Questo indica bias del surrogate/binning o varianza finita, non necessariamente memoria reale oltre Mk2. +3. **`num_var_10` produce un falso positivo debole su iid.** `poisson_iid` viene letto Layer 1 per `num_var_10` con z = 2.13. Il segnale supera appena la soglia operativa, quindi va trattato come diagnostica secondaria finché non passa audit multi-seed. +4. **Il vecchio controllo `HL_markov` va rinominato nel linguaggio dei report.** Lo script non implementa Hardy-Littlewood; implementa un AR(1) lag1-matched. I claim futuri non devono chiamarlo controllo HL aritmetico. + +## Verdict +**CONSTRAINT on META + BOUNDARY**: la decomposizione two-layer è utilizzabile solo con perimetro atomico. + +Perimetro corretto: nei 60,000 gap primi testati con 20 surrogati Mk per livello, il nucleo `SR,L1` viene catturato da Mk1 e il nucleo `L2,SR2` da Mk2. Le diagnostiche `cond_entropy`, `num_var_10`, `run_length` non sono evidenza primaria finché non passano un recovery audit multi-seed e, idealmente, surrogati Markov esatti. + +Non è una scoperta nuova di teoria Markov: è un audit metodologico vicino ai test di ordine Markov con surrogate data. La letteratura rilevante include test esatti di ordine Markov e surrogate vincolati che preservano proprietà Markov di ordine n, quindi il prossimo strumento deve avvicinarsi a quei null esatti invece di usare solo generatori bin-pool approssimati. + +## Bicono della scoperta +- **Due radici**: layer osservato dai dati · ordine noto del generatore. +- **Singolare**: la pipeline Mk stessa, prima che venga usata come prova su un dominio reale. +- **Invariante di passaggio**: un claim di memoria sopravvive solo se recupera correttamente i controlli con ordine noto. +- **Campo di possibilità**: qui diventa possibile separare nucleo robusto e diagnostiche sovrastimate; qui diventa non-possibile usare `cond_entropy` o `num_var_10` come prova primaria di profondità senza recovery audit. + +## Self-Falsification — 5 lenti +- **L1 hard constraint vs bias**: nessun claim "zero/never/always". La frazione mod3 è 0.158032, dichiarata bias, non proibizione. +- **L2 quantità vs ratio**: il claim principale usa layer recuperato e z-score, non percentuali fra spazi di taglia diversa. +- **L3 no silent patching**: dichiaro esplicitamente che `HL_markov` era nome scorretto per un controllo AR(1) lag1-matched. +- **L4 edge cases**: i fallimenti `cond_entropy` e `num_var_10` non vengono arrotondati via; diventano il vincolo. +- **L5 rediscovery**: non taggo NEW. Riferimenti classici vicini: surrogate-data tests for Markov order e constrained Markov-order surrogates. + +## References +- Kandel et al.-lineage / surrogate tests: ["Testing the order of discrete Markov chains using surrogate data", Physica D 117, 1998](https://www.sciencedirect.com/science/article/pii/S0167278997003151). +- ["Exact significance test for Markov order", Physica D 269, 2014](https://www.sciencedirect.com/science/article/pii/S0167278913003230). +- ["Constrained Markov order surrogates", Physica D 2020](https://www.sciencedirect.com/science/article/pii/S0167278919305652). + +## Files +- `tools/exp_markov_layer_recovery_audit.py` +- `tools/data/markov_layer_recovery_audit.json` +- `tools/data/meta_tautology_test.json` +- `tools/data/seme.json` +- `tools/data/reports/agent_20260504_1219.md` diff --git a/tools/exp_markov_layer_recovery_audit.py b/tools/exp_markov_layer_recovery_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..4faa72c630148cdf498c3cc2d626cca2a78efd9a --- /dev/null +++ b/tools/exp_markov_layer_recovery_audit.py @@ -0,0 +1,179 @@ +#!/usr/bin/env python3 +""" +exp_markov_layer_recovery_audit.py - META audit for the Markov-layer classifier. + +Question: + If a sequence is generated with known Markov order k, does the existing + Mk0/Mk1/Mk2 classifier recover layer <= k? + +Why this matters: + Recent reports used Mk surrogate capture to claim two visible memory layers. + This audit tests the opposite: whether the classifier itself creates apparent + higher layers on finite Markov controls. + +Controls: + - prime_gaps: unknown real target, reported for comparison only + - empirical_Mk0: shuffled prime gaps; known order 0 + - empirical_Mk1: prime-trained Mk1 surrogate; known order 1 + - empirical_Mk2: prime-trained Mk2 surrogate; known order 2 + - poisson_iid: independent exponential spacings; known order 0 + +Verdict rule: + A control fails recovery when any observable is assigned layer > known order. +""" + +import argparse +import json +from pathlib import Path + +import numpy as np + +from exp_two_layer_universality import ( + OBSERVABLES, + classify_layer, + gen_prime_gaps, + generate_markov_surrogate, +) + + +def measure_all(gaps): + out = {} + for name, fn in OBSERVABLES.items(): + try: + value = float(fn(gaps)) + except Exception: + value = float("nan") + out[name] = value + return out + + +def classifier_pass(gaps, n_surr, rng): + real_obs = measure_all(gaps) + z_scores = {} + + for mk in (0, 1, 2): + surr_obs = {name: [] for name in OBSERVABLES} + for _ in range(n_surr): + if mk == 0: + surr = rng.permutation(gaps) + else: + surr = generate_markov_surrogate(gaps, mk, rng=rng) + for obs_name, obs_fn in OBSERVABLES.items(): + try: + surr_obs[obs_name].append(float(obs_fn(surr))) + except Exception: + pass + + z_mk = {} + for obs_name in OBSERVABLES: + vals = np.asarray(surr_obs[obs_name], dtype=float) + vals = vals[np.isfinite(vals)] + if len(vals) > 2 and np.std(vals) > 1e-12 and np.isfinite(real_obs[obs_name]): + z = (real_obs[obs_name] - np.mean(vals)) / np.std(vals) + else: + z = 0.0 + z_mk[obs_name] = round(float(z), 2) + z_scores[f"Mk{mk}"] = z_mk + + layers = {} + for obs_name in OBSERVABLES: + layers[obs_name] = classify_layer( + z_scores["Mk0"][obs_name], + z_scores["Mk1"][obs_name], + z_scores["Mk2"][obs_name], + ) + + return real_obs, z_scores, layers + + +def build_controls(prime_gaps, rng): + return { + "prime_gaps": { + "known_order": None, + "gaps": prime_gaps, + }, + "empirical_Mk0": { + "known_order": 0, + "gaps": rng.permutation(prime_gaps), + }, + "empirical_Mk1": { + "known_order": 1, + "gaps": generate_markov_surrogate(prime_gaps, 1, rng=rng), + }, + "empirical_Mk2": { + "known_order": 2, + "gaps": generate_markov_surrogate(prime_gaps, 2, rng=rng), + }, + "poisson_iid": { + "known_order": 0, + "gaps": rng.exponential(1.0, len(prime_gaps)), + }, + } + + +def summarize_failure(layers, known_order): + if known_order is None: + return { + "status": "target_only", + "over_layer_observables": [], + "max_layer": int(max(layers.values())), + } + + over = [name for name, layer in layers.items() if layer > known_order] + return { + "status": "PASS" if not over else "FAIL", + "over_layer_observables": over, + "max_layer": int(max(layers.values())), + } + + +def run(N=60000, n_surr=20, seed=20260504): + rng = np.random.default_rng(seed) + prime_gaps = gen_prime_gaps(N).astype(float) + controls = build_controls(prime_gaps, rng) + + results = { + "N": int(N), + "n_surr": int(n_surr), + "seed": int(seed), + "controls": {}, + } + + print(f"N={N}, n_surr={n_surr}, seed={seed}") + print(f"{'sequence':<16} {'known':<7} {'status':<8} {'maxL':<5} over-layer observables") + print("-" * 82) + + for name, spec in controls.items(): + real_obs, z_scores, layers = classifier_pass(spec["gaps"], n_surr, rng) + summary = summarize_failure(layers, spec["known_order"]) + known = "target" if spec["known_order"] is None else f"L{spec['known_order']}" + over = ", ".join(summary["over_layer_observables"]) or "-" + print(f"{name:<16} {known:<7} {summary['status']:<8} L{summary['max_layer']:<4} {over}") + + results["controls"][name] = { + "known_order": spec["known_order"], + "n_gaps": int(len(spec["gaps"])), + "real_obs": {k: round(v, 6) for k, v in real_obs.items()}, + "z_scores": z_scores, + "layers": layers, + "recovery": summary, + } + + out_path = Path("tools/data/markov_layer_recovery_audit.json") + with out_path.open("w") as f: + json.dump(results, f, indent=2) + print(f"\nSaved to {out_path}") + return results + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--N", type=int, default=60000) + parser.add_argument("--n_surr", type=int, default=20) + parser.add_argument("--seed", type=int, default=20260504) + args = parser.parse_args() + run(N=args.N, n_surr=args.n_surr, seed=args.seed) + + +if __name__ == "__main__": + main() exec /bin/bash -lc "ls -l tools/data/reports/falsifier_20260504_1219.json 2>/dev/null && sed -n '1,120p' tools/data/reports/falsifier_20260504_1219.json 2>/dev/null || true" in /opt/MM_D-ND succeeded in 0ms: diff --git a/tools/data/reports/agent_20260504_1219.md b/tools/data/reports/agent_20260504_1219.md new file mode 100644 index 0000000000000000000000000000000000000000..b3c10211f3b81595e76259db054509f891c79118 --- /dev/null +++ b/tools/data/reports/agent_20260504_1219.md @@ -0,0 +1,104 @@ +# Agent Report — Markov Layers Pass the First Recovery Gate, but Not All Observables Pass +**Date**: 2026-05-04 12:19 +**Piano**: 61 +**Tension explored**: META (0.5) + +## Claim Under Test +> Tutti i 11 test passano — verifica che non stiamo testando solo tautologie. + +## Question +Se la pipeline Mk0/Mk1/Mk2 vede due layer, li recupera anche quando l'ordine vero della sequenza è noto? + +## Experiment Design +- **Primary audit**: `tools/exp_markov_layer_recovery_audit.py` +- **Scope**: 60,000 gap/spacings per sequenza; 20 surrogati per livello Mk; seed `20260504`. +- **Target**: prime gaps, ordine sconosciuto, solo confronto. +- **Known-order controls**: + - `empirical_Mk0`: permutazione dei gap primi, ordine noto 0. + - `empirical_Mk1`: surrogate Mk1 addestrato sui gap primi, ordine noto 1. + - `empirical_Mk2`: surrogate Mk2 addestrato sui gap primi, ordine noto 2. + - `poisson_iid`: spacings esponenziali iid, ordine noto 0. +- **Recovery rule**: un controllo passa se nessun osservabile viene classificato con layer maggiore dell'ordine noto. +- **Secondary calibration**: `tools/exp_meta_tautology_test.py --n-primes 300000 --n-trials 12`. Nota: il controllo chiamato `HL_markov` nello script è un AR(1) gap model che preserva lag-1, non un modello Hardy-Littlewood aritmetico. + +## Results + +### Recovery audit + +| Sequence | Known order | Recovery | Max layer | Over-layer observables | +|---|---:|---|---:|---| +| `prime_gaps` | target | target only | 3 | - | +| `empirical_Mk0` | 0 | PASS | 0 | - | +| `empirical_Mk1` | 1 | PASS | 1 | - | +| `empirical_Mk2` | 2 | FAIL | 3 | `cond_entropy` | +| `poisson_iid` | 0 | FAIL | 1 | `num_var_10` | + +Prime target layers, reported without recovery verdict: + +| Observable | Layer | z vs Mk0 | z vs Mk1 | z vs Mk2 | +|---|---:|---:|---:|---:| +| SR | 1 | -22.21 | -0.08 | -0.16 | +| L1 | 1 | -12.25 | 0.79 | 0.86 | +| L2 | 2 | -3.60 | -4.09 | -0.19 | +| SR2 | 2 | -6.67 | -8.80 | -0.33 | +| cond_entropy | 3 | -435.40 | -26.11 | 2.56 | +| triple_var | 1 | -15.90 | -0.53 | 0.36 | +| num_var_10 | 1 | -7.46 | -1.42 | 0.00 | +| run_length | 3 | 27.59 | 7.19 | 3.43 | + +Control failure details: + +| Control | Observable | Expected max | Assigned | Diagnostic z | +|---|---|---:|---:|---:| +| `empirical_Mk2` | `cond_entropy` | 2 | 3 | z vs Mk2 = 3.15 | +| `poisson_iid` | `num_var_10` | 0 | 1 | z vs Mk0 = 2.13 | + +### Secondary tautology calibration + +| Observable | Verdict in tested controls | Blocking control | +|---|---|---| +| r-stat | STRUCTURAL | none in this run | +| mod3 fraction | STRUCTURAL | none in this run | +| lag1 ACF | TAUTOLOGICAL | lag1-matched AR(1), z = -0.62 | +| dr_ord | TAUTOLOGICAL | lag1-matched AR(1), z = -1.90 | +| dr_mag | STRUCTURAL | none in this run | + +Numeri principali: prime mod3 same-class fraction = 0.158032. Questo è bias forte rispetto ai controlli ~0.34-0.35, non proibizione zero. + +## Key Findings +1. **Il nucleo pair/triple non cade al primo audit.** `empirical_Mk0` recupera solo Layer 0; `empirical_Mk1` recupera solo Layer 1. Questo supporta l'uso di SR e L1 come Layer 1 e di L2/SR2 come Layer 2 nel perimetro testato. +2. **`cond_entropy` non può essere usata come prova forte di Layer 3 nel setup attuale.** Anche un controllo generato come Mk2 viene letto Layer 3 per `cond_entropy` (z vs Mk2 = 3.15). Questo indica bias del surrogate/binning o varianza finita, non necessariamente memoria reale oltre Mk2. +3. **`num_var_10` produce un falso positivo debole su iid.** `poisson_iid` viene letto Layer 1 per `num_var_10` con z = 2.13. Il segnale supera appena la soglia operativa, quindi va trattato come diagnostica secondaria finché non passa audit multi-seed. +4. **Il vecchio controllo `HL_markov` va rinominato nel linguaggio dei report.** Lo script non implementa Hardy-Littlewood; implementa un AR(1) lag1-matched. I claim futuri non devono chiamarlo controllo HL aritmetico. + +## Verdict +**CONSTRAINT on META + BOUNDARY**: la decomposizione two-layer è utilizzabile solo con perimetro atomico. + +Perimetro corretto: nei 60,000 gap primi testati con 20 surrogati Mk per livello, il nucleo `SR,L1` viene catturato da Mk1 e il nucleo `L2,SR2` da Mk2. Le diagnostiche `cond_entropy`, `num_var_10`, `run_length` non sono evidenza primaria finché non passano un recovery audit multi-seed e, idealmente, surrogati Markov esatti. + +Non è una scoperta nuova di teoria Markov: è un audit metodologico vicino ai test di ordine Markov con surrogate data. La letteratura rilevante include test esatti di ordine Markov e surrogate vincolati che preservano proprietà Markov di ordine n, quindi il prossimo strumento deve avvicinarsi a quei null esatti invece di usare solo generatori bin-pool approssimati. + +## Bicono della scoperta +- **Due radici**: layer osservato dai dati · ordine noto del generatore. +- **Singolare**: la pipeline Mk stessa, prima che venga usata come prova su un dominio reale. +- **Invariante di passaggio**: un claim di memoria sopravvive solo se recupera correttamente i controlli con ordine noto. +- **Campo di possibilità**: qui diventa possibile separare nucleo robusto e diagnostiche sovrastimate; qui diventa non-possibile usare `cond_entropy` o `num_var_10` come prova primaria di profondità senza recovery audit. + +## Self-Falsification — 5 lenti +- **L1 hard constraint vs bias**: nessun claim "zero/never/always". La frazione mod3 è 0.158032, dichiarata bias, non proibizione. +- **L2 quantità vs ratio**: il claim principale usa layer recuperato e z-score, non percentuali fra spazi di taglia diversa. +- **L3 no silent patching**: dichiaro esplicitamente che `HL_markov` era nome scorretto per un controllo AR(1) lag1-matched. +- **L4 edge cases**: i fallimenti `cond_entropy` e `num_var_10` non vengono arrotondati via; diventano il vincolo. +- **L5 rediscovery**: non taggo NEW. Riferimenti classici vicini: surrogate-data tests for Markov order e constrained Markov-order surrogates. + +## References +- Kandel et al.-lineage / surrogate tests: ["Testing the order of discrete Markov chains using surrogate data", Physica D 117, 1998](https://www.sciencedirect.com/science/article/pii/S0167278997003151). +- ["Exact significance test for Markov order", Physica D 269, 2014](https://www.sciencedirect.com/science/article/pii/S0167278913003230). +- ["Constrained Markov order surrogates", Physica D 2020](https://www.sciencedirect.com/science/article/pii/S0167278919305652). + +## Files +- `tools/exp_markov_layer_recovery_audit.py` +- `tools/data/markov_layer_recovery_audit.json` +- `tools/data/meta_tautology_test.json` +- `tools/data/seme.json` +- `tools/data/reports/agent_20260504_1219.md` diff --git a/tools/exp_markov_layer_recovery_audit.py b/tools/exp_markov_layer_recovery_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..4faa72c630148cdf498c3cc2d626cca2a78efd9a --- /dev/null +++ b/tools/exp_markov_layer_recovery_audit.py @@ -0,0 +1,179 @@ +#!/usr/bin/env python3 +""" +exp_markov_layer_recovery_audit.py - META audit for the Markov-layer classifier. + +Question: + If a sequence is generated with known Markov order k, does the existing + Mk0/Mk1/Mk2 classifier recover layer <= k? + +Why this matters: + Recent reports used Mk surrogate capture to claim two visible memory layers. + This audit tests the opposite: whether the classifier itself creates apparent + higher layers on finite Markov controls. + +Controls: + - prime_gaps: unknown real target, reported for comparison only + - empirical_Mk0: shuffled prime gaps; known order 0 + - empirical_Mk1: prime-trained Mk1 surrogate; known order 1 + - empirical_Mk2: prime-trained Mk2 surrogate; known order 2 + - poisson_iid: independent exponential spacings; known order 0 + +Verdict rule: + A control fails recovery when any observable is assigned layer > known order. +""" + +import argparse +import json +from pathlib import Path + +import numpy as np + +from exp_two_layer_universality import ( + OBSERVABLES, + classify_layer, + gen_prime_gaps, + generate_markov_surrogate, +) + + +def measure_all(gaps): + out = {} + for name, fn in OBSERVABLES.items(): + try: + value = float(fn(gaps)) + except Exception: + value = float("nan") + out[name] = value + return out + + +def classifier_pass(gaps, n_surr, rng): + real_obs = measure_all(gaps) + z_scores = {} + + for mk in (0, 1, 2): + surr_obs = {name: [] for name in OBSERVABLES} + for _ in range(n_surr): + if mk == 0: + surr = rng.permutation(gaps) + else: + surr = generate_markov_surrogate(gaps, mk, rng=rng) + for obs_name, obs_fn in OBSERVABLES.items(): + try: + surr_obs[obs_name].append(float(obs_fn(surr))) + except Exception: + pass + + z_mk = {} + for obs_name in OBSERVABLES: + vals = np.asarray(surr_obs[obs_name], dtype=float) + vals = vals[np.isfinite(vals)] + if len(vals) > 2 and np.std(vals) > 1e-12 and np.isfinite(real_obs[obs_name]): + z = (real_obs[obs_name] - np.mean(vals)) / np.std(vals) + else: + z = 0.0 + z_mk[obs_name] = round(float(z), 2) + z_scores[f"Mk{mk}"] = z_mk + + layers = {} + for obs_name in OBSERVABLES: + layers[obs_name] = classify_layer( + z_scores["Mk0"][obs_name], + z_scores["Mk1"][obs_name], + z_scores["Mk2"][obs_name], + ) + + return real_obs, z_scores, layers + + +def build_controls(prime_gaps, rng): + return { + "prime_gaps": { + "known_order": None, + "gaps": prime_gaps, + }, + "empirical_Mk0": { + "known_order": 0, + "gaps": rng.permutation(prime_gaps), + }, + "empirical_Mk1": { + "known_order": 1, + "gaps": generate_markov_surrogate(prime_gaps, 1, rng=rng), + }, + "empirical_Mk2": { + "known_order": 2, + "gaps": generate_markov_surrogate(prime_gaps, 2, rng=rng), + }, + "poisson_iid": { + "known_order": 0, + "gaps": rng.exponential(1.0, len(prime_gaps)), + }, + } + + +def summarize_failure(layers, known_order): + if known_order is None: + return { + "status": "target_only", + "over_layer_observables": [], + "max_layer": int(max(layers.values())), + } + + over = [name for name, layer in layers.items() if layer > known_order] + return { + "status": "PASS" if not over else "FAIL", + "over_layer_observables": over, + "max_layer": int(max(layers.values())), + } + + +def run(N=60000, n_surr=20, seed=20260504): + rng = np.random.default_rng(seed) + prime_gaps = gen_prime_gaps(N).astype(float) + controls = build_controls(prime_gaps, rng) + + results = { + "N": int(N), + "n_surr": int(n_surr), + "seed": int(seed), + "controls": {}, + } + + print(f"N={N}, n_surr={n_surr}, seed={seed}") + print(f"{'sequence':<16} {'known':<7} {'status':<8} {'maxL':<5} over-layer observables") + print("-" * 82) + + for name, spec in controls.items(): + real_obs, z_scores, layers = classifier_pass(spec["gaps"], n_surr, rng) + summary = summarize_failure(layers, spec["known_order"]) + known = "target" if spec["known_order"] is None else f"L{spec['known_order']}" + over = ", ".join(summary["over_layer_observables"]) or "-" + print(f"{name:<16} {known:<7} {summary['status']:<8} L{summary['max_layer']:<4} {over}") + + results["controls"][name] = { + "known_order": spec["known_order"], + "n_gaps": int(len(spec["gaps"])), + "real_obs": {k: round(v, 6) for k, v in real_obs.items()}, + "z_scores": z_scores, + "layers": layers, + "recovery": summary, + } + + out_path = Path("tools/data/markov_layer_recovery_audit.json") + with out_path.open("w") as f: + json.dump(results, f, indent=2) + print(f"\nSaved to {out_path}") + return results + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--N", type=int, default=60000) + parser.add_argument("--n_surr", type=int, default=20) + parser.add_argument("--seed", type=int, default=20260504) + args = parser.parse_args() + run(N=args.N, n_surr=args.n_surr, seed=args.seed) + + +if __name__ == "__main__": + main() exec /bin/bash -lc 'python tools/structural_check.py tools/exp_markov_layer_recovery_audit.py' in /opt/MM_D-ND succeeded in 0ms: Scansione 1 file... Nessun anti-pattern trovato. diff --git a/tools/data/reports/agent_20260504_1219.md b/tools/data/reports/agent_20260504_1219.md new file mode 100644 index 0000000000000000000000000000000000000000..b3c10211f3b81595e76259db054509f891c79118 --- /dev/null +++ b/tools/data/reports/agent_20260504_1219.md @@ -0,0 +1,104 @@ +# Agent Report — Markov Layers Pass the First Recovery Gate, but Not All Observables Pass +**Date**: 2026-05-04 12:19 +**Piano**: 61 +**Tension explored**: META (0.5) + +## Claim Under Test +> Tutti i 11 test passano — verifica che non stiamo testando solo tautologie. + +## Question +Se la pipeline Mk0/Mk1/Mk2 vede due layer, li recupera anche quando l'ordine vero della sequenza è noto? + +## Experiment Design +- **Primary audit**: `tools/exp_markov_layer_recovery_audit.py` +- **Scope**: 60,000 gap/spacings per sequenza; 20 surrogati per livello Mk; seed `20260504`. +- **Target**: prime gaps, ordine sconosciuto, solo confronto. +- **Known-order controls**: + - `empirical_Mk0`: permutazione dei gap primi, ordine noto 0. + - `empirical_Mk1`: surrogate Mk1 addestrato sui gap primi, ordine noto 1. + - `empirical_Mk2`: surrogate Mk2 addestrato sui gap primi, ordine noto 2. + - `poisson_iid`: spacings esponenziali iid, ordine noto 0. +- **Recovery rule**: un controllo passa se nessun osservabile viene classificato con layer maggiore dell'ordine noto. +- **Secondary calibration**: `tools/exp_meta_tautology_test.py --n-primes 300000 --n-trials 12`. Nota: il controllo chiamato `HL_markov` nello script è un AR(1) gap model che preserva lag-1, non un modello Hardy-Littlewood aritmetico. + +## Results + +### Recovery audit + +| Sequence | Known order | Recovery | Max layer | Over-layer observables | +|---|---:|---|---:|---| +| `prime_gaps` | target | target only | 3 | - | +| `empirical_Mk0` | 0 | PASS | 0 | - | +| `empirical_Mk1` | 1 | PASS | 1 | - | +| `empirical_Mk2` | 2 | FAIL | 3 | `cond_entropy` | +| `poisson_iid` | 0 | FAIL | 1 | `num_var_10` | + +Prime target layers, reported without recovery verdict: + +| Observable | Layer | z vs Mk0 | z vs Mk1 | z vs Mk2 | +|---|---:|---:|---:|---:| +| SR | 1 | -22.21 | -0.08 | -0.16 | +| L1 | 1 | -12.25 | 0.79 | 0.86 | +| L2 | 2 | -3.60 | -4.09 | -0.19 | +| SR2 | 2 | -6.67 | -8.80 | -0.33 | +| cond_entropy | 3 | -435.40 | -26.11 | 2.56 | +| triple_var | 1 | -15.90 | -0.53 | 0.36 | +| num_var_10 | 1 | -7.46 | -1.42 | 0.00 | +| run_length | 3 | 27.59 | 7.19 | 3.43 | + +Control failure details: + +| Control | Observable | Expected max | Assigned | Diagnostic z | +|---|---|---:|---:|---:| +| `empirical_Mk2` | `cond_entropy` | 2 | 3 | z vs Mk2 = 3.15 | +| `poisson_iid` | `num_var_10` | 0 | 1 | z vs Mk0 = 2.13 | + +### Secondary tautology calibration + +| Observable | Verdict in tested controls | Blocking control | +|---|---|---| +| r-stat | STRUCTURAL | none in this run | +| mod3 fraction | STRUCTURAL | none in this run | +| lag1 ACF | TAUTOLOGICAL | lag1-matched AR(1), z = -0.62 | +| dr_ord | TAUTOLOGICAL | lag1-matched AR(1), z = -1.90 | +| dr_mag | STRUCTURAL | none in this run | + +Numeri principali: prime mod3 same-class fraction = 0.158032. Questo è bias forte rispetto ai controlli ~0.34-0.35, non proibizione zero. + +## Key Findings +1. **Il nucleo pair/triple non cade al primo audit.** `empirical_Mk0` recupera solo Layer 0; `empirical_Mk1` recupera solo Layer 1. Questo supporta l'uso di SR e L1 come Layer 1 e di L2/SR2 come Layer 2 nel perimetro testato. +2. **`cond_entropy` non può essere usata come prova forte di Layer 3 nel setup attuale.** Anche un controllo generato come Mk2 viene letto Layer 3 per `cond_entropy` (z vs Mk2 = 3.15). Questo indica bias del surrogate/binning o varianza finita, non necessariamente memoria reale oltre Mk2. +3. **`num_var_10` produce un falso positivo debole su iid.** `poisson_iid` viene letto Layer 1 per `num_var_10` con z = 2.13. Il segnale supera appena la soglia operativa, quindi va trattato come diagnostica secondaria finché non passa audit multi-seed. +4. **Il vecchio controllo `HL_markov` va rinominato nel linguaggio dei report.** Lo script non implementa Hardy-Littlewood; implementa un AR(1) lag1-matched. I claim futuri non devono chiamarlo controllo HL aritmetico. + +## Verdict +**CONSTRAINT on META + BOUNDARY**: la decomposizione two-layer è utilizzabile solo con perimetro atomico. + +Perimetro corretto: nei 60,000 gap primi testati con 20 surrogati Mk per livello, il nucleo `SR,L1` viene catturato da Mk1 e il nucleo `L2,SR2` da Mk2. Le diagnostiche `cond_entropy`, `num_var_10`, `run_length` non sono evidenza primaria finché non passano un recovery audit multi-seed e, idealmente, surrogati Markov esatti. + +Non è una scoperta nuova di teoria Markov: è un audit metodologico vicino ai test di ordine Markov con surrogate data. La letteratura rilevante include test esatti di ordine Markov e surrogate vincolati che preservano proprietà Markov di ordine n, quindi il prossimo strumento deve avvicinarsi a quei null esatti invece di usare solo generatori bin-pool approssimati. + +## Bicono della scoperta +- **Due radici**: layer osservato dai dati · ordine noto del generatore. +- **Singolare**: la pipeline Mk stessa, prima che venga usata come prova su un dominio reale. +- **Invariante di passaggio**: un claim di memoria sopravvive solo se recupera correttamente i controlli con ordine noto. +- **Campo di possibilità**: qui diventa possibile separare nucleo robusto e diagnostiche sovrastimate; qui diventa non-possibile usare `cond_entropy` o `num_var_10` come prova primaria di profondità senza recovery audit. + +## Self-Falsification — 5 lenti +- **L1 hard constraint vs bias**: nessun claim "zero/never/always". La frazione mod3 è 0.158032, dichiarata bias, non proibizione. +- **L2 quantità vs ratio**: il claim principale usa layer recuperato e z-score, non percentuali fra spazi di taglia diversa. +- **L3 no silent patching**: dichiaro esplicitamente che `HL_markov` era nome scorretto per un controllo AR(1) lag1-matched. +- **L4 edge cases**: i fallimenti `cond_entropy` e `num_var_10` non vengono arrotondati via; diventano il vincolo. +- **L5 rediscovery**: non taggo NEW. Riferimenti classici vicini: surrogate-data tests for Markov order e constrained Markov-order surrogates. + +## References +- Kandel et al.-lineage / surrogate tests: ["Testing the order of discrete Markov chains using surrogate data", Physica D 117, 1998](https://www.sciencedirect.com/science/article/pii/S0167278997003151). +- ["Exact significance test for Markov order", Physica D 269, 2014](https://www.sciencedirect.com/science/article/pii/S0167278913003230). +- ["Constrained Markov order surrogates", Physica D 2020](https://www.sciencedirect.com/science/article/pii/S0167278919305652). + +## Files +- `tools/exp_markov_layer_recovery_audit.py` +- `tools/data/markov_layer_recovery_audit.json` +- `tools/data/meta_tautology_test.json` +- `tools/data/seme.json` +- `tools/data/reports/agent_20260504_1219.md` diff --git a/tools/exp_markov_layer_recovery_audit.py b/tools/exp_markov_layer_recovery_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..4faa72c630148cdf498c3cc2d626cca2a78efd9a --- /dev/null +++ b/tools/exp_markov_layer_recovery_audit.py @@ -0,0 +1,179 @@ +#!/usr/bin/env python3 +""" +exp_markov_layer_recovery_audit.py - META audit for the Markov-layer classifier. + +Question: + If a sequence is generated with known Markov order k, does the existing + Mk0/Mk1/Mk2 classifier recover layer <= k? + +Why this matters: + Recent reports used Mk surrogate capture to claim two visible memory layers. + This audit tests the opposite: whether the classifier itself creates apparent + higher layers on finite Markov controls. + +Controls: + - prime_gaps: unknown real target, reported for comparison only + - empirical_Mk0: shuffled prime gaps; known order 0 + - empirical_Mk1: prime-trained Mk1 surrogate; known order 1 + - empirical_Mk2: prime-trained Mk2 surrogate; known order 2 + - poisson_iid: independent exponential spacings; known order 0 + +Verdict rule: + A control fails recovery when any observable is assigned layer > known order. +""" + +import argparse +import json +from pathlib import Path + +import numpy as np + +from exp_two_layer_universality import ( + OBSERVABLES, + classify_layer, + gen_prime_gaps, + generate_markov_surrogate, +) + + +def measure_all(gaps): + out = {} + for name, fn in OBSERVABLES.items(): + try: + value = float(fn(gaps)) + except Exception: + value = float("nan") + out[name] = value + return out + + +def classifier_pass(gaps, n_surr, rng): + real_obs = measure_all(gaps) + z_scores = {} + + for mk in (0, 1, 2): + surr_obs = {name: [] for name in OBSERVABLES} + for _ in range(n_surr): + if mk == 0: + surr = rng.permutation(gaps) + else: + surr = generate_markov_surrogate(gaps, mk, rng=rng) + for obs_name, obs_fn in OBSERVABLES.items(): + try: + surr_obs[obs_name].append(float(obs_fn(surr))) + except Exception: + pass + + z_mk = {} + for obs_name in OBSERVABLES: + vals = np.asarray(surr_obs[obs_name], dtype=float) + vals = vals[np.isfinite(vals)] + if len(vals) > 2 and np.std(vals) > 1e-12 and np.isfinite(real_obs[obs_name]): + z = (real_obs[obs_name] - np.mean(vals)) / np.std(vals) + else: + z = 0.0 + z_mk[obs_name] = round(float(z), 2) + z_scores[f"Mk{mk}"] = z_mk + + layers = {} + for obs_name in OBSERVABLES: + layers[obs_name] = classify_layer( + z_scores["Mk0"][obs_name], + z_scores["Mk1"][obs_name], + z_scores["Mk2"][obs_name], + ) + + return real_obs, z_scores, layers + + +def build_controls(prime_gaps, rng): + return { + "prime_gaps": { + "known_order": None, + "gaps": prime_gaps, + }, + "empirical_Mk0": { + "known_order": 0, + "gaps": rng.permutation(prime_gaps), + }, + "empirical_Mk1": { + "known_order": 1, + "gaps": generate_markov_surrogate(prime_gaps, 1, rng=rng), + }, + "empirical_Mk2": { + "known_order": 2, + "gaps": generate_markov_surrogate(prime_gaps, 2, rng=rng), + }, + "poisson_iid": { + "known_order": 0, + "gaps": rng.exponential(1.0, len(prime_gaps)), + }, + } + + +def summarize_failure(layers, known_order): + if known_order is None: + return { + "status": "target_only", + "over_layer_observables": [], + "max_layer": int(max(layers.values())), + } + + over = [name for name, layer in layers.items() if layer > known_order] + return { + "status": "PASS" if not over else "FAIL", + "over_layer_observables": over, + "max_layer": int(max(layers.values())), + } + + +def run(N=60000, n_surr=20, seed=20260504): + rng = np.random.default_rng(seed) + prime_gaps = gen_prime_gaps(N).astype(float) + controls = build_controls(prime_gaps, rng) + + results = { + "N": int(N), + "n_surr": int(n_surr), + "seed": int(seed), + "controls": {}, + } + + print(f"N={N}, n_surr={n_surr}, seed={seed}") + print(f"{'sequence':<16} {'known':<7} {'status':<8} {'maxL':<5} over-layer observables") + print("-" * 82) + + for name, spec in controls.items(): + real_obs, z_scores, layers = classifier_pass(spec["gaps"], n_surr, rng) + summary = summarize_failure(layers, spec["known_order"]) + known = "target" if spec["known_order"] is None else f"L{spec['known_order']}" + over = ", ".join(summary["over_layer_observables"]) or "-" + print(f"{name:<16} {known:<7} {summary['status']:<8} L{summary['max_layer']:<4} {over}") + + results["controls"][name] = { + "known_order": spec["known_order"], + "n_gaps": int(len(spec["gaps"])), + "real_obs": {k: round(v, 6) for k, v in real_obs.items()}, + "z_scores": z_scores, + "layers": layers, + "recovery": summary, + } + + out_path = Path("tools/data/markov_layer_recovery_audit.json") + with out_path.open("w") as f: + json.dump(results, f, indent=2) + print(f"\nSaved to {out_path}") + return results + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--N", type=int, default=60000) + parser.add_argument("--n_surr", type=int, default=20) + parser.add_argument("--seed", type=int, default=20260504) + args = parser.parse_args() + run(N=args.N, n_surr=args.n_surr, seed=args.seed) + + +if __name__ == "__main__": + main() codex Eseguito l’esperimento META e scritto il report in [agent_20260504_1219.md](/opt/MM_D-ND/tools/data/reports/agent_20260504_1219.md). Ho aggiunto lo strumento riusabile [exp_markov_layer_recovery_audit.py](/opt/MM_D-ND/tools/exp_markov_layer_recovery_audit.py) e salvato i risultati in `tools/data/markov_layer_recovery_audit.json`. Risultante: il nucleo `SR,L1` / `L2,SR2` regge al primo recovery audit, ma `cond_entropy`, `num_var_10`, `run_length` vanno trattati come diagnostiche secondarie perché producono sovrastima su controlli noti. Aggiornato anche `tools/data/seme.json` con il vincolo `META_LAYER_RECOVERY_AUDIT`. Verifiche: - `python -m py_compile tools/exp_markov_layer_recovery_audit.py` - JSON validi per `seme.json` e `markov_layer_recovery_audit.json` - `structural_check.py`: nessun anti-pattern trovato - Il `lab_falsifier.py` non ha completato: la chiamata interna a Codex è rimasta bloccata senza produrre output. Fonti consultate per L5: - https://www.sciencedirect.com/science/article/pii/S0167278997003151 - https://www.sciencedirect.com/science/article/pii/S0167278913003230 - https://www.sciencedirect.com/science/article/pii/S0167278919305652 diff --git a/tools/data/reports/agent_20260504_1219.md b/tools/data/reports/agent_20260504_1219.md new file mode 100644 index 0000000000000000000000000000000000000000..b3c10211f3b81595e76259db054509f891c79118 --- /dev/null +++ b/tools/data/reports/agent_20260504_1219.md @@ -0,0 +1,104 @@ +# Agent Report — Markov Layers Pass the First Recovery Gate, but Not All Observables Pass +**Date**: 2026-05-04 12:19 +**Piano**: 61 +**Tension explored**: META (0.5) + +## Claim Under Test +> Tutti i 11 test passano — verifica che non stiamo testando solo tautologie. + +## Question +Se la pipeline Mk0/Mk1/Mk2 vede due layer, li recupera anche quando l'ordine vero della sequenza è noto? + +## Experiment Design +- **Primary audit**: `tools/exp_markov_layer_recovery_audit.py` +- **Scope**: 60,000 gap/spacings per sequenza; 20 surrogati per livello Mk; seed `20260504`. +- **Target**: prime gaps, ordine sconosciuto, solo confronto. +- **Known-order controls**: + - `empirical_Mk0`: permutazione dei gap primi, ordine noto 0. + - `empirical_Mk1`: surrogate Mk1 addestrato sui gap primi, ordine noto 1. + - `empirical_Mk2`: surrogate Mk2 addestrato sui gap primi, ordine noto 2. + - `poisson_iid`: spacings esponenziali iid, ordine noto 0. +- **Recovery rule**: un controllo passa se nessun osservabile viene classificato con layer maggiore dell'ordine noto. +- **Secondary calibration**: `tools/exp_meta_tautology_test.py --n-primes 300000 --n-trials 12`. Nota: il controllo chiamato `HL_markov` nello script è un AR(1) gap model che preserva lag-1, non un modello Hardy-Littlewood aritmetico. + +## Results + +### Recovery audit + +| Sequence | Known order | Recovery | Max layer | Over-layer observables | +|---|---:|---|---:|---| +| `prime_gaps` | target | target only | 3 | - | +| `empirical_Mk0` | 0 | PASS | 0 | - | +| `empirical_Mk1` | 1 | PASS | 1 | - | +| `empirical_Mk2` | 2 | FAIL | 3 | `cond_entropy` | +| `poisson_iid` | 0 | FAIL | 1 | `num_var_10` | + +Prime target layers, reported without recovery verdict: + +| Observable | Layer | z vs Mk0 | z vs Mk1 | z vs Mk2 | +|---|---:|---:|---:|---:| +| SR | 1 | -22.21 | -0.08 | -0.16 | +| L1 | 1 | -12.25 | 0.79 | 0.86 | +| L2 | 2 | -3.60 | -4.09 | -0.19 | +| SR2 | 2 | -6.67 | -8.80 | -0.33 | +| cond_entropy | 3 | -435.40 | -26.11 | 2.56 | +| triple_var | 1 | -15.90 | -0.53 | 0.36 | +| num_var_10 | 1 | -7.46 | -1.42 | 0.00 | +| run_length | 3 | 27.59 | 7.19 | 3.43 | + +Control failure details: + +| Control | Observable | Expected max | Assigned | Diagnostic z | +|---|---|---:|---:|---:| +| `empirical_Mk2` | `cond_entropy` | 2 | 3 | z vs Mk2 = 3.15 | +| `poisson_iid` | `num_var_10` | 0 | 1 | z vs Mk0 = 2.13 | + +### Secondary tautology calibration + +| Observable | Verdict in tested controls | Blocking control | +|---|---|---| +| r-stat | STRUCTURAL | none in this run | +| mod3 fraction | STRUCTURAL | none in this run | +| lag1 ACF | TAUTOLOGICAL | lag1-matched AR(1), z = -0.62 | +| dr_ord | TAUTOLOGICAL | lag1-matched AR(1), z = -1.90 | +| dr_mag | STRUCTURAL | none in this run | + +Numeri principali: prime mod3 same-class fraction = 0.158032. Questo è bias forte rispetto ai controlli ~0.34-0.35, non proibizione zero. + +## Key Findings +1. **Il nucleo pair/triple non cade al primo audit.** `empirical_Mk0` recupera solo Layer 0; `empirical_Mk1` recupera solo Layer 1. Questo supporta l'uso di SR e L1 come Layer 1 e di L2/SR2 come Layer 2 nel perimetro testato. +2. **`cond_entropy` non può essere usata come prova forte di Layer 3 nel setup attuale.** Anche un controllo generato come Mk2 viene letto Layer 3 per `cond_entropy` (z vs Mk2 = 3.15). Questo indica bias del surrogate/binning o varianza finita, non necessariamente memoria reale oltre Mk2. +3. **`num_var_10` produce un falso positivo debole su iid.** `poisson_iid` viene letto Layer 1 per `num_var_10` con z = 2.13. Il segnale supera appena la soglia operativa, quindi va trattato come diagnostica secondaria finché non passa audit multi-seed. +4. **Il vecchio controllo `HL_markov` va rinominato nel linguaggio dei report.** Lo script non implementa Hardy-Littlewood; implementa un AR(1) lag1-matched. I claim futuri non devono chiamarlo controllo HL aritmetico. + +## Verdict +**CONSTRAINT on META + BOUNDARY**: la decomposizione two-layer è utilizzabile solo con perimetro atomico. + +Perimetro corretto: nei 60,000 gap primi testati con 20 surrogati Mk per livello, il nucleo `SR,L1` viene catturato da Mk1 e il nucleo `L2,SR2` da Mk2. Le diagnostiche `cond_entropy`, `num_var_10`, `run_length` non sono evidenza primaria finché non passano un recovery audit multi-seed e, idealmente, surrogati Markov esatti. + +Non è una scoperta nuova di teoria Markov: è un audit metodologico vicino ai test di ordine Markov con surrogate data. La letteratura rilevante include test esatti di ordine Markov e surrogate vincolati che preservano proprietà Markov di ordine n, quindi il prossimo strumento deve avvicinarsi a quei null esatti invece di usare solo generatori bin-pool approssimati. + +## Bicono della scoperta +- **Due radici**: layer osservato dai dati · ordine noto del generatore. +- **Singolare**: la pipeline Mk stessa, prima che venga usata come prova su un dominio reale. +- **Invariante di passaggio**: un claim di memoria sopravvive solo se recupera correttamente i controlli con ordine noto. +- **Campo di possibilità**: qui diventa possibile separare nucleo robusto e diagnostiche sovrastimate; qui diventa non-possibile usare `cond_entropy` o `num_var_10` come prova primaria di profondità senza recovery audit. + +## Self-Falsification — 5 lenti +- **L1 hard constraint vs bias**: nessun claim "zero/never/always". La frazione mod3 è 0.158032, dichiarata bias, non proibizione. +- **L2 quantità vs ratio**: il claim principale usa layer recuperato e z-score, non percentuali fra spazi di taglia diversa. +- **L3 no silent patching**: dichiaro esplicitamente che `HL_markov` era nome scorretto per un controllo AR(1) lag1-matched. +- **L4 edge cases**: i fallimenti `cond_entropy` e `num_var_10` non vengono arrotondati via; diventano il vincolo. +- **L5 rediscovery**: non taggo NEW. Riferimenti classici vicini: surrogate-data tests for Markov order e constrained Markov-order surrogates. + +## References +- Kandel et al.-lineage / surrogate tests: ["Testing the order of discrete Markov chains using surrogate data", Physica D 117, 1998](https://www.sciencedirect.com/science/article/pii/S0167278997003151). +- ["Exact significance test for Markov order", Physica D 269, 2014](https://www.sciencedirect.com/science/article/pii/S0167278913003230). +- ["Constrained Markov order surrogates", Physica D 2020](https://www.sciencedirect.com/science/article/pii/S0167278919305652). + +## Files +- `tools/exp_markov_layer_recovery_audit.py` +- `tools/data/markov_layer_recovery_audit.json` +- `tools/data/meta_tautology_test.json` +- `tools/data/seme.json` +- `tools/data/reports/agent_20260504_1219.md` diff --git a/tools/exp_markov_layer_recovery_audit.py b/tools/exp_markov_layer_recovery_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..4faa72c630148cdf498c3cc2d626cca2a78efd9a --- /dev/null +++ b/tools/exp_markov_layer_recovery_audit.py @@ -0,0 +1,179 @@ +#!/usr/bin/env python3 +""" +exp_markov_layer_recovery_audit.py - META audit for the Markov-layer classifier. + +Question: + If a sequence is generated with known Markov order k, does the existing + Mk0/Mk1/Mk2 classifier recover layer <= k? + +Why this matters: + Recent reports used Mk surrogate capture to claim two visible memory layers. + This audit tests the opposite: whether the classifier itself creates apparent + higher layers on finite Markov controls. + +Controls: + - prime_gaps: unknown real target, reported for comparison only + - empirical_Mk0: shuffled prime gaps; known order 0 + - empirical_Mk1: prime-trained Mk1 surrogate; known order 1 + - empirical_Mk2: prime-trained Mk2 surrogate; known order 2 + - poisson_iid: independent exponential spacings; known order 0 + +Verdict rule: + A control fails recovery when any observable is assigned layer > known order. +""" + +import argparse +import json +from pathlib import Path + +import numpy as np + +from exp_two_layer_universality import ( + OBSERVABLES, + classify_layer, + gen_prime_gaps, + generate_markov_surrogate, +) + + +def measure_all(gaps): + out = {} + for name, fn in OBSERVABLES.items(): + try: + value = float(fn(gaps)) + except Exception: + value = float("nan") + out[name] = value + return out + + +def classifier_pass(gaps, n_surr, rng): + real_obs = measure_all(gaps) + z_scores = {} + + for mk in (0, 1, 2): + surr_obs = {name: [] for name in OBSERVABLES} + for _ in range(n_surr): + if mk == 0: + surr = rng.permutation(gaps) + else: + surr = generate_markov_surrogate(gaps, mk, rng=rng) + for obs_name, obs_fn in OBSERVABLES.items(): + try: + surr_obs[obs_name].append(float(obs_fn(surr))) + except Exception: + pass + + z_mk = {} + for obs_name in OBSERVABLES: + vals = np.asarray(surr_obs[obs_name], dtype=float) + vals = vals[np.isfinite(vals)] + if len(vals) > 2 and np.std(vals) > 1e-12 and np.isfinite(real_obs[obs_name]): + z = (real_obs[obs_name] - np.mean(vals)) / np.std(vals) + else: + z = 0.0 + z_mk[obs_name] = round(float(z), 2) + z_scores[f"Mk{mk}"] = z_mk + + layers = {} + for obs_name in OBSERVABLES: + layers[obs_name] = classify_layer( + z_scores["Mk0"][obs_name], + z_scores["Mk1"][obs_name], + z_scores["Mk2"][obs_name], + ) + + return real_obs, z_scores, layers + + +def build_controls(prime_gaps, rng): + return { + "prime_gaps": { + "known_order": None, + "gaps": prime_gaps, + }, + "empirical_Mk0": { + "known_order": 0, + "gaps": rng.permutation(prime_gaps), + }, + "empirical_Mk1": { + "known_order": 1, + "gaps": generate_markov_surrogate(prime_gaps, 1, rng=rng), + }, + "empirical_Mk2": { + "known_order": 2, + "gaps": generate_markov_surrogate(prime_gaps, 2, rng=rng), + }, + "poisson_iid": { + "known_order": 0, + "gaps": rng.exponential(1.0, len(prime_gaps)), + }, + } + + +def summarize_failure(layers, known_order): + if known_order is None: + return { + "status": "target_only", + "over_layer_observables": [], + "max_layer": int(max(layers.values())), + } + + over = [name for name, layer in layers.items() if layer > known_order] + return { + "status": "PASS" if not over else "FAIL", + "over_layer_observables": over, + "max_layer": int(max(layers.values())), + } + + +def run(N=60000, n_surr=20, seed=20260504): + rng = np.random.default_rng(seed) + prime_gaps = gen_prime_gaps(N).astype(float) + controls = build_controls(prime_gaps, rng) + + results = { + "N": int(N), + "n_surr": int(n_surr), + "seed": int(seed), + "controls": {}, + } + + print(f"N={N}, n_surr={n_surr}, seed={seed}") + print(f"{'sequence':<16} {'known':<7} {'status':<8} {'maxL':<5} over-layer observables") + print("-" * 82) + + for name, spec in controls.items(): + real_obs, z_scores, layers = classifier_pass(spec["gaps"], n_surr, rng) + summary = summarize_failure(layers, spec["known_order"]) + known = "target" if spec["known_order"] is None else f"L{spec['known_order']}" + over = ", ".join(summary["over_layer_observables"]) or "-" + print(f"{name:<16} {known:<7} {summary['status']:<8} L{summary['max_layer']:<4} {over}") + + results["controls"][name] = { + "known_order": spec["known_order"], + "n_gaps": int(len(spec["gaps"])), + "real_obs": {k: round(v, 6) for k, v in real_obs.items()}, + "z_scores": z_scores, + "layers": layers, + "recovery": summary, + } + + out_path = Path("tools/data/markov_layer_recovery_audit.json") + with out_path.open("w") as f: + json.dump(results, f, indent=2) + print(f"\nSaved to {out_path}") + return results + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--N", type=int, default=60000) + parser.add_argument("--n_surr", type=int, default=20) + parser.add_argument("--seed", type=int, default=20260504) + args = parser.parse_args() + run(N=args.N, n_surr=args.n_surr, seed=args.seed) + + +if __name__ == "__main__": + main() tokens used 96,466 Eseguito l’esperimento META e scritto il report in [agent_20260504_1219.md](/opt/MM_D-ND/tools/data/reports/agent_20260504_1219.md). Ho aggiunto lo strumento riusabile [exp_markov_layer_recovery_audit.py](/opt/MM_D-ND/tools/exp_markov_layer_recovery_audit.py) e salvato i risultati in `tools/data/markov_layer_recovery_audit.json`. Risultante: il nucleo `SR,L1` / `L2,SR2` regge al primo recovery audit, ma `cond_entropy`, `num_var_10`, `run_length` vanno trattati come diagnostiche secondarie perché producono sovrastima su controlli noti. Aggiornato anche `tools/data/seme.json` con il vincolo `META_LAYER_RECOVERY_AUDIT`. Verifiche: - `python -m py_compile tools/exp_markov_layer_recovery_audit.py` - JSON validi per `seme.json` e `markov_layer_recovery_audit.json` - `structural_check.py`: nessun anti-pattern trovato - Il `lab_falsifier.py` non ha completato: la chiamata interna a Codex è rimasta bloccata senza produrre output. Fonti consultate per L5: - https://www.sciencedirect.com/science/article/pii/S0167278997003151 - https://www.sciencedirect.com/science/article/pii/S0167278913003230 - https://www.sciencedirect.com/science/article/pii/S0167278919305652